Auto Scaling in AWS Glue interactive sessions is now generally available. AWS Glue interactive sessions with Glue versions 3.0 or higher can now dynamically scale resources up and down based on the workload. With Auto Scaling, you no longer need to worry about over-provisioning resources for sessions, spend time optimizing the number of workers, or pay for idle workers.
AWS Glue is a serverless data integration service that allows you to schedule and run data integration and extract, transform, and load (ETL) jobs or sessions without managing any computing infrastructure. AWS Glue allows users to configure the number of works and type of workers to utilize. AWS Glue Auto Scaling monitors each stage of the session run and turns workers off when they are idle or adds workers if additional parallel processing is possible. This simplifies the process of tuning resources and optimizing costs.
This feature is now available in all commercial AWS Regions, GovCloud (US-West), and China Regions where AWS Glue interactive sessions is available.
For more details, please refer to the Glue Auto Scaling blog post and visit our documentation.
Source:: Amazon AWS