Starting today, Amazon EMR on EC2 offers improved reliability and cluster launch experience for instance fleet clusters through enhanced subnet selection. With this feature, EMR on EC2 reduces cluster launch failures caused due to IP address shortages.
Amazon EMR is a cloud big data platform for data processing, interactive analysis, and machine learning using open-source frameworks such as Apache Spark, Apache Hive, and Presto. Previously, the subnet selection for EMR clusters only considered the available IP addresses for the core instance fleet. Amazon EMR now employs subnet filtering at cluster launch and selects one of the subnets that have adequate available IP addresses to successfully launch all instance fleets. If EMR cannot find a subnet with sufficient IP addresses to launch the whole cluster, it will prioritize the subnet that can at least launch the core and primary instance fleets. In this scenario, EMR will also publish a CloudWatch warning event to notify the user. If none of the configured subnets can be used to provision core and primary fleet, EMR will fail the cluster launch and provide a critical error event. These CloudWatch events enables you to monitor your clusters and take remedial actions as necessary.
Customers will benefit from this feature on all EMR 5.12.1 and later releases when launching EMR instance fleet clusters using allocation strategies. No further action is needed from your end. This capability is available in all AWS Regions, including the AWS GovCloud (US) Regions, where Amazon EMR on EC2 is available. To learn more, please refer to the documentation here.
Source:: Amazon AWS