Skip to content

AWS Glue Crawlers support incremental Amazon S3 crawling on existing AWS Glue Data Catalog tables

AWS Glue includes crawlers based on Amazon S3 Event Notifications, a capability that make discovering datasets simpler by scanning only data based on events in Amazon S3. The Glue crawler extracts the data schema and automatically populates the AWS Glue Data Catalog, which keeps the metadata current. By crawling datasets based on S3 events, this reduces the time to insight by making newly ingested data quickly available for analysis with your favorite analytics and machine learning tools.

Source:: Amazon AWS