New files arrive daily in an S3 data lake.
Catalog schemas and partitions so Athena can analyze the data.
Which feature is most appropriate?
Use an AWS Glue Crawler to scan the data and register tables, schemas, and partitions in the Data Catalog.
Detailed explanation
A crawler inspects data and creates table, schema, and partition metadata in the catalog.
A crawler inspects data and creates table, schema, and partition metadata in the catalog.
A bucket name does not describe object content or schema.
A bucket name does not describe object content or schema.
An IAM role name is not a data catalog.
An IAM role name is not a data catalog.
DNS queries are unrelated to data-lake schema discovery.
DNS queries are unrelated to data-lake schema discovery.
Check official documentation
Compare the following points with the official documentation.
AWS Glue CrawlerとData Catalogの公式説明を確認する。
What to confirm
データ検出・カタログ化とETL変換を区別できる。
Key points
- AWS Glue Crawler
- Data Catalog
- Schema
Notes
- Environment: AWS公式Glueドキュメントの確認
- Confirm current AWS service behavior in the official documentation.
Foundation review
Discovery and transformation
A crawler catalogs existing structure; an ETL job transforms or cleans the data.
Partition
A data layout that groups records by a key so queries can scan less data.