Search transformed S3 data by date.
Write year, month, and day partitions so Athena can avoid reading unrelated files.
Which processing approach is most appropriate?
Create date partition columns in a Glue ETL job and write the result to partitioned S3 prefixes.
Detailed explanation
The date columns become partition keys and allow Athena partition pruning.
The date columns become partition keys and allow Athena partition pruning.
A single object prevents date pruning and creates size and concurrency problems.
A single object prevents date pruning and creates size and concurrency problems.
Versioning keeps generations of one key and does not create date prefixes.
Versioning keeps generations of one key and does not create date prefixes.
Logs Insights queries CloudWatch Logs and does not perform S3 ETL placement.
Logs Insights queries CloudWatch Logs and does not perform S3 ETL placement.
Check official documentation
Compare the following points with the official documentation.
AWS GlueのS3パーティション出力とAthenaのパーティションプルーニングを確認する。
What to confirm
S3のバージョン管理と分析用パーティションの違いを説明できる。
Key points
- S3 partition
- Glue ETL
- Athena
- Partition pruning
Notes
- Environment: AWS公式DEA-C01試験ガイドと各サービスの公式ドキュメントの確認
- Confirm current AWS service behavior in the official DEA-C01 documentation.
Foundation review
Partition granularity
Choose keys and granularity that improve pruning without creating excessive small files.
Catalog metadata
Register or update partitions so query engines recognize newly written data.