A table has millions of partitions queried by customer ID and date.
Avoid retrieving and filtering the complete partition list for every metadata lookup.
Which feature is most appropriate?
Configure a Glue Data Catalog Partition Index for the partition-key predicates.
Detailed explanation
An index narrows catalog partition searches using the partition-key conditions.
An index narrows catalog partition searches using the partition-key conditions.
More frequent registration does not make a lookup over existing partitions efficient.
More frequent registration does not make a lookup over existing partitions efficient.
IAM defines authorization and does not create a catalog index.
IAM defines authorization and does not create a catalog index.
Object history does not determine catalog partition-search speed.
Object history does not determine catalog partition-search speed.
Check official documentation
Compare the following points with the official documentation.
AWS Glue Data Catalog Partition Index、パーティションフィルタリングの公式説明を確認する。
What to confirm
Catalog検索インデックスとCrawler更新・S3版管理を区別できる。
Key points
- Partition Index
- Partition filtering
- Catalog
- Metadata search
Notes
- Environment: AWS公式DEA-C01試験ガイドと各サービスの公式ドキュメントの確認
- Confirm current AWS service behavior in the official DEA-C01 documentation.
Foundation review
Registration versus lookup
Crawler frequency affects freshness; a partition index affects how quickly existing metadata is searched.
Filter selectivity
Useful partition predicates reduce the amount of metadata returned to the query engine.