Parquet data in S3 is queried frequently with Athena.
Balance compression ratio, read and write speed, and analytical scan cost.
Which compression is commonly appropriate for this workload?
Consider Snappy compression for a practical balance between compression and processing speed.
Detailed explanation
Snappy commonly balances compression ratio and fast compression or decompression for analytics.
Snappy commonly balances compression ratio and fast compression or decompression for analytics.
KMS manages encryption keys and is not a compression method.
KMS manages encryption keys and is not a compression method.
Uncompressed files increase bytes stored, scanned, and transferred.
Uncompressed files increase bytes stored, scanned, and transferred.
Whole-file compression can reduce the benefits of column-level reads and distributed processing.
Whole-file compression can reduce the benefits of column-level reads and distributed processing.
Check official documentation
Compare the following points with the official documentation.
AthenaとParquetの圧縮形式、列指向読み取り、スキャン料金の公式説明を確認する。
What to confirm
暗号化と圧縮を区別し、ワークロードに応じた圧縮選定を説明できる。
Key points
- Parquet
- Snappy
- Compression
- Scan cost
Notes
- Environment: AWS公式DEA-C01試験ガイドと各サービスの公式ドキュメントの確認
- Confirm current AWS service behavior in the official DEA-C01 documentation.
Foundation review
Compression trade-off
Higher compression can increase CPU work, so measure bytes scanned and elapsed time together.
Column reads
Column pruning is effective when the format and file layout support reading only requested columns.