CSV files collected in S3 should be converted to columnar Parquet to reduce Athena scan volume.
Run the transformation as a managed service and manage data-lake metadata.
Choose TWO appropriate design elements.
AWS Glue ETL can convert CSV to Parquet, while Glue Data Catalog stores schemas and locations used by data-lake tools.
Detailed explanation
IAM grants permissions and does not perform data transformation.
IAM grants permissions and does not perform data transformation.
Glue provides managed ETL processing and can automate the format conversion.
Glue provides managed ETL processing and can automate the format conversion.
Route 53 performs DNS routing and does not transform or generate data files.
Route 53 performs DNS routing and does not transform or generate data files.
The Data Catalog stores schemas, locations, and other metadata for data-lake datasets.
The Data Catalog stores schemas, locations, and other metadata for data-lake datasets.
Check official documentation
Compare the following points with the official documentation.
AWS Glue ETLとData Catalogの公式説明を確認する。
What to confirm
取り込み後の形式変換とメタデータ管理をGlueの機能へ対応付けられる。
Key points
- AWS Glue ETL
- Apache Parquet
- Glue Data Catalog
Notes
- Environment: AWS公式Glueドキュメントの確認
- Confirm current AWS service behavior and architecture guidance in the official documentation.
Foundation review
Scan efficiency
Columnar Parquet can reduce the data read by analytical queries when only some columns are needed.
Metadata and data
Glue ETL changes data; the Data Catalog describes where the data is and how it is structured.