Logs are added to S3 every hour and processed by a recurring Glue job.
Avoid rereading processed files on every run while retaining a controlled way to reprocess after a failure.
Which feature is most appropriate?
Enable AWS Glue job bookmarks to track processed input state for incremental ETL.
Detailed explanation
Bookmarks track processed input and help the next run select unprocessed files.
Bookmarks track processed input and help the next run select unprocessed files.
Change sets describe infrastructure updates, not ETL processing progress.
Change sets describe infrastructure updates, not ETL processing progress.
Access scope does not track which input files a Glue job has processed.
Access scope does not track which input files a Glue job has processed.
Query-result reuse does not manage Glue ETL input state.
Query-result reuse does not manage Glue ETL input state.
Check official documentation
Compare the following points with the official documentation.
AWS Glue job bookmarksの有効化、リセット、S3入力の公式説明を確認する。
What to confirm
増分ETLの状態管理と、クエリキャッシュ・インフラ変更管理の違いを説明できる。
Key points
- Glue job bookmarks
- Incremental ETL
- Processing state
- Replay
Notes
- Environment: AWS公式DEA-C01試験ガイドと各サービスの公式ドキュメントの確認
- Confirm current AWS service behavior in the official DEA-C01 documentation.
Foundation review
Stateful ingestion
Bookmarks reduce repeated reads but require a plan for resets, schema changes, and output duplicates.
Failure recovery
Review bookmark behavior and error outputs before deciding the safe reprocessing boundary.