Start a transformation after a crawler completes and start aggregation only after the transformation succeeds.
Stop downstream work and notify operators when a step fails.
Which automation approach is most appropriate?
Use a Glue Workflow with conditional Triggers so crawler and job states control downstream execution, notification, and retry.
Detailed explanation
An API record does not describe whether the data-processing job succeeded.
An API record does not describe whether the data-processing job succeeded.
Crawler and job states can determine whether the next job starts or a failure path is taken.
Crawler and job states can determine whether the next job starts or a failure path is taken.
A transformation can begin before the crawler has produced usable metadata.
A transformation can begin before the crawler has produced usable metadata.
Object expiration is not a job-state result.
Object expiration is not a job-state result.
Check official documentation
Compare the following points with the official documentation.
AWS Glue Workflows、Triggers、conditional trigger predicatesの公式説明を確認する。
What to confirm
ジョブの状態を条件にするTriggerと時刻起動の違いを説明できる。
Key points
- Glue Workflow
- Conditional Trigger
- Crawler
- Success condition
Notes
- Environment: AWS公式DEA-C01試験ガイドと各サービスの公式ドキュメントの確認
- Confirm current AWS service behavior in the official DEA-C01 documentation.
Foundation review
Dependency expression
The start condition should be visible instead of being inferred from a clock.
Failure isolation
Stop downstream work and preserve the failed input for a controlled rerun.