Tens of thousands of files arrive in S3 and use the same transformation.
Avoid a single serial run and identify failed items for reprocessing.
Which automation approach is most appropriate?
Use a Step Functions Map state to apply a Task to each input item and control concurrency and item-level failures.
Detailed explanation
Execution time, temporary storage, and retry scope become too large for reliable recovery.
Execution time, temporary storage, and retry scope become too large for reliable recovery.
Audit-event aggregation is not a file-processing parallelism mechanism.
Audit-event aggregation is not a file-processing parallelism mechanism.
Map iterates over input items and can control concurrency and item-level results.
Map iterates over input items and can control concurrency and item-level results.
Archive restoration adds delay and does not provide parallel workflow control.
Archive restoration adds delay and does not provide parallel workflow control.
Check official documentation
Compare the following points with the official documentation.
AWS Step Functions Map state、Distributed Map、MaxConcurrencyの公式説明を確認する。
What to confirm
並列反復処理と単一Lambdaへの詰め込みの違いを説明できる。
Key points
- Map
- Distributed Map
- Concurrency
- Item retry
Notes
- Environment: AWS公式DEA-C01試験ガイドと各サービスの公式ドキュメントの確認
- Confirm current AWS service behavior in the official DEA-C01 documentation.
Foundation review
Capacity matching
Set concurrency to the limits of Glue, Redshift, or other downstream systems.
Result correlation
Keep an input identifier with each success or failure for targeted replay.