A Spark ETL job runs for several hours only once a week.
Obtain distributed processing capacity only while the job runs and transform S3 data.
Which service is most appropriate?
Use Amazon EMR Serverless to run Spark jobs without maintaining an always-on EMR cluster.
Detailed explanation
Instances incur management and cost during the long periods when the job is not running.
Instances incur management and cost during the long periods when the job is not running.
Lambda execution, memory, and temporary-storage limits make this unsuitable for large Spark work.
Lambda execution, memory, and temporary-storage limits make this unsuitable for large Spark work.
Object Lock protects stored objects and does not execute Spark jobs.
Object Lock protects stored objects and does not execute Spark jobs.
EMR Serverless provisions the needed capacity for a Spark job and avoids persistent cluster operations.
EMR Serverless provisions the needed capacity for a Spark job and avoids persistent cluster operations.
Check official documentation
Compare the following points with the official documentation.
Amazon EMR Serverlessのアプリケーション、ジョブ実行、容量・コストの公式説明を確認する。
What to confirm
EMRクラスタとEMR Serverlessの運用・料金・適用条件を説明できる。
Key points
- EMR Serverless
- Spark
- Intermittent ETL
- Capacity management
Notes
- Environment: AWS公式DEA-C01試験ガイドと各サービスの公式ドキュメントの確認
- Confirm current AWS service behavior in the official DEA-C01 documentation.
Foundation review
Serverless trade-off
On-demand capacity reduces idle operations but still requires runtime, dependency, and quota planning.
Job execution
Logs, retries, S3 permissions, and maximum capacity determine reliable repeatable runs.