A mobile device resends the same event after a delay.
Update daily aggregates while handling duplicates and late events correctly.
Which quality approach is most appropriate?
Use an event key for idempotency and define a recomputation window after the watermark for late events.
Detailed explanation
Valid delayed events are lost without an agreed lateness policy.
Valid delayed events are lost without an agreed lateness policy.
Object retention does not deduplicate or recompute aggregates.
Object retention does not deduplicate or recompute aggregates.
Retries are absorbed and delayed events can update their event-time period.
Retries are absorbed and delayed events can update their event-time period.
Receipt time can differ from event time, creating omissions or duplicates.
Receipt time can differ from event time, creating omissions or duplicates.
Check official documentation
Compare the following points with the official documentation.
Apache Spark watermarks、event time、idempotent processingの公式説明を確認する。
What to confirm
受信日固定・破棄と、イベント時刻に基づく遅延処理の違いを説明できる。
Key points
- Late arrival
- Watermark
- event_id
- Recomputation window
Notes
- Environment: AWS公式DEA-C01試験ガイドと各サービスの公式ドキュメントの確認
- Confirm current AWS service behavior in the official DEA-C01 documentation.
Foundation review
Window design
Balance allowed lateness, recomputation cost, and downstream update frequency.
Correction
Notify consumers when a finalized period changes because of a late event.