Find the 100 highest-selling products from hundreds of millions of events.
Reduce the cost of a full global sort while preserving correct final ranking.
Which algorithmic approach is most appropriate?
Compute local Top-K candidates per partition, then merge the small candidate sets into the final Top-K result.
Detailed explanation
Network and memory become a bottleneck and distributed parallelism is lost.
Network and memory become a bottleneck and distributed parallelism is lost.
Policy order does not represent data values or sales ranking.
Policy order does not represent data values or sales ranking.
Local candidates reduce network transfer and avoid a single-node full sort of every record.
Local candidates reduce network transfer and avoid a single-node full sort of every record.
Object retention does not execute an aggregation or ranking algorithm.
Object retention does not execute an aggregation or ranking algorithm.
Check official documentation
Compare the following points with the official documentation.
Apache SparkのorderBy、limit、分散集約、Top-Kアルゴリズムの公式説明を確認する。
What to confirm
全件ソートと分散候補統合の計算量・通信量の違いを説明できる。
Key points
- Top-K
- Distributed aggregation
- Candidate merge
- Complexity
Notes
- Environment: AWS公式DEA-C01試験ガイドと各サービスの公式ドキュメントの確認
- Confirm current AWS service behavior in the official DEA-C01 documentation.
Foundation review
Distributed reduction
Local Top-K results reduce network traffic before final aggregation.
Ranking correctness
Define tie-breaking, null handling, and the local candidate count so the global result is exact.