customer_id and order_time are required in order data.
Keep rows with missing values out of aggregates and report the count and source.
Which quality approach is most appropriate?
Measure NULL rates with completeness rules and quarantine or notify when the agreed threshold is exceeded.
Detailed explanation
Replacement hides a required-field violation instead of detecting it.
Replacement hides a required-field violation instead of detecting it.
Deletion hides the issue and removes replay input.
Deletion hides the issue and removes replay input.
Required-column omissions become measurable and can stop publication or trigger quarantine.
Required-column omissions become measurable and can stop publication or trigger quarantine.
A successful job can still produce incomplete data.
A successful job can still produce incomplete data.
Check official documentation
Compare the following points with the official documentation.
AWS Glue Data Quality completeness rules、NULL、thresholdの公式説明を確認する。
What to confirm
欠損を置換して隠す方法と、完全性ルールで検知する方法を区別できる。
Key points
- Completeness
- NOT NULL
- Missing-value rate
- Quality gate
Notes
- Environment: AWS公式DEA-C01試験ガイドと各サービスの公式ドキュメントの確認
- Confirm current AWS service behavior in the official DEA-C01 documentation.
Foundation review
Tolerance
Decide whether a field must be complete for every row or may have a bounded exception rate.
Quarantine
Keep failing rows and source metadata for correction and replay.