Fraud behavior changes and the relationship between features and labels shifts.
Reevaluate the model.
Choose two appropriate responses.
Monitor period-based performance and feature-label relationships, then evaluate retraining on recent examples before an approved switch.
Detailed explanation
The model can adapt through controlled change management.
The actual problem becomes harder to diagnose.
Concept drift can occur without covariate drift.
A changing relationship can appear even if input proportions are stable.
Check official documentation
Compare the following points with the official documentation.
Review monitoring and retraining in Task 1.3 and the AWS MLOps checklist on distribution shift, concept drift, and model-quality checks. For an example where input frequencies stay similar but fraud outcomes change, identify the required labels and time-based evaluations.
Official references
What to confirm
Distinguish changes in input distribution from changes in the input-outcome relationship, and explain outcome-based reevaluation and candidate-model approval.
Key points
- Concept drift
- Period metrics
- Retraining
Notes
- Environment: Reference review of official documentation and hypothetical examples; no AWS resources or model training are required.
- Stable input frequencies do not rule out a changed outcome relationship for the same inputs. Input-only monitoring cannot establish the absence of concept drift.
- Outcome-based quality checks require ground-truth labels after inference. Account for label delays and evaluation periods before approving and promoting a candidate.
Foundation review
Concept drift
A change over time in the relationship between input variables and the target outcome.
Period-based evaluation
Comparing model quality across time windows to reveal changing behavior.