S/ShiftWatchv1.0
Reproducible experimentView source
ML WORKSPACE / UCI Wine
DATA DRIFT & MODEL EVALUATION

Model monitoring.

Trace a distribution shift. Understand its impact on your model.

UCI WineClassification
Logistic regressionSeed 42124 reference / 54 current
Simulated shift

Four features shifted by 2 training standard deviations.

Features flagged
4/ 13
Distribution or missingness alerts
Model accuracy
53.7%
-44.4 pp vs untouched holdout
Macro F1
0.479
Equal weight across classes
Current observations
54
Perturbed test rows · original labels
FEATURE DISTRIBUTION

flavanoids

ReferenceCurrent
Observed-value distribution: flavanoidsReference mean 2.007, current mean 3.983. Each bar shows the proportion of observed values in a bin.0%7%15%22%30%Reference: 20.2%, 0.34 to 0.89Current: 0.0%, 0.34 to 0.890.3Reference: 12.1%, 0.89 to 1.45Current: 0.0%, 0.89 to 1.45Reference: 13.7%, 1.45 to 2.00Current: 0.0%, 1.45 to 2.00Reference: 16.1%, 2.00 to 2.55Current: 9.3%, 2.00 to 2.552.0Reference: 25.8%, 2.55 to 3.11Current: 18.5%, 2.55 to 3.11Reference: 10.5%, 3.11 to 3.66Current: 13.0%, 3.11 to 3.66Reference: 1.6%, 3.66 to 4.21Current: 18.5%, 3.66 to 4.213.7Reference: 0.0%, 4.21 to 4.77Current: 9.3%, 4.21 to 4.77Reference: 0.0%, 4.77 to 5.32Current: 20.4%, 4.77 to 5.32Reference: 0.0%, 5.32 to 5.88Current: 9.3%, 5.32 to 5.885.3Reference: 0.0%, 5.88 to 6.43Current: 0.0%, 5.88 to 6.43Reference: 0.0%, 6.43 to 6.98Current: 1.9%, 6.43 to 6.98
Equal-width display bins · observed values only
Reference mean 2.01Current mean 3.98KS statistic 0.625

Feature diagnostics 13

Select a feature to inspect its distribution.

FeaturePSI ↓KS q-valueMissing (current)Status
2.794< 0.0010.0%Alert
2.424< 0.0010.0%Alert
2.313< 0.0010.0%Alert
2.173< 0.0010.0%Alert
0.4700.6820.0%No alert
0.2880.9370.0%No alert
0.2770.7200.0%No alert
0.1990.8500.0%No alert
0.1830.8500.0%No alert
0.1810.7200.0%No alert
0.1720.3450.0%No alert
0.1470.9370.0%No alert
0.1370.9290.0%No alert
13 features shown · BH-adjusted KS p-values · threshold changes are exploratory