Decision-change case study — public Digits data
This file is machine-generated by examples/public_digits_xgboost/run.py.
- Evaluation records: 719
- Model accuracy: 90.26% → 91.79%
- Model ROC AUC: 0.9429 → 0.9472
- Individual final actions changed: 30 (4.17%)
- Changed nodes: model, policy, rules
Final action distributions
| action |
baseline |
candidate |
| auto_8 |
66 |
74 |
| auto_not_8 |
530 |
526 |
| manual_review |
123 |
119 |
Changed transitions
| transition |
count |
| auto_not_8->manual_review |
13 |
| manual_review->auto_8 |
8 |
| manual_review->auto_not_8 |
9 |
Software-counterfactual attribution
| node |
absolute share |
| model |
48.86% |
| policy |
46.59% |
| rules |
4.55% |
Governance contract
Result: BLOCK
| check |
status |
observed |
limit |
| overall decision shift |
PASS |
0.04172461752433936 |
0.05 |
| transition auto_not_8->manual_review rate |
PASS |
0.01808066759388039 |
0.03 |
| transition manual_review->auto_8 rate |
PASS |
0.011126564673157162 |
0.02 |
| reproducibility |
PASS |
reproducible |
reproducible |
| flow attribution efficiency MAE |
PASS |
0.0 |
1e-06 |
| cohort shift rate |
PASS |
0.09411764705882353 |
0.1 |
| actual_digit=6 shift rate |
FAIL |
0.09411764705882353 |
0.08 |
| evidence integrity |
PASS |
True |
True |
The example is descriptive evidence over this fixed public evaluation split. Software-counterfactual attribution does not establish real-world causality.