53-Model Card Template v1.0 | Chapter 12 — Monitoring, Drift & Rollback

Home ／ Docs-Technical WhitePaper ／ 53-Model Card Template v1.0

Chapter 12 — Monitoring, Drift & Rollback

I. Purpose & Scope

• Standardize deployment monitoring, drift detection, and rollback metrics, thresholds, workflows, and release conventions so failures/mismatches are detected early, safely degraded, and auditable for rollback.
• For path quantities (arrival time/phase), the text must explicitly show gamma(ell) and d ell; the data side records delta_form ∈ {general, factored}; all expressions are parenthesized; publication requires p_dim = 1.0.

II. Prerequisites & Inputs

• Data & splits: align with Dataset Card Ch. 4/6/7/11 (Schema/Splits/QC/Bench); online sampling consistent with offline evaluation.
• Training & deployment: align with this volume Ch. 6 (Training) and Ch. 10 (Deployment Interfaces); best.ckpt and env snapshot locked.
• Coverage & covariance: align with Error Budget (coverage ∈ {k, alpha, quantile}, Σ PD).
• Parameter freshness: align with Parameter Card (freshness.policy, cov_group).
• Citations & versions: “volume + version + anchor (P/S/M/I)”, anchor coverage ≥ 90%; public v1.* only.

III. Monitoring KPIs & Thresholds

• Data plane: distribution drift (KS/ψ/EMD), missing & anomaly rates, path consistency (len(gamma_ell)=len(d_ell)=len(n_eff)≥2, Δell ≤ ( c_ref / f_s ) / max(n_eff)).
• Model plane: Q_res, r_phi, ε_flux, p_dim (=1), predictive uncertainty U=k·u_c or quantile coverage.
• Timebase & sync: clock_state, δt_abs, Δτ_ch, σ_y(τ).
• Resources & performance: Latency_P95/P99, Throughput, ρ, P_avg/energy_per_req, loss_rate.
• Threshold mapping: align with Ch. 8/11 and Error Budget Ch. 9; breaches trigger degrade/rollback.

IV. Drift Detection

1. Data drift:
   • Tests: KS/χ²/AD; multivariate MMD/Energy distance; windowed stratification (batch/device/region).
   • Path quantities: interval coverage & band-width trends for T_arr/Phi; align phase within reference window first.
2. Concept drift:
   • Proxy ground truth / delayed labels: align online feedback with val/test/holdout.
   • Performance decay: ΔMAE/ΔAUC/Δr_phi over thresholds with non-overlapping CIs.
3. Uncertainty calibration: PIT/calibration curves/Brier; on failure, enable conservative intervals or robust surrogates.

V. Rollback Mechanism

1. FSM: normal → degrade → rollback → recover → normal, event-driven (gate breach/drift confirmed/resource alerts).
2. Degrade:
   • Model: route to lower-complexity path / robust surrogates (Huber/quantile).
   • Data: tighten gates, isolate risky slices.
   • Path: switch to fullband/short window or raise Δell guard (without breaking upper bounds).
3. Rollback execution: lock previous stable version (signature & checksum), keep I/O contract & coverage mode unchanged.
4. Recovery & verification: progressive canary rollout; after /validate passes G1–G8 and perf/quality thresholds, switch fully.

VI. Normative Path Forms

• Arrival (two equivalent):
  T_arr = ( 1 / c_ref ) * ( ∫ n_eff d ell )
  T_arr = ( ∫ ( n_eff / c_ref ) d ell )
• Phase accumulation:
  Phi = ( 2π / λ_ref ) * ( ∫ n_eff d ell )

Align “time → path → phase” before monitoring & alerts; record delta_form; arrays satisfy length & step constraints.

VII. Gate Mapping

• G1 Schema completeness (monitoring/drift report fields present).
• G2 Citation compliance (anchor coverage ≥ 90%).
• G3 Path conventions (blocks complete; step compliant).
• G4 Dimensional closure (online/offline calculations keep p_dim = 1.0).
• G5 Freshness (clock_state="locked").
• G6 Coverage consistency (online intervals match publication k/alpha/quantile).
• G7 Covariance consistency (Σ PD, aligned with Error Budget).
• G8 Uniqueness & acyclicity (events/artifacts with checksum, lineage acyclic).
• Trigger S1–S5 (dimension/freshness/path/covariance/citation) to degrade/rollback; tag [Restricted] when applicable.

VIII. Machine-Readable Configs
A. monitoring_rules.yaml

version: "1.0.0"

windows: { short_s: 300, long_s: 86400 }

kpis:

latency_p95_s: { target: 0.200, alert: 0.250, critical: 0.300 }

throughput_rps: { target_min: 1000 }

q_res: { target_max: 0.20 }

p_dim: { require: 1.0 }

r_phi_lb95: { target_min: 0.60 }

epsilon_flux_p95: { target_max: 0.02 }

delta_t_abs_ns: { target_max: 50 }

delta_tau_ch_ns: { target_max: 5 }

drift:

data: { test: "ks", p_crit: 0.01, strata: ["device","region"] }

concept: { metric: "val/MAE", delta_crit: 0.05, ci_agree: true }

actions:

on_alert: ["degrade"]

on_critical: ["rollback"]

B. rollback_fsm.yaml

version: "1.0.0"

states: [normal, degrade, rollback, recover]

transitions:

- { from: normal, to: degrade, when: "gate_alert or drift_alert" }

- { from: degrade, to: rollback, when: "gate_critical or perf_critical" }

- { from: rollback,to: recover, when: "stable_prev_version_ready" }

- { from: recover, to: normal, when: "validate_pass and perf_ok" }

degrade:

strategies: ["robust_surrogate","tighten_gates","isolate_slices"]

rollback:

version_tag: "v1.2.3-lock"

verify: ["checksum","/validate","SLA/SLO"]

recover:

rollout: { canary_percent: 10, steps: 3, pause_s: 600 }

C. alerts.jsonl (sample)

{
  "ts": "2025-09-24T16:10:00Z",
  "level": "critical",
  "event": "gate_fail",
  "gate": "G4",
  "detail": "p_dim < 1.0",
  "action": "rollback"
}

IX. Anti-Patterns & Fixes

• Anti: reporting means only, no intervals/CIs → Fix: add U=k·u_c or quantile bands with convergence diagnostics.
• Anti: T_arr = ∫ n_eff / c_ref d ell (no parentheses) → Fix: use parenthesized unified form.
• Anti: drift detected but no degrade/rollback → Fix: bind automatic FSM actions and approval thresholds.
• Anti: rollback version unsigned/no checksum → Fix: require signature and checksum verification.
• Anti: path block missing d ell/delta_form → Fix: complete and equalize with n_eff before alert computation.

X. Cross-References

• Dataset Card: Ch. 7 (QC Gates), Ch. 8 (UQ/Cov), Ch. 11 (Bench/Score), Ch. 10 (API).
• Error Budget Card: Ch. 8/9 (intervals & thresholds).
• Pipeline Card: Ch. 7 (State/Idempotency/Fault Tolerance), Ch. 9 (Gates/Monitoring/Alerts), Ch. 12 (Outputs/Release).
• This volume: Ch. 6 (Training), Ch. 7 (UQ), Ch. 10 (Deployment Interfaces).

XI. Checklist

• monitoring_rules.yaml / rollback_fsm.yaml / alerts.jsonl stored and active.
• For path quantities, explicit gamma/measure/delta_form; p_dim = 1.0; alerts aligned with gates.
• Drift tests (data/concept) reproducible; degrade/rollback actions & approvals clearly defined and audited.
• Resource/performance monitoring aligned with Ch. 11; thresholds & regression strategy effective.
• /validate passed G1–G8; non-compliances tagged [Restricted] and handled; anchor coverage ≥ 90%.