53-Model Card Template v1.0 | Chapter 7 — Error Budget & Uncertainty (Model Level)

Home ／ Docs-Technical WhitePaper ／ 53-Model Card Template v1.0

Chapter 7 — Error Budget & Uncertainty (Model Level)

I. Purpose & Scope

• Standardize modeling, estimation, composition, and publication conventions for model-level error budget and uncertainty (UQ), covering Delta/MC/Bootstrap, unified interval conventions, covariance block construction, and gate mapping, so model output credibility is quantifiable, auditable, and reproducible.
• For path quantities (arrival time/phase), the text must explicitly show gamma(ell) and d ell; the data side records delta_form ∈ {general, factored}; parenthesized unified forms are required; publication requires p_dim = 1.0.

II. Prerequisites & Inputs

• Data & splits: align with Dataset Card Ch. 4/6/7/8 (schema/splits/QC/UQ); isolate expired samples.
• Parameters & metrology: align freshness.policy and cov_group per the Parameter Card.
• Error Budget conventions: choose exactly one coverage mode (k/alpha/quantile) for the whole volume; ensure Σ PD and kernels/params align with Error Budget Ch. 5/6/8.
• Path consistency: len(gamma_ell)=len(d_ell)=len(n_eff)≥2, Δell ≤ ( c_ref / f_s ) / max(n_eff); align phase in the reference window before evaluation.
• Citations & versions: “volume + version + anchor (P/S/M/I)”, anchor coverage ≥ 90%.

III. Methods & Applicability

• Delta (first-order): for y=f(x), u^2(y) ≈ J · Σ · Jᵀ, with J=∂f/∂x|_{x̂}; suitable for near-linear, near-Gaussian residuals; state J source (auto/analytic).
• Monte Carlo (MC): sample x ~ P(x; Σ), compute y_b=f(x_b), B ≥ 10^4; report mean/std/quantile bands and convergence diagnostics (variance vs samples).
• Bootstrap: paired/stratified/residual bootstrap; keep {batch/device/region} strata; for heavy tails, report robust quantiles.
• Robust surrogates: under outliers/heteroscedasticity, use Huber/quantile regression to produce second-order equivalents for Delta approximations.
• Selection & switching: prefer Delta; switch to MC/Bootstrap when linearization residuals fail or nonlinearity is material; compare results in the report.

IV. Coverage & Intervals

1. Modes (choose one across the volume):
   • k coverage (expanded uncertainty): U = k·u_c;
   • alpha significance (CI/t);
   • quantile[p_lo,p_hi] (e.g., P2.5–P97.5).
2. Publication rules: report point estimates and intervals for key metrics; visualize with bars/bands and annotate coverage mode & params.
3. DOF & small samples: use Welch–Satterthwaite DOF for Delta; for small samples, use t_{ν,1−α/2}.

V. Covariance Modeling

• Intra-group: within a cov_group, use blocks or kernels (exp(−|Δx|/L_c), AR(1), Matérn, constant ρ).
• Inter-group: default independent; if coupling exists (e.g., temperature coefficient vs refractive index), register cross-covariance and align with the Error Budget.
• Numerical stability: ensure Σ PD; add jitter Σ ← Σ + εI (ε ≈ 1e−6~1e−3) if needed.
• Shape & units: Σ carries the units of targets; pass unit/dimension checks before release.

VI. Normative Path Forms

• Arrival (two equivalent):
  T_arr = ( 1 / c_ref ) * ( ∫ n_eff d ell )
  T_arr = ( ∫ ( n_eff / c_ref ) d ell )
• Phase:
  Phi = ( 2π / λ_ref ) * ( ∫ n_eff d ell )

In text, explicitly show gamma(ell) and d ell; record delta_form; arrays len(gamma_ell)=len(d_ell)=len(n_eff)≥2.

VII. Budget Decomposition & Composition

1. Decomposition: attribute variance shares (data noise, parameter uncertainty, numerical error, model mismatch, etc.); produce a contribution table and proportions.
2. Composition: sum squares for uncorrelated terms; include cross-terms or compose via Σ for correlated terms; list sensitivity ranking (|∂y/∂x_i|).
3. Gate comparison:
   • |ΔT_arr| + U(T_arr) ≤ τ_T → positive;
   • LB(r_phi) ≥ r_phi_min;
   • P95(ε_flux) ≤ ε_flux_guard;
     unmet → degrade or [Restricted].

VIII. Gate Mapping

• G1 Schema completeness (UQ fields vs contract).
• G2 Citation compliance (anchor coverage ≥ 90%).
• G3 Path conventions (gamma/measure/delta_form complete).
• G4 Dimensional closure (I70-dim_check, p_dim = 1.0).
• G6 Coverage consistency (single mode across the volume).
• G7 Covariance consistency (Σ PD and aligned with Error Budget).
• G8 Uniqueness & acyclicity (artifacts with checksum, lineage acyclic).
• Any core-gate failure triggers S1–S5, blocking release or tagging [Restricted].

IX. Machine-Readable Artifacts
A. model_uq.yaml

version: "1.0.0"

targets: ["T_arr","Phi","epsilon_flux","r_phi","Q_res"]

methods:

T_arr: { type: "delta", jacobian: "auto", cov_group: "medium" }

Phi: { type: "mc", draws: 10000, coverage: { quantile: [0.025,0.975] } }

coverage: { mode: "k", k: 2 } # k|alpha|quantile

covariance:

medium: { kernel: "exp", params: { sigma2: 9.0e-6, L_c_m: 25.0 } }

reports: ["uq_summary.json","cov_blocks.json"]

B. uq_summary.json (example)

{

"T_arr": { "point": 1.23e-8, "U_k2": 1.5e-9 },

"Phi": { "median": 0.035, "q025": 0.028, "q975": 0.043 },

"r_phi": { "value": 0.72, "lb95": 0.61, "ub95": 0.80 },

"epsilon_flux": { "p95": 0.011 },

"Q_res": 0.13

}

C. budget_breakdown.csv (headers)

source,variance_share,u_or_sigma,notes

data_noise,0.41,7.8e-10,"val split"

param_freshness,0.22,4.1e-10,"tau_calib margin"

model_mismatch,0.27,5.0e-10,"nonlinear residual"

numerical,0.10,2.1e-10,"solver tol"

X. Anti-Patterns & Fixes

• Anti: reporting means without intervals → Fix: add U=k·u_c or quantile bands with convergence diagnostics.
• Anti: T_arr = ∫ n_eff / c_ref d ell (missing parentheses) → Fix: parenthesize to the unified form.
• Anti: Σ not PD or not aligned with Error Budget → Fix: adjust kernels/params or switch robust surrogate; align configs.
• Anti: inconsistent coverage across Model/Data/Error volumes → Fix: unify mode and declare in manifests & captions.

XI. Cross-References

• Dataset Card: Ch. 7 (QC Gates), Ch. 8 (UQ & Cov), Ch. 11 (Bench/Score).
• Error Budget Card: Ch. 5/6/8/9 (covariance, propagation, intervals & thresholds).
• Pipeline Card: Ch. 10 (UQ Coupling).
• Parameter Card: Ch. 8 (parameter-level uncertainty & covariance).

XII. Checklist

• model_uq.yaml / uq_summary.json / budget_breakdown.csv generated and aligned with the Error Budget Card.
• Coverage mode unified (k/alpha/quantile); Σ PD with kernels & params recorded; J source stated.
• For path quantities, explicit gamma/measure/delta_form; I70-dim_check passed, p_dim = 1.0.
• Gate comparisons (τ_T / r_phi_min / ε_flux_guard) concluded; unmet cases handled as [Restricted].
• /validate passed G1–G8; citation anchor coverage ≥ 90%; release artifacts include checksums & signatures.