44-EFT.WP.Data.ModelCards v1.0 | Chapter 12 Calibration & Uncertainty

Home ／ Docs-Technical WhitePaper ／ 44-EFT.WP.Data.ModelCards v1.0

Chapter 12 Calibration & Uncertainty

I. Chapter Purpose & Scope

II. Terminology & Dependencies

• Terminology source: Comprehensive Template v0.1; this chapter adds only calibration/uncertainty-specific fields.
• Dependent volumes: Data contract/export Core.DataSpec v1.0; metrology/dimensions & error budget Core.Metrology v1.0; path-dependent expressions (if any) Core.Equations v1.1; evaluation protocol Chapter 11.
• Math & symbols: Use backticks for inline symbols (e.g., p(y|x,θ), q(y|x), ECE, U, u_c); any division/integral/composite operator must use parentheses; no Chinese in formulas/symbols/definitions.

III. Fields & Structure (Normative)

calibration:

method: "<temperature|vector_scale|histogram_binning|isotonic|bayesian|custom>"

params: {t: 1.7?}

eval:

report: ["ece","brier","calibration_curve"]

ece_bins: 15

significance: {test:"bootstrap", alpha:0.05}

coverage:

target_p: 0.95

method: "<tolerance|bayes>"

interval: "<two-sided|one-sided>"

notes?: "<non-normative>"

uncertainty:

model: "<GUM|linear|montecarlo|bayesian>"

components:

- {name:"thermal", type:"random", value:2.1, unit:"K", distribution:"normal", coverage:{k:1.0}}

- {name:"cal_gain", type:"systematic", value:0.8, unit:"%", distribution:"normal", coverage:{k:2.0}, corr_group:"instrument"}

correlation:

posture: "<groups|covariance>"

groups: [{name:"instrument", pairwise:"rho=0.6"}]

covariance?: {Sigma: []}

propagation:

rule: "<rss|linear|montecarlo|bayesian>"

linearization?: "first-order"

samples?: 0

coverage_policy:

target_p: 0.95

k: 2.0

report:

significant_figures: 3

unit_consistency: true

IV. Calibration Methods & Evaluation

• Temperature/Vector scaling: record t or vector dimension; suitable for multi-class softmax.
• Binning/Isotonic: specify number of bins or monotonic constraints; avoid overfitting by fitting on validation and evaluating on test.
• Bayesian calibration: state prior/likelihood and inference method; report posterior mean and quantiles.
• Evaluation metrics: report ECE, Brier, and calibration curves; for differences, provide bootstrap CIs and p-values (alpha=0.05 by default).

V. Uncertainty Modeling & Propagation

1. Component taxonomy: Random (Type A) vs. Systematic (Type B); for each component record name/type/value/unit/distribution/coverage/method.
2. Propagation rules:
   • rss: independent standard uncertainties, u_c = ( sqrt( Σ u_i^2 ) );
   • linear: first-order Taylor, u_c = ( sqrt( J Σ J^T ) ) with J = ( ∂f / ∂x );
   • montecarlo|bayesian: provide sample count or prior/likelihood; report coverage interval and target_p.
3. Expanded uncertainty: U = ( k * u_c ); under normal assumptions, k≈2 ≈ 95%.

VI. Correlation Handling

• Group method: use a shared correlation or pairwise correlations within a group; across groups default to independence.
• Covariance method: provide Σ directly, or standard deviations with correlation matrix R where Σ = D R D.
• Reporting: document correlation sources (calibration/parallel measurement/theory) and sensitivity of u_c.

VII. Metrology & Units

• Declare units for all numeric fields and pass check_dim; for physical/time/frequency/performance quantities, normalize units first before combining.
• When combining statistical (resampling/repeats) and metrological uncertainties, normalize postures first, then compose and report the total uncertainty.

VIII. Path-Dependent Quantities (e.g., T_arr)

1. Two equivalent expressions:
   • T_arr = ( 1 / c_ref ) * ( ∫ n_eff d ell )
   • T_arr = ( ∫ ( n_eff / c_ref ) d ell )
2. Registration: record delta_form, path="gamma(ell)", and measure="d ell" in the Model Card; include uncertainties for n_eff, c_ref, etc., in propagation and pass check_dim.

IX. Machine-Readable Fragment (Drop-in)

calibration:

method: "temperature"

params: {t: 1.7}

eval: {report:["ece","brier","calibration_curve"], ece_bins:15, significance:{test:"bootstrap", alpha:0.05}}

coverage: {target_p:0.95, method:"tolerance", interval:"two-sided"}

uncertainty:

model: "linear"

components:

- {name:"thermal", type:"random", value:2.1, unit:"K", distribution:"normal", coverage:{k:1.0}}

- {name:"cal_gain", type:"systematic", value:0.8, unit:"%", distribution:"normal", coverage:{k:2.0}, corr_group:"instrument"}

correlation: {posture:"groups", groups:[{name:"instrument", pairwise:"rho=0.6"}]}

propagation: {rule:"linear", linearization:"first-order"}

coverage_policy: {target_p:0.95, k:2.0}

report: {significant_figures:3, unit_consistency:true}

X. Consistency with Evaluation, Objectives & Resources

• Calibration evaluation follows Chapter 11’s seeds/repeats/ci/significance.
• The objective L(θ) output assumptions are consistent with the calibrated distribution q(y|x).
• Any compute overhead from calibration/uncertainty inference is reflected in deployment/resources and its metric impact (with significance) is reported in evaluation.

XI. Export Manifest & Audit Trail

export_manifest:

artifacts:

- {path:"calibration/report.md", sha256:"..."}

- {path:"calibration/curve.png", sha256:"..."}

- {path:"uncertainty/breakdown.csv", sha256:"..."}

- {path:"uncertainty/covariance.npy", sha256:"..."}

references:

- "EFT.WP.Core.DataSpec v1.0:EXPORT"

- "EFT.WP.Core.Metrology v1.0:check_dim"

XII. Chapter Compliance Checklist

• calibration.method/params and evaluation indicators/coverage intervals are explicit; differences include p-values and CIs.
• uncertainty.components/correlation/propagation are complete; U = k*u_c matches target_p.
• All fields with units pass check_dim; statistical and metrological uncertainties are normalized before combination.
• For path-dependent quantities (e.g., T_arr), delta_form/path/measure are registered and included in propagation.
• Export manifest lists curves, breakdown tables, covariance and reports with sha256; references use “Volume vX.Y:Anchor”.