48-Experimental Protocol Card Template v1.0 | Chapter 11 — Results Page & Comparative Scoring (Protocol Output)

Home ／ Docs-Technical WhitePaper ／ 48-Experimental Protocol Card Template v1.0

Chapter 11 — Results Page & Comparative Scoring (Protocol Output)

I. Purpose & Scope

• Standardize the required fields, metric conventions, visualization standards, and scorecard exports for the protocol Results Page, to enable baseline comparison and release review.
• Applicable to timing / imaging / spectral / mixed. When path quantities (arrival time/phase) are involved, explicitly show gamma(ell) and the measure d ell in text, and record delta_form ∈ {general, factored} in data/metadata.

II. Required Items on the Results Page

1. Method & data identifiers: dataset_id, method_id, baseline_id, versions (SemVer), random seeds, observation time window.
2. Primary metrics:
   • Arrival-time residual ΔT_arr (s): T_arr(obs) − T_arr(ref); report mean ± U(k) and histogram/KDE.
   • Phase consistency r_phi (1): corr(Phi_ref, Phi_obs); intervals via Fisher-z and back-transform.
   • Paraxial conservation error ε_flux (1): → 0 @ O(θ^2).
   • Dimensional-closure rate p_dim (1): pass fraction.
   • Robust residual metric Q_res (1): lower is better (robust quantile gap or Huber surrogate).
3. Secondary metrics: mass-conservation deviation ΔM, coherence-window adequacy κ_coh, etc.
4. Unified arrival-time/phase forms (display):
   • T_arr = ( 1 / c_ref ) * ( ∫ n_eff d ell ) or T_arr = ( ∫ ( n_eff / c_ref ) d ell )
   • Phi = ( 2π / λ_ref ) * ( ∫ n_eff d ell )
     In text, explicitly show gamma(ell) and d ell.
5. Compliance block: summary of check_dim_report.json, anchor_coverage, ver_presence, clock_state, τ_calib.

III. Comparators & Scoring

1. Baseline & comparators: provide baseline_id/version, data splits, and stratification (batch/device/region).
2. Paired comparison: per record_id, compute Δm = m_method − m_base; use paired or permutation tests for core metrics; control multiple testing with FDR ≤ 0.1.
3. Standardization & mapping:
   • Z-norm: z_m = ( m − m_baseline ) / σ_baseline
   • Sigmoid score: q_m = 1 / ( 1 + exp( a z_m + b ) ) (default a=1, b=0; invert sign if needed).
4. Aggregate score: Q = ( ∑_i w_i q_{m_i} ) / ( ∑_i w_i ); weights w_i are fixed in the evaluation sheet.
5. Decision thresholds (aligned with Chapter 8):
   • Positive: all core gates pass (e.g., |ΔT_arr| ≤ τ_T, r_phi ≥ 0.6, p_dim = 1.0) and Q ≥ Q_base + δQ_min.
   • Negative: any core gate fails or citations/dimensions are non-compliant.

IV. Visualization Standards

• Dashboard: ΔT_arr distribution (histogram/KDE), r_phi bar with intervals, ε_flux boxplot, Q_res trend, p_dim gauge.
• Agreement & residuals: residual-vs-fitted and Bland–Altman; phase scatter Phi_obs vs Phi_ref with y=x line and interval bands.
• Path & geometry: n_eff(ell) vs ell profile; caption must state Δell and delta_form; paraxial conservation heatmap with ε_flux contours.
• Error bars & intervals: means/medians with ±U(k) or quantile bands, stating k or confidence level.
• Figure export: both vector (PDF/SVG) and bitmap (PNG); captions include see[] and version; axes show explicit units (s, rad, 1).

V. Scorecard Export

• Fields: method_id, baseline_id, dataset_id, metrics{ΔT_arr,r_phi,ε_flux,p_dim,Q_res,...}, score.Q, tests{paired,perm,FDR}, seeds, references[], version.
• Example (JSON):

{

"version": "1.0.0",

"dataset_id": "ptn-demo",

"baseline": { "id": "base-001", "version": "1.2.3" },

"method": { "id": "mA-010", "version": "2.0.0" },

"metrics": {

"DeltaT_arr_s": { "mean": -2.3e-9, "std": 4.8e-9, "U_k2": 1.5e-9 },

"r_phi": { "value": 0.72, "ci95": [0.61, 0.80] },

"epsilon_flux": { "median": 0.004, "p95": 0.011 },

"p_dim": 1.0,

"Q_res": 0.13

},

"score": { "Q": 0.78 },

"tests": {

"paired": { "DeltaT_arr": { "p_perm": 0.004, "B": 10000 } },

"FDR": 0.08

},

"see": [

"EFT.WP.Core.Equations v1.1:S20-1",

"EFT.WP.Core.Metrology v1.0:check_dim",

"Data.Benchmarks v1.0:PROTO"

],

"version_lock": true

}

VI. Release Bundle & Compliance

• Required deliverables: scorecard.json, results.md, figure bundle (PDF/PNG), check_dim_report.json, quality_report.json, audit.jsonl.
• Consistency: text and exports have consistent see[]/references[]/version; anchor coverage ≥ 90%; external links and aliases are forbidden.
• Restricted mode: when Chapter 8 S1–S5 is triggered, append [Restricted] to all figure titles and publish qualitative trends and diagnostics only.

VII. Machine-Readable Configuration
A. bench_score.yaml (interface contract)

version: "1.0.0"

call: "I90-bench_score"

inputs:

results: "PTN_EXPORT/results.parquet"

baseline: "PTN_EXPORT/baseline.parquet"

metrics: ["DeltaT_arr_s","r_phi","epsilon_flux","p_dim","Q_res"]

weights: { DeltaT_arr_s: 0.35, r_phi: 0.25, epsilon_flux: 0.15, p_dim: 0.15, Q_res: 0.10 }

thresholds:

tau_T_s: "3*u(T_arr)"

r_phi_min: 0.6

flux_ok: "≈0@O(theta^2)"

p_dim: 1.0

mapping:

type: "sigmoid"

a: 1.0

b: 0.0

exports:

files: ["scorecard.json","results.md","figs/*.pdf","reports/check_dim_report.json"]

VIII. Citations & Versioning

• Fixed syntax: See "<Volume> vX.Y" <Chapter> <Anchor>; prioritize P/S/M/I; keep text and exports consistent.
• Normative anchors: EFT.WP.Core.Equations v1.1:S20-1/S21-2, EFT.WP.Core.Metrology v1.0:check_dim, Data.Benchmarks v1.0:PROTO.
• Public releases use v1.* only; if v0.* is used, mark “draft / non-committal”.

IX. Checklist

• dataset_id / method_id / baseline_id / version / seeds complete; time window specified.
• Core metrics — ΔT_arr / r_phi / ε_flux / p_dim / Q_res — reported with units, intervals, and figures.
• Scoring — Q, weights, thresholds, statistical tests, and FDR — clearly reported.
• Compliance block lists check_dim_report.json summary, anchor_coverage / ver_presence, clock_state / τ_calib.
• Exports complete: scorecard.json / results.md / figs / quality_report.json / audit.jsonl; citations compliant, versions locked.
• If [Restricted], all figures tagged and triggers listed; qualitative outlook only.