54-Reproducibility Checklist Template v1.0 | Chapter 10 — Reproduction Workflow (Train / Infer / Align)

Home ／ Docs-Technical WhitePaper ／ 54-Reproducibility Checklist Template v1.0

Chapter 10 — Reproduction Workflow (Train / Infer / Align)

I. Purpose & Scope

• Provide an end-to-end executable workflow from environment bring-up to results verification: Preflight → Data Verify → Train → Infer → Align → Evaluate & Intervals → Compare & Release → Pack & Sign, ensuring single-run reproducibility, verifiable artifacts, and auditable decisions.
• For path quantities (arrival/phase), explicitly show gamma(ell) and d ell; record delta_form ∈ {general, factored}; parenthesize all expressions; publication requires p_dim = 1.0 with check_dim_report.json.

II. Prerequisites & Inputs

• Depends on: Ch. 4 (Environment Lock), Ch. 5 (Data Snapshot & Lineage), Ch. 6 (Weights/Params/Freshness), Ch. 7 (Scripts & Commands), Ch. 8 (Seeds/Randomness/Determinism), Ch. 9 (Metrics/Intervals/Gates).
• Citations: “volume + version + anchor (P/S/M/I)”, anchor coverage ≥ 90%; cross-volume coverage.mode ∈ {k, alpha, quantile} unified.

III. E2E Flow

• Preflight: validate env_lock.json / container_spec.yaml / seed_policy.yaml; emit preflight_report.json.
• Data Verify: verify data_refs.yaml / split_manifest.json / lineage_graph.json / checksums.txt; emit data_verify.json.
• Train: execute train_config.yaml; produce weights/best.ckpt and logs; record rng_state.json.
• Infer: follow binding_spec.md / inference_openapi.yaml / inference.proto; produce outputs/preds.json.
• Align (time → path → phase): align gamma_ell/d_ell/n_eff, echo delta_form in metadata.
• Eval & UQ: run bench_plan.yaml; compute point + interval metrics; emit scorecard.json / uq_summary.json.
• Compare & Validate: use compare_spec.yaml to generate validate_report.json; decide release against gates & tolerances.
• Pack & Sign: update report_manifest.yaml; produce checksums.txt / SIGNATURE.asc; archive to PTN_EXPORT/.

IV. Critical Alignment (Time → Path → Phase)

• Time: windows/watermarks with clock_state="locked"; |ts_start − calib.timestamp| ≤ τ_calib.
• Path: arrays equal length len(gamma_ell)=len(d_ell)=len(n_eff)≥2, step Δell ≤ ( c_ref / f_s ) / max(n_eff).
• Phase: evaluate within the reference window for phase and related metrics.
• Unified forms (two equivalent):
  T_arr = ( 1 / c_ref ) * ( ∫ n_eff d ell )
  T_arr = ( ∫ ( n_eff / c_ref ) d ell )
  Phase: Phi = ( 2π / λ_ref ) * ( ∫ n_eff d ell ).

V. Executable Steps

• Preflight: make preflight or ./reproduce.sh preflight; stop on failure and write audit.jsonl.
• Data verify: make data_verify; check schema/splits/lineage/checksums.
• Train: make train IDK=<idempotency_key>; save best.ckpt/last.ckpt and reports/train.log.
• Infer: make infer IDK=<idempotency_key>; produce outputs/preds.json with path metadata.
• Eval & intervals: make eval; write eval/scorecard.json, intervals, and convergence curves.
• Compare & release: make compare; produce reports/validate_report.json and final decision pass|fail|[Restricted].
• Pack & sign: make pack; update report_manifest.yaml; write checksums.txt / SIGNATURE.asc.

VI. Artifacts & Logs

• Required: preflight_report.json / data_verify.json / best.ckpt / preds.json / scorecard.json / uq_summary.json / validate_report.json / report_manifest.yaml / checksums.txt / SIGNATURE.asc / audit.jsonl.
• Figures: figs/metric_curves.{pdf,png}, figs/interval_bands.{svg,png}, figs/path_profile.pdf (dual-export; captions include units & coverage mode).

VII. Release Decision

• Pass: /validate passes G1–G8 and key metrics (point + interval) satisfy compare_spec.yaml thresholds & tolerances.
• Restricted: if any core gate fails or bands do not overlap → [Restricted]: publish qualitative plots & diagnostics only.
• Block: dimensional closure fails (p_dim < 1.0) or time lock violation—stop immediately and rollback.

VIII. Gate Mapping

• G1 Schema completeness | G2 Citation compliance | G3 Path conventions | G4 Dimensional closure | G5 Freshness | G6 Coverage consistency | G7 Covariance consistency | G8 Uniqueness & acyclicity.
• Trigger S1–S5 (dimension/freshness/path/covariance/citation) to halt subsequent stages and log audit.

IX. Machine-Readable Configs
A. eval/compare_spec.yaml (excerpt)

version: "1.0.0"

coverage: { mode: "k", k: 2 } # k|alpha|quantile

metrics:

mae: { tolerance: 1.0e-4 }

auc: { tolerance: 2.0e-3 }

r_phi: { lb95_min: 0.60 }

delta_t_arr_s: { guard: "tau_T_s" }

epsilon_flux_p95: { guard: 0.02 }

latency_p95_s: { guard: 0.200 }

rules:

interval_overlap_required: true

same_coverage_band_required: true

B. tools/compare.py Output (example)

{

"decision": "pass",

"deltas": { "MAE": 1.0e-5, "Latency_P95_s": 0.006 },

"intervals": { "r_phi_ref":[0.61,0.80], "r_phi_repro":[0.62,0.79], "overlap": true },

"gates": { "G1": true, "G2": 0.94, "G3": true, "G4": true, "G5": true, "G6": true, "G7": true, "G8": true }

}

X. Anti-Patterns & Fixes

• Anti: means without intervals → Fix: add U = k·u_c or quantile bands with convergence diagnostics.
• Anti: T_arr = ∫ n_eff / c_ref d ell (no parentheses) → Fix: unified parenthesized form.
• Anti: missing delta_form or unequal path arrays → Fix: complete in alignment and equalize lengths.
• Anti: cross-volume coverage mismatch → Fix: unify single coverage.mode and declare in manifests & captions.
• Anti: artifacts’ sha256 change after resume → Fix: restore RNG/optimizer/AMP states and re-verify.

XI. Cross-References

• Ch. 3 (Layout & Artifacts), Ch. 4 (Env Lock), Ch. 5 (Data Snapshot), Ch. 6 (Weights/Params), Ch. 7 (Scripts & Commands), Ch. 8 (Seeds & Determinism), Ch. 9 (Metrics & Gates).
• Model Card Ch. 6/7/8/10/12; Error Budget Card Ch. 8/9; Dataset Card Ch. 11; Pipeline Card Ch. 12.

XII. Checklist

• preflight → data_verify → train → infer → align → eval → compare → pack run once end-to-end; all artifacts’ sha256 recorded.
• Path alignment explicit gamma/measure/delta_form; len(path) ≥ 2, Δell compliant; evaluate phase within reference window; p_dim = 1.0.
• Metrics point + interval satisfy compare_spec.yaml thresholds/tolerances; convergence diagnostics complete; cross-platform results remain in the same coverage band.
• /validate passes G1–G8; report_manifest.yaml / checksums.txt / SIGNATURE.asc updated; audit.jsonl complete.
• Non-compliances tagged [Restricted] with diagnostics and remediation path.