11-EFT.WP.Core.DrawingKinetics v1.0 | Chapter 13 Reproducibility Experiments and Compliance

Home ／ Docs-Technical WhitePaper ／ 11-EFT.WP.Core.DrawingKinetics v1.0

Chapter 13 Reproducibility Experiments and Compliance

I. Scope and Objectives

• Establish an end-to-end framework for reproducibility and compliance so that, after locking env.*, pipe.card, param.card, data.digest, timebase.{alpha,beta}, and TS.* clauses, any node can rerun and obtain results—T_fil(t), v(t), lambda(t), s(t), A(t), spectral metrics, and scores (see Chapter 12)—that are consistent within thresholds.
• Specify the fields, workflow, and audit trail for the “pipeline card” and the “reproducibility manifest,” and define compliance tiers and pass criteria.
• Materialize this chapter as Mx-15 reproduce-and-audit, chained with Mx-10..14, yielding a publishable repro.bundle.

II. Terminology and Symbols

1. Reproducibility artifacts
   • pipe.card: pipeline card declaring the methods and order for phases { Mx-10 , Mx-11 , Mx-12 , Mx-13 , Mx-14 }.
   • param.card: parameter card containing theta, gates, weights, and random seeds.
   • repro.list: reproducibility run list that enumerates bench.id, window.id, gamma(ell), and device sets to rerun.
   • env.lock: environment lock recording container image fingerprints, OS, libraries, and hardware traits.
   • data.digest = H(data.slice): hash of the data slice; H(•) is a fixed hash such as sha256.
   • img.digest = H(container.image), code.digest = H(repo.commit), param.digest = H(param.card).
2. Compliance model
   • C.level ∈ { L1 , L2 , L3 }: compliance level.
   • E[*]: evidence set; sig: artifact signature (algorithm and public-key reference).
   • Reproducibility distance D_rep, spectral distance d_spec, scoring delta d_score.
3. Time and causality
   tau_mono and ts = alpha + beta * tau_mono (Chapter 8); TS.hb is causality heartbeat evidence (Chapter 10).
4. Collision rules
   T_fil must never be confused with T_trans; n must never be confused with n_eff (global mandate).

III. Postulates and Minimal Equations

• P11-50 (manifest completeness)
  Given { pipe.card , param.card , env.lock , data.digest , timebase.{alpha,beta} }, inputs to the workflow function family are complete and allow deterministic replay (subject to the tolerance clauses below).
• P11-51 (time-base consistency)
  Cross-node comparisons are always made on ts; map tau_mono -> ts and record alpha, beta prior to any comparison.
• P11-52 (unit and dimension conservation as a precondition)
  If check_dim(expr) fails or gate.mass/gate.norm fails, the reproducibility run is invalid.
• S12-50 (reproducibility distance)
  Define the reproducibility distance for runs A, B on the same window.id:
  D_rep = max( D_T , D_v , D_lambda , D_s , D_A )
  where D_T = rmse( T_A , T_B ) / T_ref, rmse(x,y) = sqrt( mean_t ( x(t) - y(t) )^2 ); others are constructed analogously.
• S12-51 (spectral distance)
  d_spec = ( ∫ | log( S_A(f) ) - log( S_B(f) ) | d f ) / ( ∫ d f ), with S_xx(f) as in Chapter 7 and identical window, ENBW, and U_w.
• S12-52 (score consistency)
  d_score = | score.total_A - score.total_B |; compute sub-score deltas likewise and include them in the audit.
• S12-53 (acceptance criteria)
  A reproducibility trial passes iff
  D_rep <= gate.reprod.dist and d_spec <= gate.reprod.spec and d_score <= gate.reprod.score and TS.hb.violations == 0.
• S12-54 (hash chain and fingerprint)
  fingerprint = H( code.digest || img.digest || data.digest || param.digest || H(alpha || beta) ), with || denoting concatenation.

IV. Data and Manifest Gauges

1. pipe.card minimal fields
   • stages = [Mx-10, Mx-11, Mx-12, Mx-13, Mx-14]; each includes method.id, args, version.hash, TS.* requirements, and fallback strategy.
   • path.gamma: parameterization and support of gamma(ell); measure = d ell.
   • If the two arrival-time gauges T_arr are used, produce both in parallel and report delta_form with the path manifest (Chapter 8).
2. param.card minimal fields
   theta = { K_el , K_vis , theta_aux[*] }; gates = { gate.mass , gate.norm , gate.reprod.* , gate.spectrum.leak }; weights = { w_c , w_s , w_o , w_t }; seed.master and seed.policy.
3. env.lock minimal fields
   container.image and img.digest; OS.kernel; lib.*@version; cpu/gpu traits and fp.mode; clock.source.
4. repro.list minimal fields
   bench.id, bench.type, window.id, env.target, devices[*], repeat (repeat count).
5. Output traceability
   fingerprint, sig, TS.hb.digest, timebase.{alpha,beta}, delta_form (if T_arr is used).

V. Algorithms and Implementation Bindings

1. I10-20 build_manifest( pipe_card:dict , param_card:dict , env_probe:dict ) -> Manifest
   Aggregate the manifest and compute fingerprint; return E_CARD_INCOMPLETE if any required field is missing.
2. I10-21 lock_environment( manifest:dict ) -> LockReport
   Pull the container and verify img.digest and library versions; return E_ENV_MISMATCH on failure.
3. I10-22 reproduce_run( manifest:dict , repro_list:dict ) -> ReproReport
   • Execute on the declared repeat counts and device set; output y_pred, y_obs, TS.*, and S_xx(f).
   • Return E_THREADS_CAUSALITY if TS.hb.violations > 0.
4. I10-23 compare_runs( runA:any , runB:any , gates:dict ) -> Verdict
   Compute D_rep, d_spec, d_score and compare to gate.reprod.*; on pass=false list the quantity and time index of the worst deviation.
5. I10-24 sign_bundle( bundle:any , keyref:any ) -> Sig
   Sign the fingerprint and core files; return E_SIGNING_FAIL on error.
6. I10-25 emit_compliance_record( verdict:dict , evidence:any ) -> dict
   Generate COMPLIANCE.md and audit-trail index; idempotent.
7. Pseudocode (main flow of Mx-15 reproduce-and-audit)

Mx-15():

manifest <- I10-20.build_manifest(pipe.card, param.card, env.probe)

lock <- I10-21.lock_environment(manifest)

assert(gate.mass && gate.norm) # Chapter 11

runs <- I10-22.reproduce_run(manifest, repro.list)

verdicts <- []

for (A,B) in pairwise(runs):

verdicts += I10-23.compare_runs(A,B, gates.reprod)

bundle <- pack(manifest, runs, verdicts, methods.yaml, reports.json)

sig <- I10-24.sign_bundle(bundle, keyref)

record <- I10-25.emit_compliance_record(aggregate(verdicts), evidence={sig,TS.hb.digest})

return {bundle,bundle.sig: sig, compliance: record}

VI. Metrology Flow and Run Graph

1. Step-by-step flow
   • Environment probe and lock: produce env.lock; confirm clock.source and fp.mode.
   • Manifest validation: check consistency among pipe.card, param.card, repro.list, and data.digest.
   • Pre-gates: run check_dim(expr) and enforce gate.mass/gate.norm.
   • Re-runs: execute per repro.list across devices and repeats, enforcing seed.master and seed.policy.
   • Comparison: compute D_rep, d_spec, d_score and delta_form (if T_arr applies).
   • Audit & signing: produce fingerprint, sig, TS.hb.digest, and COMPLIANCE.md.
   • Publish & archive: emit repro.bundle to the lake; record version.hash and change entries.
2. Fallback strategies
   • On E_ENV_MISMATCH: roll back to the last passing img.digest or restore library versions.
   • If D_rep exceeds the gate: inspect timebase.{alpha,beta}, TS.hb, and fp.mode; if needed, pin BLAS threads and disable nondeterministic operators.
   • If d_spec exceeds the gate: align U_w, ENBW, and window boundaries and recompute spectra.

VII. Verification and Test Matrix

1. Minimum required
   • Same-machine, same-container repeats: repeat >= 3, must satisfy D_rep <= gate.reprod.dist_intra.
   • Cross-machine, same-container repeats: cover at least two CPU/GPU classes, D_rep <= gate.reprod.dist_cross.
   • Minor container version: when img.minor changes without touching numerical kernels, d_score <= gate.reprod.score_minor.
   • Spectral consistency: boundary-aligned window.id tests with d_spec <= gate.reprod.spec.
2. Edge and extreme
   High draw ratios and steep s(t) slopes; low-SNR heavy-tailed noise; sensor packet loss and resampling; compensation strategy for intermittent TS.hb desynchronization.
3. Statistical power
   Per Chapter 11 gauges, state the sample size and power targets for reproducibility trials; record CI_p and power pass lines.

VIII. Cross-References and Dependencies

• Chapter 2: measures of lambda, s, v, A and path gamma(ell) for manifest and window reuse.
• Chapter 3: rho_L, J conservation gates as prerequisites for Mx-15.
• Chapter 4: T_fil constitutive families and parameter theta in param.card and comparison gauges.
• Chapter 5: bc.* inlet/outlet, clamping, and slip models recorded in pipe.card.
• Chapter 7: S_xx(f), U_w, ENBW define d_spec.
• Chapter 8: Mx-11..13 calibration and audits as prerequisites; delta_form reporting rules reused.
• Chapter 10: TS.* and hb audits are mandatory compliance evidence.
• Chapter 11: error budgets and gate thresholds inform gate.reprod.*.
• Chapter 12: bench.id and scoring fields reused for reproducibility evaluation.

IX. Risks, Limits, and Open Questions

• Sources of numerical nondeterminism (parallel reduction order, GPU atomics, nondeterministic ops) can amplify small deviations; declare them explicitly in env.lock and assess impact in COMPLIANCE.md.
• Floating-point environment differences (fp.mode, FMA) and BLAS thread counts affect D_rep; default to threads=1 and disable auto-scaling.
• The two T_arr conventions can differ systematically due to discrete pathing and interpolation; mandate delta_form and detailed path publication.
• Device clock and ts mapping may be under-calibrated across nodes, yielding false mismatches; include timebase.qual confidence intervals.

X. Deliverables and Version Management

1. Deliverables list
   • repro.bundle.tar: contains manifest.json (pipe.card, param.card, env.lock, repro.list), runs/*, verdicts.json, methods.yaml, bench.report.json, TS.hb.digest, timebase.{alpha,beta}, delta_form.json (if applicable).
   • COMPLIANCE.md: compliance level, gates, evidence, and deviation notes.
   • SIGNATURE.asc: digital signature of the fingerprint and public-key reference.
   • CHANGELOG.md: ADD/MOD/FIX/PERF/SEC/DOC records and migration guidance.
2. Compliance levels and pass conditions
   • L1 (record level): manifest complete, gate.mass/gate.norm pass; allow small D_rep overruns with rationale and remediation plan.
   • L2 (engineering level): cross-machine reproducibility within the same container passes, D_rep <= gate.reprod.dist_cross, d_spec and d_score within gates; signatures and audit trail complete.
   • L3 (publication level): passes on the declared hardware matrix, TS.hb.violations == 0, fingerprint and SIGNATURE.asc verifiable; when T_arr applies, delta_form within gates and co-published.
3. Version policy
   • Any change that affects score.total comparability must bump schema.version and run a parallel dual-compute period with a migration script; minor versions are limited to documentation and non-material gauge edits.
   • Re-run Mx-15 prior to release and archive the hash list and signatures of the repro.bundle.