12-EFT.WP.Methods.Repro v1.0 | Chapter 10 Cross-Domain / Cross-Laboratory Reproducibility

Home ／ Docs-Technical WhitePaper ／ 12-EFT.WP.Methods.Repro v1.0

Chapter 10 Cross-Domain / Cross-Laboratory Reproducibility

I. Scope and Objectives

• Establish a cross-site / cross-laboratory (hereafter “Site”) framework for reproducibility assessment that spans study design, transport, head-to-head comparisons, equivalence testing, discrepancy root-cause analysis, and report publication. The full workflow is governed by EnvLock, PipelineCard, ParamCard, and the audit trail.
• Objective: after completing time-base alignment ts = alpha + beta * tau_mono and enforcing the environment lock, ensure that outputs from different Sites, y_A and y_B, satisfy the equivalence gates: delta_rep <= gate.rep, with spectral consistency and run-time indicators meeting the pass criteria of Chapters 8 and 9.

II. Terms and Symbols

• Site: an experimental station or laboratory; K_labs is the number of participating Sites.
• theta_site: Site-specific calibration parameters (e.g., a_s, b_s for affine or more general nonlinear mappings).
• b_bias: systematic bias; sigma_within / sigma_between: within-Site / between-Site variance.
• xi_transport: transport disturbance factors (temperature & humidity, vibration, EM environment, clock-sync state).
• TOST(alpha_eq, delta_eq): two one-sided tests configuration with significance alpha_eq and equivalence margin delta_eq.
• r_tb: time-base consistency metric (see Chapter 9) used for cross-Site window-alignment control.
• EnvLock, PipelineCard, ParamCard, TS.*, seed, hash(•), fingerprint: as defined throughout this volume.
• Collision notes: do not mix T_fil with T_trans; strictly distinguish n from n_eff. When arrival time T_arr is involved, compute the two gauges in parallel and publish delta_form.

III. Postulates and Minimal Equations

1. P31-18 Site-normalization invariance postulate
   With fixed EnvLock, seed, PipelineCard, ParamCard, and completed alignment ts = alpha + beta * tau_mono, cross-Site output-distribution differences can be absorbed by a finite-dimensional calibration theta_site; residuals are bounded by the equivalence gates.
2. P31-19 Transport stability postulate
   Compliant transport & hand-off procedures constrain signal distortion within the spectral gate: ( ∫ | S_xx^A(f) - S_xx^B(f) | d f ) <= tau_psd_transport, and this is verifiable in the audit trail.
3. S32-33 Bias-budget decomposition
   • b_bias^2 = b_env^2 + b_instr^2 + b_proc^2 + b_transport^2
   • sigma_between^2 = sigma_site^2 + sigma_interact^2
   • Estimate components by least squares or robust regression and report confidence intervals.
4. S32-34 Site-calibration mapping
   • Baseline: y_cal = a_s + b_s * y; generalized: y_cal = g( y ; theta_site ).
   • Fit theta_site on reference set R, and validate on an independent control set C via delta_rep( y_cal^A , y^B ).
5. S32-35 Equivalence and pass gates
   • Numerical equivalence: delta_rep = ( norm( y_A - y_B ) / norm( y_B ) ) <= delta_eq
   • Spectral equivalence: delta_psd = ( ∫ | S_xx^A(f) - S_xx^B(f) | d f ) / ( ∫ S_xx^B(f) d f ) <= tau_psd
   • Statistical equivalence: TOST(alpha_eq, delta_eq) rejects both one-sided nulls
   • Time-base consistency: r_tb <= tau_tb.

IV. Data and Manifest Gauges

1. Minimum Site metadata
   site.id, site.loc (coarse locale info), instrument.model, instrument.firmware, os/kernel/driver, EnvLock.fingerprint, clock.sync (NTP/PTP state), rng.family & seed, temperature/humidity, vibration, em_field.
2. Transport and hand-off
   • TransportBundle = { samples:[ uri , hash ] , packaging.hash , sensors.trace , custody.chain , timestamps , H_k , sig_k }.
   • Each hand-off creates an event action = transfer, chained as H_k = hash( H_{k-1} || id_k ) with signature sig_k.
3. Cards and windows
   Fix PipelineCard.hash, ParamCard.hash, window function with U_w and ENBW, statistical window T_obs, and time-base mapping { alpha, beta }.
4. Dual arrival-time gauges (when applicable)
   Publish both T_arr gauges with delta_form, and declare the path gamma(ell) with measure d ell.

V. Algorithms and Implementation Bindings

1. I30-12 plan_interlab_study(design:dict) -> StudyPlan
   • Produce a stratified randomization and blocking scheme (Sites × batches × devices).
   • Specify T_obs, delta_eq, tau_psd, tau_tb, and sample sizes.
   • Output transport, sampling, and audit-trail checkpoints.
2. I30-13 align_cross_site(traceA:any, traceB:any) -> {alpha:float, beta:float, r_tb:float}
   • Estimate alpha, beta from synchronization markers.
   • Compute r_tb and alignment diagnostics.
   • Emit an audit event action = timebase_align.
3. I30-14 fit_site_calibration(R:dict) -> theta_site
   • Fit theta_site (affine or generalized g(•)) on R.
   • Cross-validate and emit stability / drift alerts.
4. I30-15 test_equivalence(A:any, B:any, cfg:dict) -> EqReport
   • Compute delta_rep, delta_psd.
   • Run TOST(alpha_eq, delta_eq).
   • Synthesize EqReport = { pass:bool , pvalues , deltas , r_tb }.

VI. Metrology Flows and Run Graph

1. Mx-50 design-and-randomize
   • Ingest the Site roster and resource constraints.
   • Emit the StudyPlan with sampling/transport schedules.
2. Mx-51 transport-and-intake
   • Package samples and environment telemetry.
   • Create the TransportBundle and hand-off events.
   • Site intake verifies hash(•) and fingerprint.
3. Mx-52 align-and-calibrate
   • Call I30-13 to align ts.
   • Fit theta_site and generate y_cal.
   • Compute S_xx(f), delta_psd, delta_rep.
4. Mx-53 decide-and-report
   • Execute I30-15 equivalence tests.
   • If not passed, enter discrepancy root-cause: update the bias budget S32-33 and localize dominant contributors.
   • Produce the InterLabReport and public announcement package.

VII. Verification and Test Matrix

1. Minimum required
   • Alignment correctness: with synthetic data having known alpha_true, beta_true, require I30-13 estimation error below threshold.
   • Equivalence power: when the true difference <= delta_eq/2, TOST pass rate meets the target 1 - beta.
   • Spectral conservation: inject controlled transport disturbances xi_transport; delta_psd responds monotonically with disturbance magnitude.
2. Boundary and extreme cases
   • Clock-drift jumps causing r_tb > tau_tb must block the flow and raise alerts.
   • Device-firmware differences that increase b_instr must be attributed to the corresponding component in S32-33.
3. Reproducibility exercises
   • Multi-Site reruns (K_labs >= 3): after cross-calibration, R_coef remains within the target band.
   • Double-blind control: hide Site labels; verify statistical equivalence independent of human adjudication.

VIII. Cross-References and Dependencies

• Chapter 5 Environment & Dependency Locking (EnvLock); Chapter 6 Time Base & Randomness Control (alpha, beta, r_tb, seed); Chapter 7 Card Structures (PipelineCard / ParamCard); Chapter 8 Benchmark & Scoring (score, delta_psd, R_coef); Chapter 9 Audit Trail & Release (H_k, sig_k, ReleaseBundle).
• Core.DataSpec ingestion and field gauges; Core.Metrology spectral gauge S_xx(f), U_w, ENBW; Core.Threads run-time TS.*.

IX. Risks, Limits, and Open Questions

• Risks
  Hidden interaction terms (Site × sample) inflate sigma_interact; unobserved environmental factors leak into b_env; single points in the transport chain bias xi_transport.
• Limits
  In strongly nonlinear or time-varying systems, low-dimensional theta_site may not absorb all differences; under very long chains, r_tb estimation may be limited by external sync accuracy.
• Open questions
  Data-driven setting of adaptive equivalence margins delta_eq; unified weighting for cross-domain dual T_arr discrepancies delta_form; joint optimization of spectral gates and task-level score.

X. Deliverables and Version Management

1. Deliverables
   • StudyPlan (design & sample-size calculation, randomization & blocking).
   • TransportBundle (packaging, telemetry, custody & signatures).
   • AlignmentReport (alpha, beta, r_tb and diagnostics).
   • CalibrationProfile (theta_site with stability assessment).
   • InterLabReport (delta_rep, delta_psd, TOST results, and bias budget).
   • Public announcement package with verification guide (H_k, sig_k, verifier.pk_ref).
2. Version policy
   • Site add/drop or device replacement requires a minor version bump and rerun of Mx-50 → Mx-53.
   • Any failed window and rollback must append audit events without overwriting history.
   • Long-term LTS equivalence certificates require periodic re-validation and a rolling public-key trust chain.