14-EFT.WP.Methods.Inference v1.0 | Chapter 6: Online/Offline Consistency

Home ／ Docs-Technical WhitePaper ／ 14-EFT.WP.Methods.Inference v1.0

Chapter 6: Online/Offline Consistency

I. Scope & Objectives

1. Define and operationalize the consistency goals, metrology, and pass gates between offline batch inference and online real-time inference, covering time-base alignment, window replay, feature parity, model numeric conventions, and shielding strategies for runtime differences.
2. Target outputs
   • Consistency metric and thresholds: delta_offon, R_infer = 1 - delta_offon, tau_offon.
   • Feature parity and window replay conventions: delta_feat, window = {context, history, lookahead=0}.
   • Time-base alignment parameters: alpha, beta with the fitting report fit.
   • Operational playbooks and acceptance flow: executable steps and rollback conditions for Mx-54 → Mx-58.
3. Intended readers: data & feature engineering, model development, inference & scheduling platforms, SRE, and compliance teams.

II. Terms & Symbols

1. Outputs & consistency
   • y_hat_off (offline output), y_hat_on (online output),
     delta_offon = ( norm( y_hat_off - y_hat_on ) / norm( y_hat_off ) ), R_infer = 1 - delta_offon.
   • Decision consistency: err_decision = ( 1 / N ) * Σ 1[ argmax( y_hat_off ) != argmax( y_hat_on ) ].
2. Time base & windows
   • tau_mono (monotonic internal time), ts (published wall time), ts = alpha + beta * tau_mono.
   • Event vs. processing time: t_ev, t_proc; watermark; lateness threshold lateness_max.
   • Window replay window = {context, history, lookahead=0}; replay anchor anchor.
3. Features & lineage
   • Feature vector phi(x,t); feature parity
     delta_feat = ( norm( phi_off - phi_on ) / norm( phi_off ) ).
   • Versions & signatures: hash(•), fingerprint, EnvLock, ModelCard, PipelineCard.
4. Spectral & time-domain checks
   S_xx(f) (power spectral density), spectral discrepancy delta_psd, window function U_w and ENBW.
5. Concurrency & observability
   TS.latency, TS.thrpt, TS.error, hb (happens-before).

III. Postulates & Minimal Equations

• P41-2 Time-base alignment postulate
  For any trajectory trace and reference, comparisons are made after mapping ts = alpha + beta * tau_mono. Cross-device and cross-domain consistency must be defined and accepted on the common time base.
• P41-9 Window-replay postulate
  With a fixed window and identical dedup rules, lateness policy, and EnvLock, for the same anchor data slice the offline and online feature generation satisfies delta_feat <= tau_feat.
• P41-10 Inference-equivalence postulate (no-drift interval)
  When the input distribution shows no significant drift and the model graph, quantization, and numeric conventions follow ModelCard, then delta_offon <= tau_offon and err_decision <= tau_decision.
• S42-14 Consistency metric (unified vector/scalar form)
  On dataset D = { x_i }:
  delta_offon(D) = ( norm( Y_off - Y_on ) / norm( Y_off ) ), where Y_* = concat_i y_hat_*(x_i).
• S42-15 Probabilistic consistency
  If outputs are distributions p_off(y|x) and p_on(y|x), also report
  D_KL = ( 1 / N ) * Σ KL( p_off || p_on ) and bucketed ECE_off-on in parallel.
• S42-16 Spectral consistency
  For a time-series output z(t),
  delta_psd = ( ∫ | S_zz_off(f) - S_zz_on(f) | df ) / ( ∫ S_zz_off(f) df ),
  with window and measure explicitly declaring U_w and ENBW.
• S42-17 Arrival-time window consistency
  When arrival time T_arr is involved, compute both formulations in parallel and record the discrepancy delta_form:
  Constant-factored: T_arr = ( 1 / c_ref ) * ( ∫ n_eff d ell );
  General: T_arr = ( ∫ ( n_eff / c_ref ) d ell );
  consistency must be compared under the same gamma(ell) and d ell.

IV. Data & Manifest Conventions

• Minimal observation & comparison fields
  request_id, anchor, model.version, ModelCard.fingerprint, PipelineCard.fingerprint, EnvLock, ts, tau_mono, t_ev, t_proc, watermark, lateness, dedup_key, feature_fingerprint, rng.seed, rng_family, dtype_policy, quant_scheme, inputs.hash, y_hat_off, y_hat_on.
• Lateness & dedup conventions
  Dedup uses dedup_key with hb ordering. Records with lateness > lateness_max are labeled and either excluded from consistency statistics or analyzed separately.
• Units & dimensions
  ts unified to UTC or a specified timezone. TS.latency is in milliseconds. All normalized metrics are dimensionless; run check_dim(expr) prior to publication.

V. Algorithms & Implementation Bindings

1. Recommended prototypes & responsibilities
   • I40-8 align_timebase(trace:any, reference:any) -> {alpha:float, beta:float, fit:dict}: fit and validate the ts ↔ tau_mono mapping.
   • I40-10 compare_offline_online(off:any, on:any, policy:dict) -> ConsistencyReport: align windows and time base; compute delta_feat, delta_offon, err_decision, delta_psd and the pass verdict.
   • I40-7 monitor_drift(stream:any, spec:dict) -> DriftReport: pause consistency conclusions and enter rollback path if a drift alert fires.
   • I40-4 score_predictions(y_true:any, y_pred:any, metrics:dict) -> ScoreReport: joint reporting of consistency and accuracy.
   • I40-23 rebuild_feature_window(log:any, window:dict) -> batch: replay offline windows based on anchor and window.
   • I40-24 reconcile_dedup(stream:any, policy:dict) -> stream: unify dedup and lateness handling.
2. ConsistencyReport minimal fields
   delta_feat, delta_offon, R_infer, err_decision, delta_psd, alpha,beta,fit, coverage (valid sample ratio), policy (thresholds), pass (bool), notes.

VI. Metrology Flows & Run Diagram

• Mx-54 Alignment preparation
  Pin EnvLock and ModelCard. Pull off/on logs. Normalize timezone, check missing/exception policies, record coverage.
• Mx-55 Time-base fitting & window replay
  Call I40-8 to obtain alpha,beta,fit. Use I40-23 to replay offline windows. Align watermark and lateness_max; separate out-of-bound samples.
• Mx-56 Feature consistency check
  Compare features phi_off vs. phi_on layer-by-layer, compute and publish delta_feat. If delta_feat > tau_feat, block further release and enter rollback.
• Mx-57 Output parity & spectral check
  Compute delta_offon, err_decision, delta_psd. Generate reliability diagrams and ECE_off-on in parallel.
• Mx-58 Acceptance & rollback
  Pass if delta_offon <= tau_offon, err_decision <= tau_decision, delta_psd <= tau_psd, and fit.R2 >= r2_min. Otherwise roll back to the last stable channel and open an incident ticket.

VII. Verification & Test Matrix

1. Minimum required cases
   • Time-base alignment regression: fit alpha,beta on multi-segment traces, require fit.R2 >= r2_min.
   • Window replay correctness: construct data with lateness and duplicates; validate the unified conventions of I40-23 and I40-24.
   • Feature parity: set gates for delta_feat across four canonical feature types—standardization, discretization, temporal smoothing, and missing imputation.
   • Output parity: validate delta_offon (regression) and err_decision (classification).
   • Spectral parity: compute delta_psd for time-series outputs and publish U_w and ENBW.
   • Masking nondeterministic sources: verify that disabling rng and atomic* reduces delta_offon into the gate.
   • Exception propagation: unify handling for nan/inf, extremes, and empty windows.
2. Boundary & extreme scenarios
   Heavy lateness lateness >> lateness_max, reordering and bulk backfills; peak-time resource jitter causing TS.latency spikes; abrupt changes from quantization or kernel-algorithm switches.

VIII. Cross-References & Dependencies

IX. Risks, Limitations & Open Questions

• Backfill and replays can shift watermark/lateness distributions, making offline and online windows inherently non-comparable; these must be reported separately and excluded from parity stats.
• Compiler/runtime implicit rewrites, quantization drift, and fast_math divergences can inflate delta_offon; pin ModelCard.kernel_rev and dtype_policy.
• Distribution drift may pass consistency while accuracy drops; consistency does not replace accuracy/calibration acceptance—use jointly with ScoreReport and drift findings.
• Multi-tenant resource contention can induce time-dependent bias; condition analyses on TS.*.

X. Deliverables & Versioning

1. Deliverables
   • ConsistencyReport.json (containing delta_feat, delta_offon, R_infer, err_decision, delta_psd, alpha, beta, fit, coverage, pass),
   • AlignTimebaseReport.md,
   • WindowRebuildLog and DedupPolicy.yaml,
   • Snapshot fingerprints for the corresponding ModelCard, PipelineCard, and EnvLock.
2. Versioning policy
   Any change to window, dedup policy, lateness thresholds, dtype_policy, quant_scheme, kernel versions, or operator-fusion rules must bump the minor version, re-run Mx-54 → Mx-58, and register in Appendix C with
   fingerprint = hash( policy || window || kernels || quant ).