21-EFT.WP.Metrology.Sync v1.0 | Chapter 8 — Estimating Offset / Skew / Jitter (Robust & Sequential)

Home ／ Docs-Technical WhitePaper ／ 21-EFT.WP.Metrology.Sync v1.0

Chapter 8 — Estimating Offset / Skew / Jitter (Robust & Sequential)

One-line objective: Provide a unified estimation-and-release framework for offset, frequency offset skew (ppm), and jitter J that combines robust statistics with sequential filtering (α–β / Kalman / adaptive gating), so that online synchronization remains SLO-compliant and auditable under anomalies and drift.

I. Scope & Objects

1. Targets
   • End-to-end offset, skew (ppm), and J estimation for PTP/NTP/SyncE/White Rabbit sessions.
   • Hardware/software/hybrid timestamps; unicast/multicast/BC scenarios; batch and streaming online estimation.
2. Inputs
   • Events & timestamps: t1,t2,t3,t4, hwts ∈ {0,1}, session id, profile_id.
   • Windows & bases: sliding window W_k on tau_mono, step Delta_t.
   • Delay & asymmetry model (from Ch. 7): d_up, d_down, asym for bias correction.
   • Quality signals: q_len, rho, J_ref.
3. Outputs
   • Estimands & uncertainty: hat_offset, hat_skew_ppm, hat_J, with intervals or U = k * u_c.
   • Servo injections: offset_corr (with asym correction), online auto-tuning of process noise Q and measurement noise R.
   • Audit & manifest: manifest.sync.est.* (models, parameters, thresholds, P95/P99, signature).

II. Terms & Variables

1. Random terms & state
   • y_k: k-th bias-corrected observation.
   • theta_k: true offset; s_k: frequency slope d theta / dt.
   • e_k: measurement noise; w_k: process noise; J: short-term scatter of offset.
   • State vector x_k = [ theta_k, s_k ]^T.
2. Timing & windows
   • t_k on tau_mono; Delta_t = t_k - t_{k-1}; W_k = {y_{k-m+1}..y_k}.
   • Sliding stats: MAD, IQR, P95/P99.
3. Units & dimensions
   unit(theta)="s", unit(J)="s", unit(s)="1" (often published as ppm); dim(*)="[T]" or dimensionless.

III. Axioms P608- **

• P608-1 (Linear state–observation locally): Over short horizons, theta_{k+1} ≈ theta_k + s_k * Delta_t.
• P608-2 (Unified time-base): Estimate on tau_mono, publish on ts with offset/skew/J.
• P608-3 (Robust-first): Obtain coarse estimates & scales by robust statistics, then drive sequential filters; prevent anomalies from integrating into the filter.
• P608-4 (Adaptive noise): Tune Q,R online, driven by J and residual gating.
• P608-5 (Traceable & dimensional): check_dim(expr) must pass; estimators/thresholds/budgets are reproducible and signed.
• P608-6 (Dual-form commitment): If observations depend on arrival time, record both T_arr forms and delta_form in parallel.

IV. Minimal Equations S608- **

• S608-1 (Symmetric offset & RTT)
  offset_sym = ( ( t2 - t1 ) - ( t4 - t3 ) ) / 2
  delay_sym = ( ( t2 - t1 ) + ( t4 - t3 ) ) / 2
  offset_corr = offset_sym + asym
• S608-2 (Observation & state evolution)
  y_k = theta_k + e_k
  theta_{k+1} = theta_k + s_k * Delta_t + w_{theta,k}
  s_{k+1} = s_k + w_{s,k}
• S608-3 (Robust scale & gating)
  MAD = median( | y_i - median(y) | ), sigma_hat ≈ 1.4826 * MAD
  Gate: accept y_k iff | y_k - median(W_k) | ≤ k_g * sigma_hat
• S608-4 (Skew estimation: weighted/robust slope)
  WLS slope: s_hat = ( ∑ w_i (t_i - t̄)(y_i - ȳ) ) / ( ∑ w_i (t_i - t̄)^2 )
  Theil–Sen: s_hat = median_{i<j} ( (y_j - y_i) / (t_j - t_i) )
  skew_ppm = 1e6 * s_hat
• S608-5 (Huber M-estimation)
  Residual r_i = y_i - ( a + s * t_i ), standardized u_i = r_i / sigma_hat;
  ψ(u) = u (|u| ≤ c)；ψ(u) = c * sign(u) (|u| > c)；
  Solve iteratively ∑ ψ(u_i) = 0 and ∑ t_i ψ(u_i) = 0 to update a,s.
• S608-6 (α–β filter)
  Predict: theta_{k|k-1} = theta_{k-1} + s_{k-1} * Delta_t
  Residual: r_k = y_k - theta_{k|k-1}
  Update: theta_k = theta_{k|k-1} + α * r_k
  s_k = s_{k-1} + ( β / Delta_t ) * r_k
• S608-7 (Second-order Kalman)
  A = [[1, Delta_t],[0,1]], H = [1,0]
  Predict: x_{k|k-1} = A x_{k-1}, P_{k|k-1} = A P_{k-1} A^T + Q
  Gain: K_k = P_{k|k-1} H^T ( H P_{k|k-1} H^T + R )^{-1}
  Update: x_k = x_{k|k-1} + K_k ( y_k - H x_{k|k-1} ), P_k = (I - K_k H) P_{k|k-1}
• S608-8 (Jitter estimation & mapping)
  J_k^2 = (1 - beta_J) * J_{k-1}^2 + beta_J * r_k^2
  Alt. diff approximation: J_k^2 ≈ 0.5 * E[ ( y_k - y_{k-1} )^2 ]
  Short-τ mapping to Allan: sigma_y^2(τ) ≈ ( J^2 ) / ( 2 * τ^2 )
• S608-9 (Sequential change detection)
  Standardize z_k = r_k / sigma_hat;
  g_k^+ = max( 0, g_{k-1}^+ + z_k - nu ), g_k^- = max( 0, g_{k-1}^- - z_k - nu )
  Trigger when max( g_k^+, g_k^- ) ≥ h → declare drift/step, reset or inflate Q.

V. Statistical Flow M60-8 (Robust + Sequential Estimation Loop)

1. Ready & checks
   • Align to tau_mono; build W_k; verify unit/dim and timestamp integrity; persist TraceID.
   • Apply asym correction from Chapter 7 to obtain y_k.
2. Robust coarse estimation
   • hat_offset_init = median(W_k); sigma_hat = MAD → gate threshold k_g.
   • hat_skew_ppm_init via Theil–Sen or WLS+Huber.
3. Initialize sequential filter
   • Seed x_0, P_0 from coarse estimates; initialize R from J_ref or sigma_hat, Q from historical drift.
   • Choose α–β or Kalman; prefer smaller R under HW timestamps.
4. Online update & adaptation
   • Gate new observations; update filter state and J_k.
   • Adapt Q,R from residual stats; inflate R under rising load rho.
   • Output hat_offset / hat_skew_ppm / hat_J with U.
5. Change detection & action
   • Run CUSUM/GLRT; upon trigger, temporarily raise Q or reset state; enter holdover if needed.
   • Log audit events: change_point, Q/R trajectories.
6. Publish & sign
   • Emit manifest.sync.est.*: model, window, α/β or Q/R, P95/P99, failure counts, signature.
   • Provide offset_corr and process parameters for the servo (Chapter 6).

VI. Contracts & Assertions

• C60-81 (Dimensions): check_dim(hat_offset) = [T], check_dim(hat_J) = [T].
• C60-82 (SLOs): |hat_offset|_p99 ≤ tol_offset, |hat_skew_ppm|_p99 ≤ tol_skew_ppm, hat_J_p99 ≤ J_budget.
• C60-83 (Stability): cond(P_k) ≤ kappa_max; on divergence, down-weight or reset.
• C60-84 (Outlier rate): outlier_rate ≤ tol_outlier; if exceeded, escalate robustness (Huber → Tukey biweight).
• C60-85 (Dual-form alignment): for arrival-dependent observations, delta_form_p99 ≤ tol_Tarr.
• C60-86 (Lock times): T_lock ≤ T_lock_slo, and after unlock T_relock ≤ T_relock_slo.

VII. Implementation Bindings I60- (Estimators & APIs)*

• prep_observation(exchange, asym_model) -> y_k, Delta_t, meta
• estimate_init_robust(Y, T) -> {hat_offset_init, hat_skew_ppm_init, sigma_hat}
• fit_huber_wls(Y, T, c) -> {a, s, sigma_hat}
• alpha_beta_init(alpha, beta, Q, R, x0, P0) -> filter / kalman_init(A,H,Q,R,x0,P0) -> filter
• filter_step(filter, y_k, Delta_t, gate) -> {x_k, P_k, r_k, J_k}
• adaptive_qr_tuner(residuals, rho, policy) -> {Q', R'}
• detect_change(residuals, nu, h) -> flags
• emit_est_manifest(est, policy) -> manifest.sync.est
• Invariants: reproducible(seed); gated records replayable; units/dimensions valid; delta_form ≤ tol_Tarr (when applicable).

VIII. Cross-References

• Timestamps & link discipline: Chapter 4.
• Protocol implementations & fields: Chapter 5.
• Servo coupling & gain scheduling: Chapter 6.
• Delay & asymmetry models: Chapter 7.
• Allan family & frequency-stability gauges: EFT.WP.Metrology.TimeBase v1.0 Chapter 7.
• Anomaly & drift governance: Methods.Cleaning v1.0 Chapter 8.
• Uncertainty propagation: Methods.CrossStats v1.0 Appendix E.

IX. Quality SLIs & Risk Control

1. Metrics: offset_p50/p95/p99, skew_ppm_p95/p99, J_p95/p99, outlier_rate, gate_ratio, lock_time, relock_time; filter health via trace(P_k), cond(P_k), R/Q ratio, change_events.
2. Risk actions:
   • If offset_p99 breaches → de-weight, tighten gates, and trigger routing/media review (Chapter 7).
   • If J_p99 spikes → increase R, narrow servo bandwidth, or enable WR/TSN scheduling.
   • For frequency steps → trigger CUSUM, temporarily raise Q and rate-limit; holdover if necessary.
   • If outlier rate rises → switch to a stronger robust estimator or lengthen the window.

Summary