08-EFT.WP.Core.Sea v1.0 | Chapter 9 — Quality and Drift Monitoring

Home ／ Docs-Technical WhitePaper ／ 08-EFT.WP.Core.Sea v1.0

Chapter 9 — Quality and Drift Monitoring

I. Objectives and Scope

• Define a decomposable and aggregable q_score, spanning completeness, continuity, freshness, amplitude and noise, synchronization and arrival-time consistency, and other dimensions. Produce a unified set of quality indicators that are both persistable and alertable.
• Provide baseline modeling for drift, detection statistics, and multi-scale monitoring. Integrate with I80-7 quality_metrics / monitor_drift / raise_alert, and with Chapters 2/3/4/5/7/8 covering quantization, synchronization, filtering, spectra, and arrival-time models.

II. Observed Objects and Windows

• Windowing
  Delta_t: evaluation-window length; [t0, t1) is the window on tau_mono.
  N: samples per window; now_ts: audit timestamp (on ts).
• Core fields
  m ∈ {0,1}: missingness mask; x(t): calibrated signal; fs_nom: nominal sample rate.
  A_gain, B_bias, C_offset, DR, ENOB: from device & calibration (Chapters 1, 2).
  alpha, beta, J: clock skew/offset/jitter (Chapter 3); H(f) and tau_g(H) (Chapter 4).
  S_xx(f): PSD; BW, f_c: bandwidth and cutoff (Chapter 5).
  T_arr, delta_form: arrival time and dual-form gap (Chapter 8).

III. Postulates P89- (Quality and Monitoring)*

• P89-1 (Mask precedence)
  All statistics and features are computed with m as weight; unobserved samples are excluded or treated as missing.
• P89-2 (Clock consistency)
  All quality and drift evaluations are performed on tau_mono; audit and publishing are recorded on ts.
• P89-3 (Factorizable dimensions)
  q_score decomposes into weighted sub-scores s_i ∈ [0,1]; low sub-scores should be penalized more strongly.
• P89-4 (Traceable baselines)
  Drift judgments compare against a versioned baseline baseline@version; baseline source, window, and sample size must be traceable.
• P89-5 (Convention coherence)
  If q_score uses T_arr or H(f), the versions and conventions of c_ref, gamma(ell), d ell, H(f) MUST be specified.

IV. Minimal Equations S89- (Quality Dimensions and Sub-scores)*

• S89-1 (Completeness and freshness)
  comp = ( 1 / N ) * ∑_t m(t)
  fresh = exp( - ( now_ts - t1 ) / tau_fresh )
  s_comp = comp, s_fresh = min(1, fresh).
• S89-2 (Continuity and gaps)
  With sorted times tau_i, nominal spacing Delta_nom = 1 / fs_nom, gap_i = max( 0, tau_i - tau_{i-1} - Delta_nom ).
  max_gap = max_i gap_i, s_cont = exp( - max_gap / tau_gap ).
• S89-3 (Amplitude and clipping)
  clip_rate = ( 1 / N ) * |{ t | |x(t)| >= DR_lim }|
  s_clip = exp( - alpha_clip * clip_rate ).
• S89-4 (Noise and in-band SNR; spectral method)
  For signal band [f_lo, f_hi] ⊆ [0, BW] and a stop-band noise region [f_n1, f_n2]:
  P_sig = ( ∫_{f_lo}^{f_hi} S_xx(f) d f ), P_noise = ( ∫_{f_n1}^{f_n2} S_xx(f) d f ).
  SNR_dB = 10 * log10( P_sig / P_noise ); s_snr = 1 / ( 1 + exp( - k_snr * ( SNR_dB - thr_snr ) ) ).
• S89-5 (Sync and group-delay compensation quality)
  s_sync = exp( - |beta| / beta_thr ) * exp( - |alpha - 1| / alpha_thr ) * exp( - J / J_thr )
  s_fe = exp( - |tau_g(H) - tau_g_ref| / tau_fe_thr ).
• S89-6 (Arrival-time consistency)
  s_toa = exp( - delta_form / eps_form ) (if T_arr is unused, set s_toa = 1).
• S89-7 (Geometric aggregation)
  With weights w_i >= 0, ∑ w_i = 1, S = { s_comp, s_cont, s_fresh, s_clip, s_snr, s_sync, s_fe, s_toa }:
  q_score = exp( ∑_i w_i * ln( s_i ) ), clipped to [0,1].

V. Drift Measures and Tests

• Location & scale drift
  Delta_med = median_cur - median_base; z_robust = Delta_med / MAD_base.
  r_sigma = sigma_cur / sigma_base; r_gain = A_gain_cur / A_gain_base; r_bias = B_bias_cur - B_bias_base.
• Distributional drift (bounded statistics first)
  Jensen–Shannon: JSD(p,q) = 0.5 * D_KL( p || m ) + 0.5 * D_KL( q || m ), m = 0.5 * ( p + q ).
  D_KL( p || q ) = ∑_b p_b * ln( p_b / q_b ), with epsilon smoothing.
  KS = sup_x | F_p(x) - F_q(x) | (1-D).
• Spectral shape drift
  Delta_PSD = ( ∫_{0}^{BW} | S_xx_cur(f) - S_xx_base(f) | d f ) / ( ∫_{0}^{BW} S_xx_base(f) d f ).
  BER = ( P_sig_cur / P_noise_cur ) / ( P_sig_base / P_noise_base ).
• Rate and time-base drift
  fs_hat = ( N - 1 ) / ( tau_{N-1} - tau_0 ); drift_fs = | fs_hat - fs_nom | / fs_nom.
• Composite drift (dimensionless)
  D = w_loc * | z_robust | + w_scale * | ln( r_sigma ) | + w_shape * JSD_norm + w_spec * Delta_PSD + w_rate * drift_fs, with JSD_norm = JSD / ln(2).
  drift_flag = 1( D >= thr_drift ).

VI. Control Charts and Temporal Smoothing

• EWMA
  S_t = lambda * x_t + ( 1 - lambda ) * S_{t-1 }; z_t = ( S_t - mu_0 ) / sigma_0; trigger if |z_t| >= L.
• CUSUM
  C_t^+ = max( 0, C_{t-1}^+ + x_t - ( mu_0 + k ) ), C_t^- = max( 0, C_{t-1}^- + ( mu_0 - k ) - x_t ); alert if C_t^+ >= h or C_t^- >= h.
• Guidance
  Use EWMA for low-SNR/sparse samples; use CUSUM for small mean shifts. Chart inputs can be z_robust, ln(r_sigma), Delta_PSD, etc.

VII. Quality Gates and Action Matrix

• Decisions
  Pass if q_score >= q_pass; degrade if q_degrade <= q_score < q_pass; fail if q_score < q_degrade.
• Remediations
  Low s_sync: run I80-2 sync_clocks; re-estimate alpha, beta, J if needed.
  Low s_clip: reduce A_gain or increase DR; adjust H(f) if needed.
  Low s_snr: increase averaging, change window, or adjust bandwidth; check shielding/grounding.
  Low s_toa: recheck gamma(ell), c_ref, n_eff (Chapter 8); correct T_fe (Chapter 4).
  Persistent drift: trigger recalibration or baseline retraining; bump baseline@version+1.

VIII. Workflow Mx-9 (Closed-loop Quality and Drift)

• Select window [t0, t1) and manifest; load baseline@version.
• Compute core stats: comp, fresh, max_gap, clip_rate, SNR_dB, alpha/beta/J, tau_g(H).
• Build sub-scores S = { s_comp, s_cont, s_fresh, s_clip, s_snr, s_sync, s_fe, s_toa }; aggregate via S89-7 to q_score.
• Produce drift stats: z_robust, r_sigma, JSD_norm, Delta_PSD, drift_fs; compose D and drift_flag.
• Per Section VII thresholds and actions, call I80-7 raise_alert(kind,payload) if needed; record action_taken.
• Persist: update manifest and monitoring (Chapter 7) with {q_score, S, D, drift_flag, baseline_version, actions}.
• Adapt: if q_score is stable for K_pass windows and drift_flag=0, allow slow baseline updates (EWMA or sliding-window refresh).

IX. Interface Bindings and Suggested Fields

• I80-7 quality_metrics(data:any) -> dict
  Suggested output: {"comp":..., "fresh":..., "max_gap":..., "clip_rate":..., "SNR_dB":..., "s_comp":..., "s_cont":..., "s_fresh":..., "s_clip":..., "s_snr":..., "s_sync":..., "s_fe":..., "s_toa":..., "q_score":...}.
• I80-7 monitor_drift(baseline:any, current:any, fields:list[str]) -> dict
  Suggested output: {"z_robust":..., "r_sigma":..., "JSD_norm":..., "Delta_PSD":..., "drift_fs":..., "D":..., "drift_flag":...}.
• I80-7 raise_alert(kind:str, payload:dict) -> None
  kind ∈ {"quality_degrade","drift_detected","sync_fault","clipping","snr_low","toa_inconsistency"}; payload carries window and version metadata.
• I80-8 export_manifest(data:any) -> dict
  Add fields: {"q_score":..., "S":..., "D":..., "drift_flag":..., "baseline_version":..., "c_ref":..., "H_rev":..., "toa_delta_form":...}.

X. SLIs and Run Metrics (with Core.Threads and Chapter 7)

• Quality SLIs
  Quality.PassRate = mean( 1( q_score >= q_pass ) )
  Quality.P95 = P95( q_score )
  Quality.StaleRatio = mean( 1( fresh < fresh_min ) )
  Drift.AlertRate = mean( drift_flag )
• Windows and alert throttling
  Aggregate over SLA_window; throttle and coalesce like alerts to avoid storms.

XI. Default Parameters and Suggested Values

XII. Cross-Chapter Interlocks and Traceability

• With Chapter 2: ENOB/DR bound the ceilings of s_clip and s_snr; quantization noise contributes to P_noise.
• With Chapter 3: alpha/beta/J drive s_sync and drift_fs; all timing on tau_mono.
• With Chapter 4: H(f) and tau_g(H) feed s_fe; filtering reshapes energy in S_xx(f).
• With Chapter 5: spectral conventions determine the stability of P_sig/P_noise and Delta_PSD.
• With Chapter 7: dataset manifests and ingestion governance gate and audit with this chapter’s fields.
• With Chapter 8: delta_form directly enters s_toa and drives toa_inconsistency alerts.