16-EFT.WP.Methods.Cleaning v1.0 | Chapter 14 Quality Scoring, SLOs, and Audit

Home ／ Docs-Technical WhitePaper ／ 16-EFT.WP.Methods.Cleaning v1.0

Chapter 14 Quality Scoring, SLOs, and Audit

One-Sentence Goal
Close the quality loop—score → gate → release/rollback—by coupling measurable SLIs, enforceable SLOs, and a traceable audit chain.

I. Scope & Objects

1. Objects & I/O
   • Input: cleansed dataset ds', process metrics TS.sli.*, assertion report assert_report, uncertainties u(x), manifests manifest.*, version/tag tag, and trace identifier TraceID.
   • Output: quality score q_score, SLO evaluation result slo_ok, error budget EB and burn rate burn, audit log audit.log, rollback/freeze decisions and signatures.
2. Applicability
   Works for both offline batches and the streaming execution graph G=(V,E); all statistics are evaluated in the tau_mono window Delta_t and published on ts.

II. Terms & Variables

1. Quality score and components
   • q_score ∈ [0,1], aggregated from the component vector s = { s_cover, s_integrity, s_timeliness, s_consistency, s_contract } with weights w = { w_i }.
   • clip(z,a,b) truncates z to [a,b].
2. SLIs and SLOs
   • SLIs: TS.sli.lat_ms (end-to-end latency), TS.sli.err_rate (failure rate), TS.sli.fresh_age (data staleness), TS.sli.miss_ratio (missing share), TS.sli.delta_form (two-form gap), TS.sli.drift (drift magnitude).
   • Quantiles: P99(x) = inf { z : F_x(z) ≥ 0.99 }, similarly for P95(x).
   • SLO decision: SLO = ( P( metric ≤ target ) ≥ 1 - alpha ) or Px(metric) ≤ target.
3. Error budget and burn
   • Period budget: EB_total = alpha * N_req (by requests or by windows).
   • Consumed: EB_used = ∑ breach_events; remaining: EB = EB_total - EB_used.
   • Burn rate: burn = ( EB_used / EB_total ) / ( t / T_budget ).
4. Audit & signatures
   Hash chain: h_k = hash_sha256( h_{k-1} || record_k ), with seed h_0; each record includes ts, actor, action, object, delta, result, signature, TraceID.

III. Axioms (P114-*)

• P114-01 Explicit-metrics axiom
  All quality measurements must appear as named SLIs, annotated with Delta_t, measure, and sampling strategy.
• P114-02 Units & dimensions axiom
  SLIs must be comparable to their targets in unit/dimension: check_dim( metric - target ) = 0 (see Chapter 4).
• P114-03 Robust-quantiles axiom
  Prefer P95/P99 over means for SLOs; handle anomalies and missingness via m ∈ {0,1} and q_score down-weighting.
• P114-04 Traceable-audit axiom
  Any decision affecting publication must append to a signed hash chain, ensuring “changes are visible and rollbacks are provable.”
• P114-05 Error-budget governance axiom
  Govern with the single variable EB; excessive burn triggers degradation, throttling, and rollback (coordinated with Chapter 11 backpressure).

IV. Minimal Equations (S114-*)

• S114-01 Quality score aggregation
  Geometric (penalizes weakest link):
  q_score = clip( exp( ( ∑_i w_i * ln( max( s_i, eps ) ) ) / ( ∑_i w_i ) ), 0, 1 )
  Linear alternative: q_score = clip( ( ∑_i w_i * s_i ) / ( ∑_i w_i ), 0, 1 ).
• S114-02 Subscore mapping (smaller is better)
  Given metric x, threshold t, slope k:
  s(x; t, k) = 1 / ( 1 + exp( k * ( x - t ) ) ).
  “Interval-optimal” uses piecewise or dual-threshold mappings.
• S114-03 SLO decision (quantile-based)
  slo_ok(metric) = [ P99(metric) ≤ target ] ;
  Breach counter: breach = 1( P99(metric) > target ).
• S114-04 Error-budget burn
  burn = ( EB_used / EB_total ) / ( t / T_budget ), alert when burn ≥ burn_hi;
  Hard stop: EB ≤ 0 triggers forced rollback.
• S114-05 Audit hash chain
  h_k = hash_sha256( h_{k-1} || record_k ), verification: verify(signature, record_k).

V. Process (M10-14 Scoring—SLO—Audit)

1. Collect & standardize SLIs
   Produce TS.sli.*, unify units/dimensions; set Delta_t, sampling, unit for each.
2. Subscore mapping
   Map TS.sli.* to s_* ∈ [0,1] using S114-02; apply m and q_score for weighting/down-weighting missing/outlier segments.
3. Aggregation & thresholding
   Aggregate to q_score via S114-01; compute key quantiles P95/P99.
4. SLO decision & error budget
   Evaluate slo_ok with S114-03; update EB_used, EB, and burn per S114-04.
5. Decide & execute
   • If slo_ok ∧ EB > 0 ∧ assert_report.pass, enter freeze-candidate;
   • If burn ≥ burn_hi or EB ≤ 0, degrade/throttle and roll back (see Chapter 11);
   • Gate gray releases by q_score and EB.
6. Audit & signatures
   Append to audit.log, update h_k, sign, and cross-reference manifest; export quality_panel.
7. Persist & publish
   If gated, run freeze_release(ds', tag); otherwise quarantine and emit a remediation ticket change_ref.

VI. Contracts & Assertions

1. Scores & quantiles
   • q_score ≥ q_min ; P99(TS.sli.lat_ms) ≤ L_target ; P99(TS.sli.err_rate) ≤ E_target ;
   • P95(TS.sli.fresh_age) ≤ A_target ; P99(TS.sli.delta_form) ≤ tol_Tarr.
2. Error budget
   EB ≥ 0 ; burn ≤ burn_hi ; consecutive B windows with burn > burn_mid trigger degradation.
3. Audit & traceability
   verify_chain( audit.log ) = true ; manifest.signature verifiable ; TraceID consistent end-to-end.
4. Compliance coupling
   assert_contract.pass = true (see Chapter 10); dimensional and two-form arrival-time assertions must hold (Chapters 4 and 6).

VII. Implementation Binding (I10-14)

• compute_sli(ds', policy, window) -> TS.sli.*
  Produce standardized SLIs with window and sampling metadata.
• score_quality(TS.sli.*, weights, maps) -> s_*, q_score
  Apply mappings and aggregation; return per-component subscores and total score.
• assert_slo(TS.sli.*, targets, alpha) -> slo_ok, breaches
  Compute quantiles and evaluate SLOs; return breach events.
• update_error_budget(breaches, N_req, alpha, t, T_budget) -> EB, burn
  Maintain error budget and burn rate.
• emit_audit(record) -> h_k
  Append an audit record and return the latest hash head.
• rollback_release(tag, reason) -> status
  Trigger rollback and write audit; interoperate with Chapter 10 freeze APIs.
• publish_quality_panel(q_score, TS.sli.*, EB, burn) -> panel
  Produce the quality dashboard and visualization artifacts.

VIII. Cross-References

• Contracts & release freeze: Chapter 10.
• Time windows & quantiles: Chapter 5.
• Two forms & delta_form: Chapter 6.
• Anomalies & drift coordination: Chapter 8.
• Density & normalization quality: Chapter 13.
• Schema & traceability fields: EFT.WP.Core.DataSpec v1.0.

IX. Quality Metrics & Risk Control

1. Core panel
   • q_score, P99(TS.sli.lat_ms), P99(TS.sli.err_rate), P95(TS.sli.fresh_age), P99(TS.sli.delta_form), EB, burn, viol_rate.
   • Distribution monitoring: hist(lat_ms), cdf(err_rate); window parity: |metric_stream - metric_batch|.
2. Risk playbooks
   • If burn ≥ burn_hi: immediately degrade and throttle; switch to read-only publication.
   • If P99(lat_ms) repeatedly exceeds target: expand buffers or adjust scheduling priority (see Chapter 11).
   • If P99(delta_form) exceeds threshold: trigger arrival-time recomputation and environmental correction (see Chapter 12).
   • If the audit chain is inconsistent: freeze publication and force signature/TraceID review.

Summary
With standardized SLIs, formal SLOs, and an error-budget backbone, this chapter defines q_score aggregation and gating criteria, reinforced by a hash-chain audit and rollback mechanism. Together they form a measure–commit–trace quality loop. Outputs include quality_panel, audit.log, and EB/burn, with end-to-end verifiable compliance via manifest.signature and TraceID.