16-EFT.WP.Methods.Cleaning v1.0 | Chapter 2 Axioms and Minimal Equations

Home ／ Docs-Technical WhitePaper ／ 16-EFT.WP.Methods.Cleaning v1.0

Chapter 2 Axioms and Minimal Equations

One-Sentence Goal
Consolidate the cleaning domain’s prerequisite axioms and the minimal equation set, so all subsequent processes M10-* and implementation bindings I10-* share a single source of truth for criteria and citations.

I. Scope & Preconditions

1. Scope
   • Applies to three cleaning modes: offline batch, online services, and event streams, all governed by this chapter’s axioms and minimal equations.
   • Applies to time series, path-parameterized observations, event logs, scalar and tensor fields, and reference-environment records.
2. Preconditions
   • Unified numbering: axioms P10x-*, minimal equations S10x-*.
   • Unified time base and arrival-time convention: internal evaluation on tau_mono, public release on ts. Arrival time must compute both forms and record their difference.
   • Unified measure and units: every integral declares domain and measure explicitly, and every expression passes check_dim(expr) prior to release.

II. Symbols & Fixed Constants

• Time & synchronization: tau_mono, ts, offset, skew, J, Delta_t.
• Path & measure: gamma(ell), d ell, L_gamma.
• Arrival time & medium: T_arr, n_eff, c_ref, delta_form, tol_Tarr.
• Metrology & units: unit(x), dim(x), check_dim(expr), u(x), U = k * u_c.
• Quality & missingness: m ∈ {0,1}, q_score ∈ [0,1], drift.
• Environment & correction: RefCond, corr_env(x; RefCond).
• Reserved conflicts: never mix T_fil with T_trans; strictly distinguish n and n_eff.

III. Axiom Set (P102-*)

• P102-01 Units-and-dimensions consistency
  Any expression destined for the release plane must satisfy check_dim(expr) = true; mixing dimensionless with dimensional quantities is forbidden.
• P102-02 Two-form concurrency
  Any scenario involving arrival time computes both forms and persists their difference; downstream contracts judge against a threshold on that difference.
• P102-03 Explicit path and measure
  Every integral declares domain and measure explicitly, including gamma(ell) and d ell for path integrals; omission is not allowed.
• P102-04 Monotone time and path
  non_decreasing(tau_mono) = true, non_decreasing(ell) = true; violations require correction or quarantine.
• P102-05 Unified time base
  Internal evaluation on tau_mono; external publication maps to ts with estimates of offset, skew, J and their uncertainties.
• P102-06 Explicit environmental reference
  Any environmental correction is expressed as corr_env(x; RefCond), with RefCond persisted for audit.
• P102-07 Explicit missingness
  Missingness is flagged by m ∈ {0,1}; imputation or deletion must be recorded in the manifest with method and scope.
• P102-08 Quality grading and down-weighting
  Data quality is represented by q_score; cleaning may down-weight or quarantine low-quality segments, with strategy declared in contracts.
• P102-09 Contracts before release
  Artifacts failing contractual thresholds must not proceed to freeze and signature.
• P102-10 Traceability and signature
  Release artifacts and manifests are traceable and signed via hash_sha256(blob), ensuring non-repudiation and reproducibility.
• P102-11 Auditable, configurable thresholds
  All thresholds are parameterized in the manifest with sources and change logs to guarantee audit consistency.
• P102-12 Reserved-name enforcement
  Conflicting names such as T_fil vs T_trans and n vs n_eff must not be mixed; any ambiguity constitutes a contract failure.

IV. Minimal Equation Set (S102-*)

• S102-01 Time mapping
  ts = map_to_pub( tau_mono ; offset, skew, J )
• S102-02 Arrival-time two forms
  T_arr_form1 = ( 1 / c_ref ) * ( ∫ n_eff d ell )
  T_arr_form2 = ( ∫ ( n_eff / c_ref ) d ell )
• S102-03 Two-form difference metric
  delta_form = | T_arr_form1 - T_arr_form2 |
• S102-04 Path length and monotonicity constraint
  L_gamma = ( ∫_gamma 1 d ell ), and check non_decreasing(ell) = true
• S102-05 Probability and physical-density conventions
  ( ∫_Omega p(x) dx ) = 1
  mass_conserve ≈ sum_i( rho_i * DeltaV_i ) = const ± tol_mass
• S102-06 Missingness and coverage
  coverage = 1 - mean(m), where m = 1 denotes missing.
• S102-07 Quality-score base formula
  q_score = w_cov * coverage + w_cons * consis + w_unit * unit_ok + w_arr * ( 1 - norm_delta ) + w_drift * ( 1 - drift_norm )
  where unit_ok ∈ {0,1}, norm_delta = min( 1 , delta_form / tol_Tarr ), drift_norm ∈ [0,1], and the weights satisfy sum w_* = 1.
• S102-08 Uncertainty propagation (no-correlation approximation)
  u^2(y) = sum_j ( ( ∂f / ∂x_j )^2 * u^2(x_j) )
  If correlations are needed, add 2 * sum_{i<j} ( ( ∂f / ∂x_i )( ∂f / ∂x_j ) * cov(x_i, x_j) ).
• S102-09 Drift metrics (candidates)
  drift = D_KL( p_t || p_ref ), or drift = KS( F_t , F_ref ); the method is declared in the manifest with a unified tol_drift.
• S102-10 Deduplication retention rule
  keep(k) = argmax_{cand ∈ C(k)} { α1 * q_score - α2 * | ts - ts_ref | - α3 * dup_penalty }
  Parameters α1, α2, α3 are declared in the manifest.
• S102-11 Overall contract pass
  contract_ok = all( tests_i = true )
  where tests_i ∈ { unique(pk), non_decreasing(ts|ell), check_dim_set, delta_form ≤ tol_Tarr, eps_norm ≤ tol_norm, res_mass ≤ tol_mass }.
• S102-12 Signature and release freeze
  signature = sign( hash_sha256( blob ), issuer )
  manifest_signed = verify( signature , issuer ) = true

V. Control Equation & Default Threshold Conventions

1. Control equation
   pass = check_dim ∧ arrival_forms ∧ contract_ok ∧ manifest_signed
2. Threshold conventions
   • Arrival-time coherence: delta_form ≤ tol_Tarr.
   • Probability normalization: eps_norm ≤ tol_norm.
   • Quality floor: q_score ≥ q_min.
   • Drift ceiling: drift ≤ tol_drift.
   • Conservation residual: res_mass ≤ tol_mass.
     Concrete values are parameterized in the manifest and tracked in the change log.

VI. Usage Notes & Citation Rules

• Order of application
  Constrain semantics and feasible domain with P102-*, compute and assert with S102-*, then execute and persist via M10-*.
• Cross-chapter reuse
  Two forms and time mapping are reused in Chapters 5 and 6; probability and density in Chapter 13; uncertainty propagation in Appendix E; deduplication and contracts in Chapters 9 and 10.
• Conflict handling & priority
  Axioms outrank processes, contracts outrank release, and signature outranks distribution.

VII. Cross-References

• Time semantics and arrival time: see Chapters 5 and 6.
• Contracts and release freeze: see Chapter 10.
• Quality scoring and audit: see Chapter 14.
• Probability density and normalization: see Chapter 13.
• Interfaces and implementation bindings: see Appendix A.
• Thresholds and policy library: see Appendix B.
• Uncertainty propagation: see Appendix E.

Summary
This chapter fixes semantic boundaries and hard constraints via P102-*, and provides the executable core via S102-*. From here on, all cleaning stages share a unified arrival-time convention, time-base semantics, explicit measures, and dimension-conservation standards, with contracts and signatures serving as the final gates for compliant release.