15-EFT.WP.Methods.Falsification v1.0 | Chapter 4: Data & Adversarial Sample Conventions

Home ／ Docs-Technical WhitePaper ／ 15-EFT.WP.Methods.Falsification v1.0

Chapter 4: Data & Adversarial Sample Conventions

I. Scope & Objectives

1. Define unified conventions for the data assets and adversarial sample families required for falsification — covering collection, normalization, generation, filtering, fingerprinting, lineage, and signed publication — to ensure reproducibility, forensic traceability, and quantitative comparability under EnvLock and a shared time base ts = alpha + beta * tau_mono.
2. Goals & pass criteria
   • Establish the sample family D_family = { D_base, D_neg, D_boundary, D_ood, D_adv, D_mr } with generation and acceptance rules. Any adversarial perturbation must satisfy the budgeted feasible-set constraint
     x_adv = proj_X( x + delta ), with || delta ||_p ≤ epsilon_lp.
   • The bank is evaluated by coverage and kill:
     cov_spec = ( |C_hit| / |C_total| ), kill_rate = ( |mut_killed| / |mut_all| ); thresholds must be met, and statistical tests must pass FDR/FWER control (see Chapter 3 S52-*).

II. Terms & Symbols

• Data & domains: X (input domain), Y (label domain), D_base (baseline), D_neg (negative), D_boundary (boundary), D_ood (out-of-distribution), D_adv (adversarial), D_mr (metamorphic), Omega_support (support).
• Mutation & budgets: op_mut (mutation operator), MR_k (metamorphic relation), delta (perturbation), epsilon_linf, epsilon_l2, attack_steps, step_size, targeted ∈ {true,false}, budget.cpu/gpu/mem, budget.power.
• Time & spectra: ts, tau_mono, U_w (window energy), ENBW, S_xx(f).
• Fingerprints & signatures: hash(•), fingerprint, anchor, Graph.sig, ParamCard.sig, InferPipelineCard.sig.

III. Postulates & Minimal Equations

1. P51-4 (Feasible-set & dimensional-conservation postulate)
   Any generated sample x_gen must lie in the feasible set X and pass dimensional checks check_dim(expr). If x + delta exits X, project back via proj_X(•) and record the overflow magnitude and correction path.
2. P51-5 (Metamorphic invariance postulate)
   For any MR_k, there exists an invariant Inv_k such that
   Inv_k( x ) = Inv_k( MR_k(x) ).
   A violation constitutes an assertion failure and routes to falsification (see Chapter 5).
3. S52-5 (Adversarial budget & success criteria)
   Budget: || delta ||_p ≤ epsilon_lp, x_adv = proj_X( x + delta ).
   Success (choose one):
   • Label flip: argmax f( x_adv ) ≠ argmax f( x )
   • Loss delta: ΔL = L( y, f( x_adv ) ) - L( y, f( x ) ) ≥ tau_attack
4. S52-6 (OOD distance & gate)
   With embedding z(•) and metric dist(•,•),
   d_ood(x) = min_{x' ∈ D_train} dist( z(x), z(x') ).
   Flag OOD when d_ood(x) ≥ eta_ood, where eta_ood is tuned by target FPR/TPR on a dev ROC.
5. S52-7 (Family sampling quotas)
   The sampling distribution pi_family satisfies Σ_j pi_family(j) = 1. During execution, the expected per-family quota is
   E[ n_j ] = pi_family(j) * N_total,
   and the mixture-quality gate is
   || \hat{pi} - pi_family ||_1 ≤ tau_mix.

IV. Data & Manifest Conventions

• DataCard (minimum fields)
  schema_id, sources, license, pii_policy, features[{name, dtype, unit}], window{history, context, lookahead=0}, sampling{rate, jitter}, preprocess{norm, clip, resample}, spectral{U_w, ENBW}, timebase{alpha, beta}, EnvLock, anchor, hash(dataset).
• AdvOpsCard (adversarial & mutation config)
  ops[ {id, method, norm ∈ {linf,l2}, epsilon, steps, step_size, targeted, random_start, seed} ], constraints{proj_X, masks}, success_metric ∈ {label_flip, loss_delta}, tau_attack, pi_family, eta_ood.
• Fingerprint & lineage
  fingerprint = hash( schema_id || preprocess || AdvOpsCard || pi_family || seed || anchor ).
  Any field change must yield a new fingerprint and increment version.
• Labels & quality
  Label consistency
  acc_annot = ( 1/M ) * Σ [ y_i = y_i^{ref} ].
  If acc_annot < tau_annot, mark the slice weakly labeled and down-weight or exclude in tests.

V. Algorithms & Implementation Bindings

1. Prototypes (extending I50-*)
   • I50-3 generate_counterexamples(runtime:any, hypothesis:Hypothesis, ops:list, budget:dict) -> CEReport
   • I50-4 metamorphic_transform(x:any, MR:dict) -> x_prime:any
   • I50-5 adversarial_attack(runtime:any, x:any, method:str, eps:dict) -> AttackReport
   • I50-15 build_sample_bank(cards:dict, data:any) -> Bank (construct D_family and return indices)
   • I50-16 filter_ood(data:any, z:any, eta_ood:float) -> {in:list, ood:list}
   • I50-17 balance_sampler(bank:Bank, pi_family:dict, N:int, seed:int) -> Batch
   • I50-18 sign_bank(bank:Bank, anchor:str) -> {fingerprint:str, sig:str}
   • I50-19 verify_budget(batch:Batch, constraints:dict) -> BudgetReport
2. Idempotency & exceptions
   Idempotent if EnvLock, cards, seed, and anchor are fixed — build_sample_bank must keep its fingerprint.
   Possible exceptions: E_SCHEMA_MISMATCH, E_DIMENSION_MISMATCH, E_PRIVACY_VIOLATION, E_RESOURCE_EXCEEDED, E_NONDETERMINISM.

VI. Metrology Flows & Run Diagram

• Mx-55 Lake ingestion & normalization
  Validate DataCard → run preprocess → record hash(dataset) and timebase{alpha,beta} → ensure check_dim(expr) passes in full.
• Mx-56 Sample generation & constraint validation
  Create D_neg/D_boundary (boundary & constraint-fuzzing), D_mr (apply MR_k), D_adv (adversarial_attack) → verify_budget to enforce || delta ||_p ≤ epsilon_lp and proj_X feasibility.
• Mx-57 OOD filtering & privacy review
  Compute d_ood(x) and split by eta_ood → trigger pi_family rebalancing → run PII/compliance scans and redaction.
• Mx-58 Bank signing & publication
  sign_bank produces {fingerprint, sig} → generate Coverage.report, KillRate.report, OOD.eval → publish into the evidence bundle and register the anchor.
• Runtime observability & rollback
  Observe: TS.latency (generation latency), TS.error (generation failure rate), budget.* (resource spend).
  Roll back: if || \hat{pi} - pi_family ||_1 > tau_mix or epsilon violations occur, revert to the previous signed version and degrade generation strategy (e.g., fewer steps, different method).

VII. Verification & Test Matrix

1. Compliance basics
   • Budget check: random x, verify max_p || delta ||_p ≤ epsilon_lp and x_adv ∈ X.
   • Metamorphic invariance: for each MR_k, sample-check Inv_k(x) = Inv_k(MR_k(x)); failure rate ≤ pre-registered threshold.
   • OOD threshold: on a labeled dev set, achieve TPR@FPR = tau_fpr.
2. Coverage & kill
   • Spec coverage: cov_spec ≥ tau_cov.
   • Kill-rate: kill_rate ≥ tau_kill, with power interval or bootstrap band 1 - delta.
3. Stability & replay
   Replay R = 3: fingerprint unchanged; attack_success_rate coefficient of variation < tau_cv; offline/online generation, after ts alignment, satisfies delta_offon ≤ tau_offon.

VIII. Cross-References & Dependencies

IX. Risks, Limitations & Open Questions

• Risks & limitations
  Gradient masking producing false negatives; sensitivity of z(•) to OOD decisions; boundary cases escaping the physically observable domain; privacy re-identification risk; high generation cost stressing TS.latency and budget.power.
• Open questions
  Adaptive quotas for adversarial families under streaming FDR; cross-domain migration & sharing of eta_ood; verifiability and bias bounds for generator-driven MR_k.

X. Deliverables & Versioning

1. Deliverables
   DataCard.yaml, AdvOpsCard.yaml, Bank.index.json (family & quota stats), Coverage.report, KillRate.report, OOD.eval, BudgetReport.json, Bank.sig.
2. Versioning & changes
   • Any change to schema_id / preprocess / ops / epsilon / pi_family / eta_ood increments at least the minor in major.minor.patch; update fingerprint and anchor.
   • Link to TestPlan.version for backreference during acceptance and continuous falsification (see Chapter 12).