15-EFT.WP.Methods.Falsification v1.0 | Chapter 6: Falsification Operators & the Automation Framework

Home ／ Docs-Technical WhitePaper ／ 15-EFT.WP.Methods.Falsification v1.0

Chapter 6: Falsification Operators & the Automation Framework

I. Scope & Objectives

1. Define the families of falsification operators at three layers — input / model / pipeline — together with their composition and an automation & orchestration framework. Coverage includes adversarial generation, mutation & fuzzing, time-base and resource perturbations, and online injection with rollback. All operators execute under a locked environment EnvLock and a shared time base ts = alpha + beta * tau_mono, and must produce reproducible, auditable evidence and metrics (cov_spec, kill_rate, TS.latency / TS.error, FDR/FWER).
2. Pass criteria
   • The operator library covers the target domains and threat models; automated orchestration can generate samples that satisfy resource budgets budget.cpu/gpu/mem, budget.power and distance constraints (epsilon_linf or epsilon_l2).
   • Each layer’s injection points are idempotent and rollback-safe; generation logs carry hash(•) and fingerprint signatures.

II. Terms & Symbols

1. Operators & budgets
   • Input-level operators: mutate(x; op, budget) such as noise, blur, occlude, synonym, prompt_shift.
   • Model-level operators: gradient_attack, weight_jitter, quantize_jitter(qbits), drop_layer(k).
   • Pipeline-level operators: delay(dt), jitter(sigma), drop(rate), reorder(k), clock_skew(alpha,beta).
   • Distances & projections: B_p(x, epsilon) (the l_p ball), projection Pi_{B_p}; step size eta; iterations attack_steps.
2. Injection points & graph
   Inject@Input, Inject@Preproc, Inject@Model, Inject@Postproc, Inject@IO; runtime graph nodes are tagged with Graph.sig.
3. Randomness & determinism
   rng.seed, rng_family; deterministic execution deterministic = true/false; nondeterminism masking is recorded as NonDetMask.

III. Postulates & Minimal Equations

• P51-8 (Operator composability postulate)
  If op_a and op_b are compatible in dimension and domain, then for their composition op_b ∘ op_a the distance obeys the triangle inequality:
  dist( (op_b ∘ op_a)(x), x ) ≤ dist( op_a(x), x ) + dist( op_b(x), x ).
• P51-9 (Injective commutativity boundary postulate)
  For a commuting pair (op_i, op_j) there exists a domain D_comm such that, if x ∈ D_comm, then
  op_i ∘ op_j (x) = op_j ∘ op_i (x);
  outside D_comm, execution must declare a fixed ordering and side-effects.
• S52-13 (FGSM with projection constraint)
  x_adv = clip_D( x + epsilon * sign( grad_x L( model, x, y ) ) ),
  where clip_D(•) truncates results to the input domain and constraint set.
• S52-14 (PGD iterative attack on the l_inf ball)
• x_0 = x ; for t = 0..attack_steps-1:
• x_{t+1} = Pi_{B_inf(x, epsilon)}( x_t + eta * sign( grad_x L( model, x_t, y ) ) )
• ;
• x_adv = clip_D( x_{attack_steps} )
• S52-15 (Black-box gradient via SPSA)
  g_hat = ( 1 / m ) * Σ_{i=1..m} ( L( x + c * delta_i ) - L( x - c * delta_i ) ) * ( 1 / ( 2c ) ) * delta_i^{-1},
  with delta_i ∈ {−1, +1}^d and coordinatewise reciprocal delta_i^{-1}.
• S52-16 (Weight quantization jitter)
  W_q = Q_qbits(W), W_j = W_q + U( -delta_q, +delta_q ); enforce check_dim(W_j) and keep Graph.sig unchanged.
• S52-17 (Time-base perturbation consistency)
  For pipeline-level clock_skew(alpha,beta) the mapping satisfies ts' = alpha' + beta' * tau_mono; for online–offline parity,
  | (alpha', beta') - (alpha, beta) | ≤ tau_skew
  defines the no-drift region.

IV. Data & Manifest Conventions

• Op.card (minimum fields)
  { id, layer ∈ {input, model, pipeline}, threat_model, domain, constraints, params, rng_family, determinism, cost_model, safety_guard }.
  Examples for threat_model: epsilon_linf, epsilon_l2, semantic_edit, timing_fault.
  constraints must state check_dim(expr), domain bounds, and the FDR/FWER linkage policy.
• Inject.graph
  { nodes, edges, inject_points[ { node_id, hooks, precedence, rollback } ], signature: Graph.sig }, with hook flavors pre / post / around; rollback supplies recovery actions.
• Budget.card
  { cpu, gpu, mem, power, walltime, retries, max_fail_rate }, coupled to TS.latency / TS.error for automatic degradation triggers.

V. Algorithms & Implementation Bindings

1. Interface extensions (building on I50-*)
   • I50-11 register_operator(op_card:dict) -> OpHandle
   • I50-12 compose_operators(chain:list[OpHandle]) -> OpChain
   • I50-13 plan_injection(graph:Graph, chain:OpChain, policy:dict) -> InjectionPlan
   • I50-14 execute_injection(rt:Runtime, plan:InjectionPlan, inputs:any) -> Logs
   • I50-15 estimate_gradient(model:any, x:any, method:str, opts:dict) -> grad:any
   • I50-16 perturb_timebase(stream:any, alpha:float, beta:float) -> stream'
   • I50-17 audit_signature(artifact:any) -> {hash:str, fingerprint:str}
   • I50-18 rollback(rt:Runtime, anchor:str) -> Runtime
   • I50-19 enforce_determinism(rt:Runtime, mask:any) -> Runtime
2. Orchestration strategy (pseudocode)
3. 1. op_set ← registry.filter(threat_model, layer)
4. 2. chain ← compose_operators( select(op_set, policy.compat) )
5. 3. plan ← plan_injection(graph, chain, budget ∪ precedence)
6. 4. rt' ← enforce_determinism(rt, NonDetMask) ; sig ← audit_signature(rt')
7. 5. logs ← execute_injection(rt', plan, inputs)
8. 6. metrics: cov_spec, kill_rate, TS.latency, TS.error
9. if TS.error > tau_err or cov_spec not improved → rollback(rt', anchor)

VI. Metrology Flows & Run Diagram

• Mx-55 Operator admission
  Validate Op.card and safety_guard; verify dimensions/domains; smoke-test gradient_attack on a small sample, logging effect_size and resource curves.
• Mx-56 Injection-plan synthesis
  Combine Inject.graph and Budget.card under constraints to produce an InjectionPlan; declare execution order, compatibility, and projection function Pi_{B_p}.
• Mx-57 Execution & online observability
  Run on shadow/canary streams; collect TS.latency / TS.error, violation streams, and the coverage increment Δcov_spec. Anomalies trigger GateDecision ∈ {hold, block}.
• Mx-58 Rollback & audit
  On failure, rollback to anchor; generate hash(•) and fingerprint; archive logs and policy snapshots. On success, update anchor and signatures.

VII. Verification & Test Matrix

1. Operator correctness
   • For each op, apply boundary triplets inside/outside the domain; verify check_dim(expr) and domain constraints.
   • For gradient_attack, compare FGSM / PGD / SPSA violation rates and variance under the same epsilon.
2. Resources & timing
   • Stress under budget.*; record TS.latency p50/p95 and TS.error; exceeding thresholds must trigger degradation.
   • For clock_skew injections, verify the tau_skew gate in S52-17.
3. Replay & consistency
   Fix rng.seed and replay 3×; ensure delta_offon ≤ tau_offon and the coefficients of variation for cov_spec and kill_rate are ≤ tau_cv.
4. Statistics & error control
   Route assertion tests through the gatekeeping program, enforcing FDR ≤ q_star or FWER ≤ alpha_family; for multi-operator chains, compose family-wise error rates.

VIII. Cross-References & Dependencies

• Depends on: Core.Threads (orchestration & resources), Core.DataSpec (fields & dimensions), Core.Errors (exception taxonomy & gating), Core.Metrology (metrics & confidence).
• Cross-links: Chapter 4 (sample conventions) for input domains & lineage; Chapter 5 (test cases & coverage) for C_total / C_hit; Chapter 7 (statistical testing) for alpha/beta/FDR; EFT.WP.Methods.Inference Chapter 5 (operator stability & numeric safety).

IX. Risks, Limitations & Open Questions

• Risks & limitations
  Oracle ambiguity → misclassification; composite operators introduce non-commutative side-effects; high variance in black-box gradients → weak perturbations; quantization jitter may alter the semantic equivalence class behind Graph.sig.
• Open questions
  Unified distance metrics & projections for multimodal inputs; adaptive orchestration (op selection and step size eta from online feedback); migration & recalibration of shared InjectionPlan across domains.

X. Deliverables & Versioning

1. Deliverables
   Op.catalog.json, Inject.graph.json, InjectionPlan.json, Budget.card, Run.logs, Coverage.report, KillRate.report, TS.observability.csv, GateDecision.log, anchor.sig.
2. Versioning policy
   • Adding/removing operators or changing the threat model → minor bump; changing orchestration and gating policy → major bump.
   • Every version must produce hash(•) and fingerprint, and update the EnvLock annotation.