19-EFT.WP.Methods.SynthData v1.0 | Chapter 5 — Generation Engines II: Deep Generators (VAE/GAN/Flow/Diffusion)

Home ／ Docs-Technical WhitePaper ／ 19-EFT.WP.Methods.SynthData v1.0

Chapter 5 — Generation Engines II: Deep Generators (VAE/GAN/Flow/Diffusion)

I. Scope & Targets

1. Goals
   • For high-dimensional and multimodal data (images/speech/text/time series/graphs), implement high-fidelity, controllable, and auditable synthetic generation using VAE/GAN/Flow/Diffusion.
   • Maintain statistical consistency and downstream utility relative to p_data, while satisfying DP(eps,delta), unified time-base/arrival conventions, and dimensional conservation.
2. Inputs
   Normalized reference data D_ref (see Chapter 3), canonical schema SRef, deep engine specification EngineSpec, privacy budget, and SLO targets.
3. Outputs
   Deep generation engine engine_deep (family, architecture, weights, noise schedule), synthetic samples D_syn, metric reports, and manifest.synth.deep.*.
4. Applicability
   For strong physical constraints or exact unit conservation, prefer Chapter 6 (physics/scene graphs). For low-dimensional structured data, prefer Chapter 4 (statistical & explicit models).

II. Terms & Variables

• General: x ∈ X, z ~ p(z), c (condition/prompt), theta (generator params), phi (inference/discriminator params), seed, rng.
• VAE: q_phi(z|x), p_theta(x|z), ELBO, beta (regularization weight).
• GAN: G(z,c; theta), D(x,c; w), critic f, lambda_gp (gradient penalty).
• Flow: z = f_theta(x), p0(z), J_f (Jacobian), bpd (bits per dim).
• Diffusion: x_t, t ∈ [0,1], beta_t, alpha_bar_t, eps_theta(x_t,t,c), NFE (function evals), w_cfg (classifier-free guidance weight).
• Time/path: tau_mono, ts, T_arr, gamma(ell), delta_form, offset/skew/J.
• Privacy: DP(eps,delta); accountant returns eps_total.

*III. Axioms P405- **

• P405-1 (Explicit Measures): Training/sampling objectives—densities, likelihoods, or divergences—must be explicit; heuristic-only metrics are forbidden as substitutes.
• P405-2 (Controllability): The distribution, interface, and coverage of condition variables c must be registered and persisted in manifests.
• P405-3 (Stability First): Define monitorable convergence criteria and rollback strategies (see §IX).
• P405-4 (Privacy Upfront): When using DP-SGD/PATE/DP-noise, compute and publish eps_total, delta.
• P405-5 (Unified Time Base): Train/sample on tau_mono, publish on ts, and record offset/skew/J.
• P405-6 (Dual Arrival Forms): When T_arr is relevant, compute both formulations and record delta_form.
• P405-7 (Dimensional Conservation): unit/dim normalization and inverse transforms must preserve conservation (leveraging Chapter 3 & Chapter 4 constraint layers).
• P405-8 (Reproducibility): Persist seed/rng/solver/NFE to enable reproducible experiments and deferred evaluations.
• P405-9 (Evaluation Separation): Strict isolation of train/val/test; synthetic evaluation uses the metrics family aligned with Chapter 12.
• P405-10 (Accountability): Any distillation/acceleration/quantization requires re-evaluating fidelity, privacy, and SLOs.

*IV. Minimal Equations S405- **

1. S405-1 (VAE–ELBO)
   • ELBO(theta,phi; x) = E_{q_phi(z|x)}[ log p_theta(x|z) ] - KL( q_phi(z|x) || p(z) )；
   • ELBO_beta = E_{q_phi}[ log p_theta(x|z) ] - beta * KL( q_phi(z|x) || p(z) )。
2. S405-2 (GAN/WGAN Losses)
   • min_G max_D E_x[ log D(x) ] + E_{z}[ log( 1 - D( G(z) ) ) ]。
   • WGAN : min_G max_{||f||_L ≤ 1} E_x f(x) - E_z f( G(z) )，
     GP : + lambda_gp * E_{x_hat}( ( || ∇_{x_hat} f(x_hat) ||_2 - 1 )^2 )。
3. S405-3 (Flow Transform)
   • z = f_theta(x)，log p_theta(x) = log p0(z) + log | det J_f(x) |，
   • bpd = ( - log p_theta(x) ) / ( n_dims * log 2 )。
4. S405-4 (Diffusion Forward & Training)
   • x_t = ( alpha_bar_t )^{1/2} * x_0 + sigma_t * eps, eps ~ N(0,I)；
   • L_simple = E_{t,x,eps}( || eps - eps_theta(x_t, t, c) ||_2^2 )。
5. S405-5 (Score-SDE Reverse)
   d x = f(x,t) dt + g(t) d w_t，反向 d x = [ f - g^2 ∇_x log p_t(x) ] dt + g d w_bar_t。
6. S405-6 (CFG)
   eps_cfg = eps_theta(x_t,t,c) + w_cfg * ( eps_theta(x_t,t,c) - eps_theta(x_t,t,∅) )。
7. S405-7 (Dual Arrival Forms)
   delta_form = | ( 1 / c_ref ) * ( ∫ n_eff d ell ) - ( ∫ ( n_eff / c_ref ) d ell ) |。
8. S405-8 (DP-SGD Clipping & Noise)
   g_i ← clip( g_i, C )，ḡ = ( 1 / m ) * ( ∑ g_i ) + N( 0, ( sigma * C )^2 I )，会计器输出 eps_total。

V. Metrology Flow M40-5 (Deep Generation Loop)

1. Ready
   Choose family/architecture (VAE/GAN/Flow/Diffusion), objectives, and metrics; fix DP(eps,delta) and SLOs.
2. Preprocessing & Schema Binding
   Standardize data, map unit/dim, align tau_mono (see Chapter 3 and Chapter 2 axioms); split train/val/test.
3. Training
   • VAE: schedule reconstruction vs. KL (beta/annealing/free-bits).
   • GAN: stabilization via WGAN-GP/SN; augmentations and discriminator regularization.
   • Flow: coupling/affine layers, invertibility, and stable log-determinants.
   • Diffusion: beta_t schedules, v/eps/x0 heads, discrete or SDE solvers.
4. Conditioning & Control
   Define the c space, w_cfg bounds, and caps for rejection sampling or posterior projection costs.
5. Sampling & Acceleration
   Record NFE/solver/latency; if needed, apply distillation (e.g., DPM-Solver/Consistency/Teacher–Student).
6. Rules & Time Base
   enforce_constraints and align_timepath (write offset/skew/J, T_arr, delta_form).
7. Evaluation
   • Fidelity: FID/KID/PR-curve/coverage; likelihood/invertibility: bpd/NLL; VAE: ELBO/recon-PSNR; Diffusion (video): FVD.
   • Statistical consistency: see Chapter 12; drift alignment: see CrossStats Chapter 7.
8. Privacy & Safety
   Compute eps_total and MI risk; apply watermarking and provenance.
9. Persist & Publish
   Emit manifest.synth.deep and sign/freeze (see Chapter 13).

VI. Contracts & Assertions C40-5xx

• C40-511 (Fidelity Thresholds): FID ≤ tol_FID, KID_mean ± se ≤ tol_KID, PR_recall ≥ recall_min.
• C40-512 (Likelihood/Invertibility – Flow): bpd ≤ tol_bpd, inv_error ≤ tol_inv.
• C40-513 (VAE Calibration): ELBO_val improves and posterior_collapse_rate ≤ tol_collapse, recon_PSNR ≥ psnr_min.
• C40-514 (Diffusion Latency): NFE ≤ NFE_max, latency_ms_p99 ≤ SLO_latency, w_cfg ≤ cfg_max.
• C40-515 (Distribution Coverage): coverage ≥ covg_min, mode_drop ≤ tol_mode_drop.
• C40-516 (Privacy): eps_total ≤ eps_budget, MI ≤ mi_max.
• C40-517 (Time/Arrival): non_decreasing(tau_mono), J ≤ J_max, delta_form ≤ tol_Tarr.
• C40-518 (Dimensions): all check_dim(expr)=true.
• C40-519 (Reproducibility): seed/rng/solver/NFE persisted and replayable.

VII. Implementation Bindings I40-5*

• fit_engine_vae(ds, arch, beta, privacy) -> engine_vae
• fit_engine_gan(ds, arch, loss, lambda_gp, aug, privacy) -> engine_gan
• fit_engine_flow(ds, arch, base_dist, schedule) -> engine_flow
• fit_engine_diffusion(ds, arch, noise_schedule, loss_head, privacy) -> engine_diff
• sample_vae(engine_vae, n, condition, seed) -> ds_syn
• sample_gan(engine_gan, n, condition, seed) -> ds_syn
• sample_flow(engine_flow, n, condition, seed) -> ds_syn
• sample_diffusion(engine_diff, n, solver, NFE, w_cfg, condition, seed) -> ds_syn
• distill_diffusion(engine_diff, solver_src, solver_tgt, NFE_tgt) -> engine_diff_tiny
• measure_gen_metrics(real, syn, metrics) -> report
• privacy_accountant(logs) -> {eps_total, delta}
• emit_deep_manifest(artifacts) -> manifest.synth.deep
• Invariants: reproducible(seed); eps_total within budget; latency_p99 meets SLO; delta_form ≤ tol_Tarr; unit/dim conserved.

VIII. Cross-References

• This volume: Chapter 4 (statistical/explicit models) for low-dimensional or strongly constrained cases; Chapter 6 (physics/scene graphs) to reinforce physical consistency; Chapter 12 (evaluation) and Chapter 13 (release).
• Methods.Cleaning v1.0: Chapters 4/5/6 (units, time base, arrival time).
• Methods.CrossStats v1.0: Chapter 7 (drift & alignment), Chapter 14 (SLO & audit).

IX. Quality Metrics & Risk Control

1. Training stability
   Track grad_norm, real/fake loss gap, critic_drift, KL/ELBO curves, bpd trend, NFE and latency_p99.
2. Typical risks & actions
   • VAE posterior collapse → KL annealing/free-bits/hierarchical VAE.
   • GAN mode collapse/oscillation → WGAN-GP/SN, discriminator regularization, data augmentation, LR/momentum tuning.
   • Flow numerical instability → change coupling layers, constrain Lipschitz constants, increase precision.
   • Diffusion too slow → DDIM/DPM-Solver/Consistency, distillation, adaptive NFE; reduce w_cfg if drift occurs.
   • Excess privacy spend → adjust clipping C, increase noise, cut epochs, or bucketized training.

Summary

• This chapter defines a unified contract for VAE/GAN/Flow/Diffusion: P405-* guardrails, computable bases S405-*, the closed-loop flow M40-5, contracts C40-5xx, and bindings I40-5*.
• The deliverables—engine_deep, D_syn, metric reports, and manifest.synth.deep—provide a reproducible, auditable foundation for Chapter 6 (physics/scene graphs), Chapter 12 (evaluation), and Chapter 13 (release).