03-EFT.WP.Core.Parameters v1.0 | Chapter 4 — Priors, Likelihoods, and Posteriors

Home ／ Docs-Technical WhitePaper ／ 03-EFT.WP.Core.Parameters v1.0

Chapter 4 — Priors, Likelihoods, and Posteriors

I. Aims and Scope

• Establish unified notation and implementation conventions for prior(theta), L(data | theta), post(theta | data), and evidence Z.
• Provide common likelihood templates, prior families, and hyper-parameter specifications, and explain coupling with transforms T_map and the associated Jacobian corrections.
• Output anchors: minimal equations S41-1…S41-3; interface use cases I30 3, I30 5; workflow Mx-2 (calibration pipeline).

II. Symbols and Objects

• Parameters and data: theta ∈ Theta, data = { y_k, x_k }_{k=1..N}; grouped data data = ⋃_g data_g.
• Prior/likelihood/posterior: prior(theta), L(data | theta), post(theta | data), Z.
• Transform and Jacobian: phi = T_map(theta), J_T = ∂theta/∂phi.
• Noise and scales: sigma(x) (heteroscedastic scale), tau (precision), Corr (correlation matrix), Cov (covariance).
• Windows and domains (aligned with the statistics chapter): avg_t[•; Δt], avg_V[•; V].
• Path/arrival-time references (cross-volume): gamma(ell), d ell, T_arr = ( ∫ ( n_eff / c_ref ) d ell ).

III. Bayes Mother Form and Evidence (Minimal Equation S41-1)

1. S41-1 (Bayes mother form)
   • post(theta | data) = L(data | theta) * prior(theta) / Z
   • Z = ∫_Theta L(data | theta) * prior(theta) d theta
   • log post(theta | data) = log L(data | theta) + log prior(theta) - log Z
2. Extremum definitions:
   • theta_MLE = argmax_theta L(data | theta)
   • theta_MAP = argmax_theta post(theta | data)
3. Dimensional closure requirement:
   prior(theta) and L(data | theta) are densities or probability masses; Z is a normalizing constant; verify with check_dim(expr:str).

IV. Likelihood Construction Templates

1. Conditional-independence factorization (state explicitly):
   • L(data | theta) = ∏_{k=1}^N L_k(y_k | x_k, theta)
   • If grouped: L(data | theta) = ∏_g ∏_{k ∈ g} L_{g,k}(y_k | x_k, theta)
2. Additive Gaussian noise:
   • Observation model y_k = f(x_k; theta) + ε_k, ε_k ~ Normal(0, sigma_k^2)
   • L_k = Normal(y_k | f(x_k; theta), sigma_k)
3. Heteroscedastic Gaussian:
   sigma_k = sigma(x_k); same likelihood as above with sigma_k varying over x_k.
4. Multiplicative LogNormal noise:
   • y_k = f(x_k; theta) * η_k, log η_k ~ Normal(0, s^2)
   • L_k = LogNormal(y_k | log f(x_k; theta), s)
5. Counting models:
   L_k = Poisson(y_k | λ_k(theta)) or Binomial(y_k | n_k, p_k(theta))
6. Path/arrival-time measurements (cross-volume alignment):
   • Predictor f(x_k; theta) def= T_arr(theta) = ( ∫ ( n_eff / c_ref ) d ell ), with gamma(ell) and d ell declared explicitly
   • Observation model y_k = T_arr(theta) + ε_k, commonly ε_k ~ Normal(0, sigma^2)
7. Mixtures/compound models:
   Component mixture L_k = Σ_j w_j * L_{k,j}, w_j ≥ 0, Σ_j w_j = 1; priors on w via Dirichlet.

V. Prior Families and Hyper-Parameters (Canonical Conventions)

1. Real, unbounded:
   • Normal(mu, sigma); hyper-parameters {mu, sigma>0}
   • Laplace(mu, b) (L1-regularization equivalent); b>0
2. Positive domain (0, +inf):
   LogNormal(mu, sigma); Gamma(shape, rate); HalfNormal(sigma)
3. Bounded interval (lb, ub):
   Beta(a, b) on s ∈ (0,1); interval map theta = lb + (ub - lb) * s
4. Covariance/correlation structures:
   InvWishart(ν, S) or factored priors Sigma = D * R * D, with R ~ LKJ(η) and diagonal D elements HalfNormal
5. Proportions and mixture weights:
   Dirichlet(alpha_vec); alpha_vec > 0
6. Prior-writing postulate:
   Every prior family must list all hyper-parameters and support. Example: prior(c_ref) = LogNormal(mu_c, sigma_c) with units stated.

VI. Posterior in the Transform Domain (Minimal Equation S41-2)

1. Let phi = T_map(theta) be invertible, with J_T = ∂theta/∂phi. Then
   log post(phi | data) = log L(data | theta(phi)) + log prior_theta(theta(phi)) + log | det(J_T(phi)) | - log ZS41-2 (log posterior in transform space)
2. Common special cases:
   • Log transform (positive domain): theta = lb + exp(phi), log | det(J_T) | = Σ_i phi_i
   • Logit transform (interval): theta = lb + (ub - lb) * σ(phi),
     log | det(J_T) | = Σ_i [ log(ub_i - lb_i) + log σ(phi_i) + log(1 - σ(phi_i)) ]

VII. Regularization Equivalences and Selection Guide

• Ridge (L2): prior(theta_i) = Normal(0, σ^2) is equivalent to adding (1/(2σ^2)) * theta_i^2 to the objective.
• Lasso (L1): prior(theta_i) = Laplace(0, b) is equivalent to adding (1/b) * |theta_i|.
• Sparse scales: Horseshoe(λ, τ) suits strongly sparse priors; implement via local–global hierarchical decomposition.

VIII. Hierarchies and Shared Parameters

1. Group-level priors (partial pooling):
   • theta_g | mu, tau ~ Normal(mu, tau^{-1})
   • mu ~ Normal(mu0, s0), tau ~ Gamma(a0, b0)
2. Sharing/coupled coefficients:
   If theta_a = r * theta_b, put a LogNormal or Normal prior on r and infer jointly.

IX. Model Comparison and Information Criteria (Minimal Equation S41-3)

1. S41-3 (information-criteria short list)
   • AIC = 2k - 2 log L(data | theta_MLE)
   • BIC = k * log N - 2 log L(data | theta_MLE)
   • DIC approx= 2 * avg_theta[ -log L(data | theta) ] - ( -2 log L(data | theta_hat) )
2. Evidence and Bayes factors:
   Z = ∫ L * prior d theta; for two models M1, M2, the ratio BF_{12} = Z_1 / Z_2 (estimate numerically via bridge sampling/thermodynamic integration).

X. Implementation Binding and Minimal Working Examples (I30 3 / I30 5)

1. Prior registration (I30 3):
   • set_prior(code="c_ref", family="LogNormal", hyper={"mu":mu_c, "sigma":sigma_c})
   • set_prior(code="n_eff.alpha", family="Gamma", hyper={"shape":a, "rate":b})
2. Inference and sampling (I30 5):
   • theta_mle = infer_mle(model=S20_arrival, data=D, params=["c_ref","n_eff.alpha"])
   • theta_map = infer_map(model=S20_arrival, data=D, params=[...])
   • samples = posterior_sample_mcmc(model=S20_arrival, data=D, params=[...], n=2000, burn=500, method="NUTS")
3. Evidence approximations and comparison:
   After sampling, call the information-criteria or bridge-sampling module; report AIC/BIC and BF.

XI. Calibration Pipeline (Mx-2)

• Choose a likelihood template and declare independence or correlation structure; if correlated, provide a prior on Cov or Corr.
• For each parameter, set prior(theta_i) and bounds/transform T_map, and pass validate_param_set.
• Perform prior predictive checks: sample from the prior to generate pseudo-data and compare with observation scales.
• Run infer_mle for initialization, then infer_map or posterior_sample_mcmc.
• Conduct posterior predictive checks and residual diagnostics; adjust priors or likelihoods if necessary.
• Compute information criteria and (optionally) evidence ratios to select or weight candidate models.

XII. Arrival-Time Sidebar (Cross-Volume Consistency)

1. If parameters affect T_arr, all formulas must retain the fully parenthesized form and explicit path:
   T_arr = ( 1 / c_ref ) * ( ∫ n_eff d ell ) or T_arr = ( ∫ ( n_eff / c_ref ) d ell )
2. Implementation bindings:
   • propagate_time(n_eff_path, ds, c_ref) -> float
   • Propagate posterior samples to the uncertainty of T_arr; avg_gamma may serve as the statistical window for path averages.

XIII. Misuse and Conflict List

• Do not write ∫ n d ell / c; always use ( ∫ ( n_eff / c_ref ) d ell ) or ( 1 / c_ref ) * ( ∫ n_eff d ell ).
• Prior families must not omit hyper-parameters; write prior(theta_i) = Normal(mu_i, sigma_i) rather than a bare Normal.
• In transform-space inference, include log | det(∂theta/∂phi) |; avoid mixing densities of phi and theta.
• Never mix T_fil with T_trans; never interchange n and n_eff.
• For information criteria, k is the effective number of parameters (after transforms and constraints); sample size N is the total number of points—or the effective count after window aggregation—and must be stated in text.

XIV. Output Anchors and Citations

• Minimal equations: S41-1 (Bayes mother form), S41-2 (log posterior in transform space), S41-3 (information criteria).
• Interfaces: I30 3 (prior registration), I30 5 (inference and sampling).
• Workflow: Mx-2 (calibration pipeline).
• Cross-volume citations: consistent with EFT.WP.Core.Equations S20-* (arrival time) and S40-* (tension field).