09-EFT.WP.Core.Density v1.0 | Chapter 10 — Uncertainty and Information Bounds

Home ／ Docs-Technical WhitePaper ／ 09-EFT.WP.Core.Density v1.0

Chapter 10 — Uncertainty and Information Bounds

I. Purpose and Scope

• Define and harmonize the metrological vocabulary for u(x) (standard uncertainty), u_c (combined standard uncertainty), and U = k * u_c (expanded uncertainty). Establish rules to propagate uncertainty from density models to parameter estimates and functionals via Delta and Monte Carlo methods, and provide executable statements for Fisher information and CRLB.
• Interoperate with this volume’s anchors S92-14/15 (transformations & measure preservation), S92-8/9 (spectral metrics), S92-10/11 (discrete conservation), and S92-41 (regularized objectives); align implementation with I90 7 : fisher_information / crlb.

II. Notation and Objects

• Parameter vector theta∈R^p; MLE theta_hat; log-likelihood ell(theta); score s(theta) = ∂_theta ell(theta).
• Covariances and uncertainties: cov(z), u(z) (standard), u_c (combined), U = k * u_c (expanded).
• Mappings and Jacobians: phi = h(theta), J_{theta→phi} = ∂theta/∂phi; for a functional y = g(theta), gradient G = ∂g/∂theta, Hessian H_g.

III. Metrological Uncertainty System

• Combined standard uncertainty (linearized, with correlations):
  S92-72 : u_c^2( y ) = G * cov(theta_hat) * G^T, where y = g(theta_hat) and G = ∂g/∂theta |_{theta_hat}.
• Expanded uncertainty (coverage factor k):
  S92-73 : U = k * u_c (commonly k≈2 for ~95% coverage).
• Law of total variance:
  S92-74 : var(X) = E[ var( X | Z ) ] + var( E[ X | Z ] ).

IV. Fisher Information: Definition, Properties, Reparameterization

• Definition (expected score outer product):
  S92-16 : I_F(theta) = E[ s(theta)^T s(theta) ] = E[ ( ∂_theta log p(X | theta) )^T ( ∂_theta log p(X | theta) ) ].
• Equivalent form (negative expected Hessian):
  S92-70 : I_F(theta) = - E[ ∂^2_{theta,theta} ell(theta) ] (under regularity).
• Additivity for i.i.d. samples:
  S92-71 : I_F^{(N)}(theta) = N * I_F^{(1)}(theta).
• Covariance under reparameterization:
  S92-85 : I_F(phi) = J_{theta→phi}^T * I_F(theta) * J_{theta→phi} for invertible phi = h(theta).

V. Cramér–Rao Lower Bounds (CRLB)

• Matrix bound for unbiased estimators:
  S92-17 : cov( theta_hat ) ≥ I_F^{-1}(theta) (Löwner partial order).
• Biased case (MSE bound):
  S92-86 : MSE( theta_hat ) ≥ ( I + ∂b/∂theta ) * I_F^{-1}(theta) * ( I + ∂b/∂theta )^T + b b^T, with bias b = E[theta_hat] - theta.
• Efficiency and attainability: in regular families, theta_hat_MLE is asymptotically normal and efficient: sqrt(N) ( theta_hat - theta ) → N( 0 , I_F^{-1} ).

VI. Uncertainty Propagation: Delta vs. Monte Carlo

• First-order Delta method: variance via S92-72; second-order bias correction:
  S92-80 : bias( g(theta_hat) ) ≈ ( 1 / 2 ) * tr( H_g( theta ) * cov( theta_hat ) ).
• Multiple outputs y = g(theta): stack rows of G per output and apply G cov G^T.
• Monte Carlo: sample {theta^(m)} from N( theta_hat , cov( theta_hat ) ) (or posterior), compute {y^(m)}, and form intervals by sample variance/quantiles; record convergence criteria and random seed.

VII. Model Specializations: Density / Intensity / Spectrum

• Inhomogeneous Poisson intensity (spatial A, measure dV):
  S92-78 : I_F( theta ) = ( ∫_A ( 1 / lambda(x;theta) ) * ( ∂_theta lambda ) ( ∂_theta lambda )^T dV ).
• Counting processes (time [0,T]):
  S92-79 : I_F( theta ) ≈ ( ∫_0^T E[ ( ∂_theta lambda(t;theta) / lambda(t;theta) ) ( ∂_theta lambda(t;theta) / lambda(t;theta) )^T ] dt ) (mean-field for Hawkes, etc.).
• KDE pointwise asymptotic variance (kernel K, R(K) = ( ∫ K(u)^2 du )):
  S92-81 : var( kde_h(x) ) ≈ ( 1 / ( N * h ) ) * R(K) * p(x) (consistent with Chapter 4 bandwidth trade-offs).
• Welch PSD: effective degrees of freedom and relative variance (K segments, window w[n]):
  S92-75 : nu ≈ 2 * K * c_4 , c_4 = ( ( ∑ w[n]^2 )^2 ) / ( ∑ w[n]^4 );
  S92-76 : var( S_hat(f) ) / S_xx(f)^2 ≈ 1 / nu;
  S92-77 : var( log S_hat(f) ) ≈ 2 / nu (natural log). Report along with ENBW_Hz, U_w per S92-8/9.

VIII. Regions and Intervals: Likelihood and Bayesian Approximations

• Likelihood-ratio confidence region (Wilks):
  S92-82 : C_{1-α} = { theta : 2 ( ell( theta_hat ) - ell( theta ) ) ≤ chi2_{p, 1-α} }.
• Laplace posterior covariance:
  S92-83 : p( theta | data ) ≈ N( theta_hat , I_F^{-1}( theta_hat ) ) for weak priors; with informative priors, use I_post ≈ I_F + I_prior.
• Jeffreys prior:
  S92-84 : pi_J( theta ) ∝ sqrt( det( I_F( theta ) ) ) (parameterization invariant).

IX. Workflow Mx-99: Density → Uncertainty → Information Bounds

• Model & calibrate. From Chapters 3/4/5/6/9 obtain p(•) or lambda(•), and S_xx(f); align units and TOA; retain delta_form.
• Estimate & covariance. Using I90 2/3/6, compute estimates and residuals; obtain I_F from a numerical Hessian or I90 7 : fisher_information, and set cov( theta_hat ) ≈ I_F^{-1}.
• Propagate to targets. For y = g(theta), apply S92-72 (and S92-80 if bias matters); for spectra, build frequency-wise intervals via S92-75/76/77.
• Bounds & intervals. Report S92-17/86 bounds and either the likelihood-ratio region S92-82 or a Laplace interval S92-83.
• QC & publication. Verify S92-74 variance closure; check conservation S92-11/63 and energy consistency; compute U = k u_c and write to metadata.

X. Reporting Standard (Minimum Required Set)

• Model & data: sample size, weighting, independence assumptions; window/segmentation for spectral tasks.
• Fisher information: I_F(theta_hat), computation mode (score vs. -Hessian), whether I90 7 was used.
• Covariance & intervals: cov(theta_hat), key functional u_c/U and coverage factor k; likelihood-ratio or Laplace regions.
• Propagation basis: explicit g(theta) and gradient G; if Monte Carlo, report draws and convergence.
• Spectral-only: nu, var(S_hat)/S^2, ENBW_Hz, U_w.
• Cross-checks: conservation & energy consistency, delta_form.

XI. Cross-Chapter and Interface Alignment

• Chapter 2: propagate source/sink and boundary uncertainties to total mass M via S92-72/74.
• Chapter 6: preserve coherence for nu/ENBW_Hz/U_w.
• Chapter 8: approximate uncertainty of regularization parameters from observed Hessian I_F; select via Mx-97.
• Chapter 9: under transformations, map I_F via S92-85; record TOA discrepancy delta_form.
• Interfaces: I90 7 : fisher_information / crlb as primary bindings; others via I90 2/3/6/8.

XII. Key Formula Index