18-EFT.WP.Methods.CrossStats v1.0 | Chapter 11 | Time Series and Panel Data (ARIMA/State Space/ITS)

Home ／ Docs-Technical WhitePaper ／ 18-EFT.WP.Methods.CrossStats v1.0

Chapter 11 | Time Series and Panel Data (ARIMA/State Space/ITS)

One-sentence goal: Using tau_mono as the unified time base, deliver auditable modeling and forecasting for trend/seasonality, interventions, and dynamic panels within the ARIMA, state-space, and ITS frameworks—backed by contracts that enforce stationarity, white-noise residuals, and arrival-time consistency.

I. Scope and Objects

1. Scope
   • Univariate and multivariate series y_t, including seasonal and holiday regressors; structured state-space models with Kalman filtering; interrupted time series (ITS); and static/dynamic panels y_{i,t}.
   • Supports offline batch and online streaming rolling forecasts, window Δt, and multi-step horizon h.
2. Objects
   • Inputs: D = { (y_t, x_t, ts_t, m_t) } or panel D = { (y_{i,t}, x_{i,t}, ts_{i,t}, i) }, missingness mask m ∈ {0,1}, and alignment metadata offset/skew/J.
   • Outputs: model parameters, diagnostic dashboards, forecasts hat{y}_{t+h} with intervals, and manifest.stats.ts.*.

II. Terms and Variables

• Lags & differencing: L y_t = y_{t-1}, (1 - L)^d y_t; seasonal difference D_s(L) = (1 - L^s)^D.
• ARIMA polynomials: phi(L), theta(L); seasonal Phi(L^s), Theta(L^s).
• State space: state x_t, matrices A,B,C,D,Q,R; process noise w_t ~ N(0,Q), observation noise v_t ~ N(0,R).
• ITS indicators: I_t = 1(t ≥ T0), slope interaction I_t * (t - T0).
• Panel effects: unit fixed effects alpha_i, time effects lambda_t, dynamic coefficient rho.
• Residuals & diagnostics: e_t = y_t - hat{y}_t, ACF_k, LB_p (Ljung–Box), NRMSE, PI_coverage.

III. Postulates P311-*

• P311-1 (Unified time base): model on tau_mono, publish with ts, and attach offset/skew/J; if an arrival-time quantity T_arr is used, record both formulations with delta_form.
• P311-2 (Units & dimensions): declare unit(y), dim(y); all regression coefficients must pass check_dim( y - f(x) ).
• P311-3 (Stationarity & invertibility): AR roots lie outside the unit circle; MA is invertible; for state space, max_abs_eig(A) < 1 (stable case).
• P311-4 (Explicit missingness policy): for points with m = 0, use explicit imputation or filter predictions—no implicit fills.
• P311-5 (Panel independence & clustering): errors cluster by i or by (i,t); variance uses cluster-robust estimators.
• P311-6 (Intervention causal order): in ITS, interventions I_t and covariates x_t occur before the response y_t.

IV. Minimal Equations S311-*

1. S311-1 (ARIMA/SARIMA)
   phi(L) * Phi(L^s) * (1 - L)^d * (1 - L^s)^D y_t = theta(L) * Theta(L^s) * epsilon_t, epsilon_t ~ WN(0, sigma^2).
2. S311-2 (State space)
   • State: x_t = A x_{t-1} + B u_t + w_t; Observation: y_t = C x_t + D u_t + v_t.
   • Kalman prediction: x_{t|t-1} = A x_{t-1|t-1}; P_{t|t-1} = A P_{t-1|t-1} A' + Q.
   • Update: K_t = P_{t|t-1} C' ( C P_{t|t-1} C' + R )^{-1}; x_{t|t} = x_{t|t-1} + K_t ( y_t - C x_{t|t-1} ).
   • Log-likelihood: logL = -0.5 * ∑ ( log|S_t| + e_t' S_t^{-1} e_t + const ), S_t = C P_{t|t-1} C' + R.
3. S311-3 (ITS baseline form)
   y_t = beta0 + beta1 * t + beta2 * I_t + beta3 * I_t * ( t - T0 ) + gamma' z_t + epsilon_t, with epsilon_t optionally ARMA-structured.
4. S311-4 (Panel FE/RE and two-way fixed effects)
   y_{i,t} = alpha_i + lambda_t + beta' x_{i,t} + e_{i,t}; treat alpha_i as fixed or random; compute cluster-robust SEs.
5. S311-5 (Dynamic panel, Arellano–Bond)
   y_{i,t} = rho * y_{i,t-1} + beta' x_{i,t} + eta_i + e_{i,t}; difference and use lagged endogenous variables as instruments; estimate via GMM with Hansen_p ≥ p_min.
6. S311-6 (Forecasts & intervals)
   One-step: hat{y}_{t+1|t}; multi-step via recursion x_{t+h|t} = A^h x_{t|t}; construct intervals from innovation variance stacking or out-of-sample bootstrap.
7. S311-7 (White-noise residuals & coverage)
   Require LB_p pval ≥ p_min; PI_coverage ≈ target; ACF_k within confidence bands.
8. S311-9 (Temporal consistency & arrival time)
   When T_arr enters the model or metrics, evaluate on tau_mono, publish with ts, and log delta_form for the two T_arr formulations.

Unified arrival-time & path-measure convention

1. Constant-factored: T_arr = ( 1 / c_ref ) * ( ∫ n_eff d ell )
2. General: T_arr = ( ∫ ( n_eff / c_ref ) d ell )
   Always declare the path gamma(ell) and the measure d ell; track delta_form where applicable.

V. Statistical Workflow M30-11 (Ready → Model → Diagnose → Publish)

• Ready
  Run time_align_for_stats to align on tau_mono and enrich with holiday features; declare unit(y) and window Δt; complete missing/outlier handling.
• Candidate models & order selection
  Test differencing on y_t, use ACF/PACF and information criteria to choose ARIMA; or start with a structured state-space (local linear trend, seasonality, regressors).
• Fit & estimate
  fit_arima or fit_state_space (Kalman filter/smoother); in ITS, include level and slope break terms; for panels, choose FE/RE or dynamic GMM.
• Diagnostics & tuning
  Check residual white noise, stability, parameter bounds, panel cluster-robust SEs, and prediction interval coverage; introduce exogenous x_t or transformations as needed.
• Forecast & rolling evaluation
  Produce hat{y}_{t+h} and intervals; compute rolling NRMSE/MAE/CRPS; record latency_ms_p99 and resource usage.
• Publish & persist
  Use emit_ts_manifest to write model signature, parameters, diagnostics, SLOs, TraceID, and (if used) the two-arrival-time delta_form.

VI. Contracts and Assertions C30-111x

• C30-1111 (Stationary/invertible): roots(phi) outside unit circle and roots(theta) outside unit circle; for state space, max_abs_eig(A) < 1 or explicitly document non-stationarity.
• C30-1112 (White-noise residuals): LB_p pval ≥ 0.05 and max_k |ACF_k| ≤ band_k.
• C30-1113 (Coverage & error): PI_coverage ∈ [target - tol_cov, target + tol_cov]; NRMSE ≤ tol_nrmse.
• C30-1114 (Panel robustness): cluster-robust SEs are bounded; for dynamic GMM, Hansen_p ≥ 0.05; no second-order autocorrelation with AR(2) pval ≥ 0.05.
• C30-1115 (Time-base & arrival time): if the model includes T_arr, enforce delta_form ≤ tol_Tarr; offset/skew/J ≤ policy.max.
• C30-1116 (Resources & latency): latency_ms_p99 ≤ SLO_latency; memory footprint and model size within quota.

VII. Implementation Bindings I30-*

• I30-111 time_align_for_stats(ds, sync_ref) -> ds' (attach offset/skew/J and both formulations)
• I30-112 fit_arima(y, x_exog, order, seasonal) -> model
• I30-113 fit_state_space(y, ss_spec) -> {A,B,C,D,Q,R,init}
• I30-114 fit_its(y, t, T0, z, arma_spec) -> model
• I30-115 fit_panel_fe(ds, two_way, cluster) -> model
• I30-116 fit_panel_gmm(ds, spec) -> {params, hansen_p, ar2_p}
• I30-117 forecast_ts(model, h, exog_future) -> {y_hat, PI}
• I30-118 residual_diagnostics(model) -> {LB_p, ACF_band, NRMSE, PI_coverage}
• I30-119 evaluate_ts_contracts(report, rules) -> contract_report
• I30-11A emit_ts_manifest(results, policy) -> manifest.stats.ts

Invariants: manifest.stats.ts.TraceID is unique; contract_report.pass == true is required for publication; unit(y_hat) == unit(y); rolling evaluation window coverage ≥ cov_min.

VIII. Cross-References

• Time axis & synchronization: see Methods.Cleaning v1.0, Chapter 5.
• Arrival-time formulations & path: see Methods.Cleaning v1.0, Chapter 6.
• Multiple comparisons & sequential budgeting (multi-window alerts): see Chapter 6 of this volume.
• Drift detection & baseline refresh: see Chapter 7.
• Imaging timelines & arrival-time consistency: see Methods.Imaging v1.0, Chapter 13.

IX. Quality & Risk Control

• SLI/SLO
  LB_p pval ≥ 0.05; PI_coverage@0.95 ≥ 0.92; NRMSE ≤ target; latency_ms_p99 ≤ 500; retrain_interval_days ≤ 14.
• Risk controls
  Failure paths: if C30-1112 fails, increase order or retune state noise Q,R; if C30-1114 fails, reduce instrument count or revise moment conditions; on drift beyond threshold, trigger retraining and fallback to the last frozen parameters.

Summary