24-EFT.WP.Particle.TopologyAtlas v1.0 | Appendix F — Datasets and Benchmarks

Home ／ Docs-Technical WhitePaper ／ 24-EFT.WP.Particle.TopologyAtlas v1.0

Appendix F — Datasets and Benchmarks

One-Sentence Goal
Provide public datasets, evaluation protocols, and benchmark metrics for topological-atlas tasks to ensure comparability, reproducibility, and auditability across teams.

I. Scope and Objects

• Data modalities: 2-D/3-D scalar fields x(r), phase/orientation fields in S^1 / S^2 (θ(r), n(r)), spatiotemporal series x(r,t), and worldline sets Γ(t).
• Evaluation targets: topological density q(x), persistence diagrams D and landscapes λ_D, atlas Atlas = { U_a, φ_a, ψ_a, T_ab }, invariants Inv = { Q, Lk, Tw, Wr, ... }, retrieval embeddings Z, runtime indicators, and metrics stored in manifest.topo.*.
• Outputs: benchmark task definitions, scoring formulas, submission & verification workflows, licensing & provenance fields.
• Boundaries: all evaluations must use consistent RefCond, τ_policy, and unit systems; otherwise scores are invalid.

II. Terms and Variables

• Dataset strata: DS.syn (analytical/controllable synthetic), DS.sim (physics simulation), DS.exp (real-world experimental), DS.stream (streaming).
• Splits & difficulty: split ∈ { train, val, test, test-hard, OOD }; noise levels { low, med, high }.
• Evaluation kernels: d_B(D̂, D_ref), W_p(D̂, D_ref), Δ_cyc, sup|Σψ−1|, Recall@K, delta_form, delta_form_rt.
• Licensing & lineage: license, provenance, seed, data_hash, RefCond.

III. Dataset Catalog & Contents (suggested)

• DS.syn.GL-2D: Ginzburg–Landau phase fields with vortices; analytic Q = Σ winding; provide θ(r), q(r), Q_gt.
• DS.syn.O3-SkX: S^2 orientation fields for skyrmion text patterns; Q ∈ Z, includes half-integer noise variants.
• DS.syn.Hopf-3D: Hopf fiber fields; controllable ground-truth linkings Lk_gt; voxelized orientation fields and Γ_ref.
• DS.sim.FLUID-PIV: vorticity fields with unwrapping residuals; test robustness of q(x) and D.
• DS.exp.MFM-SKX: magnetic-force microscopy skyrmion pairs; paired orientation reconstructions.
• DS.exp.HOLO-PHASE: interferometric phases and wraps; for cleaning-chain and topology-conservation evaluations.
• DS.stream.RECONNECT: reconnection events in x(r,t); include E_reconn labels and windowed invariant changes.
• OOD add-ons: configurations with mismatched noise/scale/resolution for generalization tests.

Each sample includes:

• data/ (raw & intermediates); gt/ (analytic or reference products Q_gt, Γ_ref, D_ref);
• meta.json with TraceID, license, provenance, RefCond, units/dim, seed, split, noise;
• README.md (generation/acquisition notes, coordinates, masks).

IV. Axioms P90F-*

• P90F-1 (Traceability) — Every data file must include data_hash, provenance, and license.
• P90F-2 (Scale alignment) — Before comparison, declare RefCond and τ_policy, then normalize scales.
• P90F-3 (Parallel forms) — If an algorithm exposes formA/B or stream/offline, submissions must include both and delta_form.
• P90F-4 (No leakage) — test / test-hard / OOD splits must not use supervision/meta for fitting; audit scripts check for leakage.
• P90F-5 (Explicit units/dimensions) — require unit(field) and dim(field) with check_dim pass.

V. Minimal Equations S90F-* (Scoring & Aggregation)

• S90F-1 (Invariant error): err_Inv = Σ_{i∈Inv} w_i * | Inv̂_i − Inv_{i,gt} | / scale_i.
• S90F-2 (PH error): err_PD = α * d_B(D̂, D_ref) + (1−α) * W_p(D̂, D_ref), α ∈ [0,1].
• S90F-3 (Atlas quality):
  score_atlas = exp( − ( Δ_cyc / tol_cyc + sup|Σψ−1| / tol_ψ + κ / κ_max ) ), with κ = cond_max(∂φ).
• S90F-4 (Retrieval metric):
  score_retr = 0.5 * ( Recall@K / R_min ) + 0.5 * ( 1 − mean(d_exact) / tol_rank ).
• S90F-5 (Streaming consistency):
  penalty_stream = β1 * p95(delta_form_rt) / tol_rt + β2 * lag_ms_p95 / ( 2 * τ_window ).
• S90F-6 (Overall score):
• Score = GM( clip(1 − norm(err_Inv)),
• clip(1 − norm(err_PD)),
• score_atlas,
• score_retr,
• clip(1 − penalty_stream) )

where GM is the geometric mean and clip(z) = min(max(z, 0), 1); norm(•) is normalized by baseline stats.
unit(Score) = "[1]"; report per-task subscores and the total.

VI. Evaluation Workflow M90-F1 (Submit → Verify → Publish)

1. Prepare & self-test
   • Pull the eval-kit and canonical RefCond; generate manifest.topo.case/atlas/runtime on val.
   • Verify dual-form records, check_dim, and contracts (Appendix B).
2. Produce submission
   • For each sample, write manifest.* and artifact references; aggregate into submission.jsonl (one result URI or hash:// per line).
   • Include code.ver / git / seed / runtime and deps.
3. Online evaluation
   • The evaluator recomputes D_ref / Inv_ref or loads references and applies S90F-*;
   • Verify license / provenance and P90F-*; leakage → invalidate submission.
4. Publish & archive
   • Generate report.json with contract outcomes and per-part scores;
   • Write to the leaderboard and dashboard mappings.

VII. Contracts & Gates (C90F-*)

• C90F-01 (Completeness): submission must include manifest.*, submission.jsonl, signature; otherwise fail.
• C90F-02 (Invariant gate): err_Inv ≤ tol_Inv and integer-invariant rounding meets Appendix E C90-166.
• C90F-03 (PH stability): d_B ≤ tol_B and W_p ≤ tol_Wp.
• C90F-04 (Atlas gate): Δ_cyc ≤ tol_cyc, sup|Σψ−1| ≤ 1e-6.
• C90F-05 (Retrieval gate): Recall@K ≥ R_min and mean(d_exact) ≤ tol_rank.
• C90F-06 (Streaming gate): p95(delta_form_rt) ≤ tol_rt and lag_ms_p95 ≤ 2 * τ_window.
• C90F-07 (License gate): license ∈ { Apache-2.0, MIT, CC-BY-4.0, CC-BY-SA-4.0 } or an explicit commercial license; otherwise no score.
• C90F-08 (Reproducibility): missing seed / deps / code.ver or replay failure → degrade or reject.

VIII. Implementation Bindings I90-* (Data & Evaluation)

• I90-201 load_dataset(uri, split, filters) -> iterator(samples)
• I90-202 render_refcond(meta) -> RefCond
• I90-203 compute_reference_products(sample, policy) -> { Q_gt, Γ_ref, D_ref }
• I90-204 evaluate_submission(submission, refs, rules) -> report
• I90-205 aggregate_scores(per_sample, scheme = "GM") -> leaderboard_entry
• I90-206 check_leakage(train_meta, test_meta) -> { pass, evidence }
• I90-207 emit_benchmark_manifest(results) -> manifest.bench

Invariants: non_decreasing(τ); Σ_a ψ_a ≡ 1; check_dim(all); parallel dual forms with delta_form persisted.

IX. Cross-References
Ch. 2 (math & topology), Chs. 3/5/6 (object/event labeling), Ch. 7 (complexes & filtrations), Ch. 8 (persistence stability & landscape norms), Chs. 9/10/11 (atlas/transition quality), Ch. 12 (noise & cleaning), Ch. 13 (retrieval metrics), Ch. 14 (runtime fields & panel mapping), and Appendices A/B/C/E/D for interfaces, contracts, manifests, and alert thresholds.

X. Quality & Risk Control

• Suggested SLOs: d_B_p95 ≤ 0.02 * scale_τ; Δ_cyc ≤ 1e-3; Recall@10 ≥ 0.9; coverage ≥ 0.95.
• Audit checklist: verify license / provenance / data_hash; spot-check replay consistency; cross-run on two independent environments and compare delta_form.
• Fallback: fail-fast → minimal-subset recheck → rescore → publish diffs, logging evidence hashes and signatures end-to-end.

Summary
This appendix defines dataset families, evaluation protocols, and aggregation scores for the Topological Atlas, closing the loop data → algorithms → manifests → leaderboard via P90F / S90F / M90-F1 / C90F / I90. With strict RefCond, dual forms, and explicit units/dimensions, results across implementations are comparable, reproducible, and auditable.