52-Dataset Card Template v1.0 | Chapter 6 — Splits, Versioning & Freshness

Home ／ Docs-Technical WhitePaper ／ 52-Dataset Card Template v1.0

Chapter 6 — Splits, Versioning & Freshness

I. Purpose & Scope

• Standardize management and validation of dataset splits, versioning (SemVer), and freshness/validity, ensuring independence & comparability for train/val/test, traceable version evolution, and consistent timing & calibration freshness.
• For samples involving path quantities (arrival time/phase), the text must explicitly show gamma(ell) and d ell, and the data side records delta_form ∈ {general, factored}; publication requires p_dim = 1.0 with check_dim_report.json attached.

II. Prerequisites & Inputs

• Structure & contract: schema.json/contract.yaml complete (Ch. 4), fields/units/dimensions aligned with TARR.
• Provenance & lineage: provenance.yaml/lineage_graph.json complete (Ch. 3); split operations recorded in lineage DAG.
• Metrology & coverage: aligned with Error Budget (cov_group/Σ, coverage ∈ {k, alpha, quantile}); version/freshness aligned with Parameter Card.
• Citations & versioning: “volume + version + anchor (P/S/M/I)”, anchor coverage ≥ 90%, no external links/aliases.

III. Splits Strategy

1. Split set: train / val / test / holdout / slice_k; each split uniquely named in split.yaml with intent documented.
2. Leakage prevention:
   • Temporal leakage: cut by time windows (TS → {train < val < test}), forbid cross-window mixing.
   • Entity leakage: split by group_by(entity) so entities do not cross splits.
   • Path consistency: within any split, arrays gamma_ell/d_ell/n_eff have consistent length and sampling step.
3. Stratification & sampling: stratify by batch/device/region/quality.flags; maintain class/hard-case ratios when needed.
4. Slices: define slice_k for key subgroups (extreme conditions/low SNR/specific regions) with explicit selection rules.
5. Reproducibility: record random seed, algorithm, and params in split.yaml; produce split_manifest.json with counts & checksums.

IV. Versioning (SemVer)

• MAJOR: breaking changes to schema/semantics/units/dimensions or split semantics (must provide migration guide & rollback point).
• MINOR: backward-compatible additions (new fields, new slice_k, or extra metadata).
• PATCH: fixes to defaults/descriptions/minor errors without semantic change.
• Tagging: root dataset version and per-split versions labeled in manifest.yaml/split_manifest.json; public releases use v1.* only.
• Dependency closure: synchronize with Pipeline/Parameter/Error volumes’ see[]/references[]/version; update audit.jsonl on version changes.

V. Freshness / Validity

• Fields: freshness{ valid_from, valid_to, policy{ tau_calib_s_max, clock_state } }.
• Decision: fresh if |ts_start − calib.timestamp| ≤ tau_calib_s_max and clock_state="locked"; otherwise isolate or tag [Restricted].
• Rolling update: produce the next version before valid_to and switch seamlessly; retain prior versions for backtracking & comparative evals.
• Expiry treatment: expired samples only for historical recompute/diagnostics; do not include in externally quantitative splits.

VI. Gate Mapping

• G1 | Schema completeness: split.yaml/split_manifest.json fields complete; indices & counts consistent.
• G2 | Citation compliance: see[]/references[] anchor coverage ≥ 90%.
• G3 | Path conventions: path block gamma/measure/delta_form present; len(path)≥2, Δell compliant.
• G4 | Dimensional closure: pass I70-dim_check, require p_dim = 1.0.
• G5 | Freshness: clock_state="locked", τ_calib valid; expired samples isolated.
• G6 | Coverage: statistical intervals within splits use the same mode as publication (k/alpha/quantile).
• G7 | Covariance consistency: stratification/slices’ assumptions for cov_group/Σ align with the Error Budget; Σ PD.
• G8 | Uniqueness: record_id unique within a split; across splits no duplication unless slice_k explicitly shares.
• Stops (S1–S5): any core-gate failure → reject/rollback; tag [Restricted] when needed.

VII. Machine-Readable Configs
A. split.yaml

version: "1.0.0"

seed: 20250924

strategy:

group_by: ["entity_id"]

time_ordered: true

splits:

train: 0.70

val: 0.15

test: 0.15

constraints:

leakage:

time: { enforce: true }

entity: { enforce: true }

path:

require_alignment: true

delta_form: "general"

coverage:

mode: "k" # k|alpha|quantile

k: 2

B. split_manifest.json (excerpt)

{

"dataset_version": "1.2.0",

"splits": {

"train": { "count": 120345, "checksum": "sha256:..." },

"val": { "count": 25780, "checksum": "sha256:..." },

"test": { "count": 25812, "checksum": "sha256:..." }

},

"slices": {

"low_snr": { "count": 8142, "rule": "snr<5" }

},

"freshness": { "valid_from": "2025-09-01T00:00:00Z", "valid_to": "2026-03-01T00:00:00Z",

"policy": { "tau_calib_s_max": 86400, "clock_state": "locked" } }

}

C. version_matrix.yaml

dataset: "ds-core"

current: "1.2.0"

compatibility:

"1.2.x": { api: ">=1.2,<2.0", schema: ">=1.2,<2.0" }

"1.1.x": { api: ">=1.1,<1.3", schema: ">=1.1,<1.3" }

migration:

from: "1.1.x"

to: "1.2.x"

steps:

- change: "add slice 'low_snr'"

- change: "add field quality.score_Q"

rollback:

tag: "v1.1.3-lock"

VIII. Anti-Patterns & Fixes

• Anti: sharing the same entity across splits → Fix: group_by(entity) and deduplicate.
• Anti: T_arr = ∫ n_eff / c_ref d ell (missing parentheses) → Fix: T_arr = ( ∫ ( n_eff / c_ref ) d ell ).
• Anti: missing seed/strategy leads to non-reproducibility → Fix: pin RNG and algorithm in split.yaml.
• Anti: expired samples included in train → Fix: filter per freshness.policy or isolate & tag.
• Anti: coverage modes differ between data and publication → Fix: unify coverage.mode and params.

IX. Cross-References

• Source & lineage: Ch. 3; Structure & Schema: Ch. 4; Gates & Integrity: Ch. 7; Uncertainty & Covariance: Ch. 8.
• Pipeline Card: inbound contracts (Ch. 4), gates & monitoring (Ch. 9), outputs & release (Ch. 12).
• Error Budget Card: threshold mapping (Ch. 9), intervals & coverage (Ch. 8).

X. Checklist

• split.yaml/split_manifest.json complete & consistent, recording seed/strategy/count/checksum.
• Leakage prevention in effect (time/entity); group_by and window ordering explicit.
• For path quantities, explicit gamma/measure/delta_form; len(path) ≥ 2, Δell compliant; I70-dim_check passed, p_dim = 1.0.
• Versioning follows SemVer; compatibility matrix and migration/rollback notes complete; manifest and audit.jsonl updated.
• Freshness policy enforced (clock_state="locked", τ_calib valid); expired samples isolated or [Restricted].
• /validate passes G1–G8; citations anchor coverage ≥ 90%; release bundle signed with checksums.