Run the synthetic harness
One deterministic command runs the TargetSpace v1.0 scoring spine end to end on a known synthetic process — proper scores, R1 and R2 baselines, calibration, the permutation specificity check, and the evidence-tier ablation. It is a smoke test and a reference path, not a benchmark result.
Before you run
The harness is built to be trivial to reproduce: no install step, no packages, no network, no data to download.
Run it
From the repository root, run the demonstration in examples/:
python examples/targetspace_synthetic_demo.py
A single-file copy of the same demonstration ships as an ancillary file at the repository root, so it can be run without the surrounding tree:
python targetspace_synthetic_demo.py
Both entry points run the identical scoring spine and print the same result. Use whichever matches how you obtained the repository.
Expected output
The run prints a short, deterministic report: finite proper scores for each predictor, skill in bits over R1 and over R2, a calibration summary, the evidence-tier ablation, and the permutation check. It ends with a single-line smoke assertion. Magnitudes below are artifacts of the synthetic settings and carry no external meaning.
Proper scores over the sealed evaluation set -------------------------------------------------------------------------- predictor logloss(bit) Brier skill/R1(bit) skill/R2(bit) R1 population prior 0.8376 0.1957 0.0000 -0.0130 R2 own routine 0.8245 0.1932 0.0130 0.0000 L1 R2 + digital 0.8244 0.1931 0.0132 0.0002 L2 R2 + attention 0.7891 0.1829 0.0485 0.0355 Permutation specificity (toy L2 skill over R1: true vs wrong-target) -------------------------------------------------------------------------- true target-specific skill : +0.0485 bits/forecast permuted (wrong-target) skill: -0.0594 bits/forecast => matched to another target, the skill vanishes. [smoke test] PASSED: scores finite; attention adds lift; permutation collapses.
The final line is the pass condition: every score is finite, the informative evidence tier adds lift over the own-routine baseline R2, and the measured skill collapses under matched target permutation. If any of those fails, the assertion does not print PASSED.
What it verifies
The harness exercises the computational spine of the v1.0 protocol on synthetic data, so you can confirm the pieces run and compose:
Verified by the run
- Schemas — forecast instances and sealed probability vectors over a discrete answer space are constructed and consumed in the expected shape.
- Metric computation — log score in bits and Brier are computed over the full probability distribution.
- R1 / R2 baseline execution — the population-prior baseline (R1) and the walk-forward own-routine baseline (R2) both run, and skill is measured in bits over each.
- Calibration checks — a reliability summary and a coarse ECE diagnostic are produced from the sealed forecasts.
- Permutation controls — the specificity gate runs: skill is re-scored against a wrong target and is shown to collapse.
- Evidence ablation — the evidence-tier comparison runs, measuring the lift each additional stream adds over R2 rather than assuming it.
What it does not verify
This is a synthetic demonstration, not evidence about the world. It uses no human data and provides no empirical validation: it does not show that any system has target-specific predictive skill, does not report a benchmark result or leaderboard standing, and makes no product claim. A passing smoke test confirms only that the scoring spine computes end to end on a known synthetic process. It does not certify calibrated prospective predictive skill on real targets, and it says nothing about understanding, inner life, causation, or permission to act on any person.
Where to look next
The examples directory
The demonstration source and its inline documentation live in examples/ in the public repository.
The schemas
The forecast-instance and submission schemas the harness reads and writes are specified separately.
A real-data submission follows the same spine on compliant, consented longitudinal evidence. Consent, privacy, and governance requirements are binding: derived, consented artifacts only, never raw bystander media. See the submission page and the baselines.