Deep Field Labs · published 2026-10-04 · v1, frozen on publication · source: docs/dfl-prediction-grading-standard-v1.md. The live numbers it governs are on the track record; how we compute them is on methods.

DFL Prediction Grading Standard — v1

Status: v1 — FROZEN on publication, 2026-10-04 (DecayGuard public launch). Later changes create v2; v1 is never edited. Published at https://decayguard.deepfieldlabs.dev/grading-standard.html

The problem this solves: every vendor quotes an accuracy number; almost none say what would have counted as a miss, when a prediction is allowed to be judged, or whether the history behind the number can be checked. This standard defines those terms precisely enough that any vendor's re-entry (or comparable event) predictions can be graded the same way. We hold ourselves to it in public; buyers are invited to ask every vendor the questions at the bottom.

1. The prediction record

A gradable prediction is a dated, immutable claim:

One open claim per object: re-issuing daily while a claim is open would inflate the sample. The earliest call is the graded one.

2. Grading states

• confirmed — independent evidence (for re-entry: a Space-Track Historical decay message; forecasts are never treated as truth) dates the event inside the window, after the issue date.
• expired — the window closed AND the evidence grace elapsed (§3) with no confirming evidence. This is a miss and it stays on the record.
• void — evidence shows the event predates the claim (predicting the past). Removed from the denominator, disclosed in counts.

Grading is causal: evidence dated after the grading clock is invisible to that grading run. Nothing may be graded with information that did not exist at grading time.

3. Evidence grace

Evidence publishes late (measured on Space-Track Historical decay messages: median ≈ 6 days, p90 ≈ 26 days after the event). A window that closed yesterday is therefore not yet a miss. A prediction only matures — becomes judgeable — once window_end + grace has passed. DFL uses grace = 28 days, calibrated from that lag distribution and recalibrated as the archive grows. Publishing precision over unmatured cohorts is survivorship bias in the favourable direction; this standard forbids it in the headline.

4. The three precision figures (all published, one headline)

"Backlog cohort": predictions issued while first draining a pre-existing alert backlog measure the backlog, not the system; they are excluded from steady/mature and included in closed. The cutoff date is fixed and published (backlog_cohort_end).

n_pending_grace (issued, window closed, not yet matured) must be published next to any precision figure.

5. Capture (the other half)

Precision without capture rewards silence. capture_decay = of the observed events in the period that the system could have seen (object in its candidate universe before the event), the fraction an issued prediction warned about, with median_lead_days alongside. Both numbers published; a precision quoted without its capture is not compliant with this standard.

6. Verifiability (what makes any of this checkable)

• Receipts chain: every emission file hashed at issue time; running SHA-256 chain, head republished daily.
• Anchor chain: one line per day hashing every published artifact + the previous line. Dated artifacts immutable once anchored.
• Independent verifier: a standard-library script re-downloads everything over HTTPS and recomputes all hashes. Published with the data.

A track record that cannot be independently re-hashed is a claim, not a record.

7. Pre-registration

Go/no-go launch criteria, metric definitions, and any candidate change to the interpretation are frozen in writing before the evaluation runs. A negative result (a rejected candidate) is published to the same standard as a positive one.

8. Questions to ask any prediction vendor

1. What is your evidence-mature precision — and what grace period and cohort rules produced it? 2. What is your capture rate over the same period, and its median lead time? 3. How many predictions are currently pending grace? 4. Can I re-verify your published history cryptographically, without trusting your servers? 5. Were your evaluation criteria written down before or after the results existed?

DFL's answers are on the DecayGuard track-record page, recomputed nightly, hash-anchored daily.

Exact-text fingerprint of this standard (SHA-256 of the source markdown): 6efc07595a3ff3552b37dc6c34de08fca03b4e7ef85ca180bddc15fa10e01103. If this page ever differs from that hash, it is not v1.