Executed through the lab protocol with an immutable artifact, repeat measurements and a valid receipt.
A result is only as useful as its receipt.
The lab preserves raw measurements, versions every formula and makes uncertainty visible. Estimates help choose a test; they never become tests themselves.
EVIDENCE STATES
Five labels, no blending
A submitted result whose artifact, environment and receipt passed consistency review.
A named vendor or third-party source. Attribution is visible and reproduction is not implied.
A versioned formula using source-backed inputs and an explicit uncertainty range.
The evidence is insufficient. Unknown is an acceptable and useful answer.
ESTIMATOR v0.1.0
Memory calculation
peak = weights + KV cache + runtime range + device reserveWeights use parameter count × estimated bits per weight. KV cache uses the model’s full-attention layers, KV heads, head dimension, K/V pair and cache precision. Runtime allocation is shown as a range. Discrete GPUs reserve 6%; unified-memory systems reserve 12% for the operating system and shared use.
RUN PROTOCOL
What an observed run requires
- Lock the artifact.Exact filename, size, SHA-256, repository and revision.
- Lock the environment.Hardware, OS, driver, runtime build and command arguments.
- Warm up once.Exclude first-run compilation and allocation effects.
- Measure at least three times.Record raw runs, median, spread and anomalies.
- Probe the boundary.Keep failures and OOMs; do not publish only the successful case.
- Hash the receipt.Review before changing evidence state to Observed.
CORRECTIONS
History stays visible
A source correction or estimator change creates a dataset version and changelog entry. Raw run receipts are never rewritten. Material changes update the reviewed date; cosmetic edits do not.