Overall accuracy
100.0%36 / 36Exact label matches across all fixed cases, including abstentions.
A FIXED SET. AN OPEN NOTEBOOK.
Show your work.
Every label, every abstention, every source reference.
A reproducible snapshot of the deterministic baseline.
36
synthetic cases
This report describes an invented set, not real-world performance. Human reviews do not change the baseline or expected labels.
Abstention rate
25.0%9 / 36Cases where the baseline asks for human clarification.
Ambiguity precision
100.0%9 / 9Abstentions that belong to ambiguity-expected cases.
Unsupported statements
0.0%0 / 85Reasons without valid source fields, across all factual reasons.
CATEGORY BY CATEGORY
Accuracy & confusion counts
Per-category accuracy counts correct non-abstained proposals over expected-category support. Expected abstentions therefore show 0% here, while matching abstentions count toward overall accuracy.
| Expected category | Accuracy | Support | Ready for review | Needs evidence | Out of scope | Abstain |
|---|---|---|---|---|---|---|
| Ready for review | 100.0%9 / 9 | 9 | 9 | 0 | 0 | 0 |
| Needs evidence | 100.0%9 / 9 | 9 | 0 | 9 | 0 | 0 |
| Out of scope | 100.0%9 / 9 | 9 | 0 | 0 | 9 | 0 |
| Abstain | 0.0%0 / 9 | 9 | 0 | 0 | 0 | 9 |
WHERE LABELS DIFFER
Disagreements 0
All fixed labels match.
36 of 36 expected labels match, including 9 abstentions. These fixtures exercise the documented rules; they do not establish generalization.
Differences are ordered by expected category, rule version, then case ID.
REPRODUCIBLE BY DESIGN
The fixture notebook
- Entirely invented records
- 64
- Ambiguity-expected records
- 16
- Fixed / exploratory records
- 36 / 28
- Generator seed
- 23092026
- Schema
- claims/1.0.0
Frozen IDs and expected labels live in data/evaluation.v1.json. Changes require a recorded version. Zero denominators display N/A.
Inspect all 36 frozen cases
| Synthetic case | Expected | Proposed | Ambiguity expected | Result |
|---|---|---|---|---|
| CB-0001 ↗ | Ready for review | Ready for review | No | Match |
| CB-0003 ↗ | Ready for review | Ready for review | No | Match |
| CB-0005 ↗ | Ready for review | Ready for review | No | Match |
| CB-0007 ↗ | Ready for review | Ready for review | No | Match |
| CB-0009 ↗ | Ready for review | Ready for review | No | Match |
| CB-0011 ↗ | Ready for review | Ready for review | No | Match |
| CB-0013 ↗ | Ready for review | Ready for review | No | Match |
| CB-0015 ↗ | Ready for review | Ready for review | No | Match |
| CB-0016 ↗ | Ready for review | Ready for review | No | Match |
| CB-0017 ↗ | Needs evidence | Needs evidence | No | Match |
| CB-0019 ↗ | Needs evidence | Needs evidence | No | Match |
| CB-0021 ↗ | Needs evidence | Needs evidence | No | Match |
| CB-0023 ↗ | Needs evidence | Needs evidence | No | Match |
| CB-0025 ↗ | Needs evidence | Needs evidence | No | Match |
| CB-0027 ↗ | Needs evidence | Needs evidence | No | Match |
| CB-0029 ↗ | Needs evidence | Needs evidence | No | Match |
| CB-0031 ↗ | Needs evidence | Needs evidence | No | Match |
| CB-0032 ↗ | Needs evidence | Needs evidence | No | Match |
| CB-0033 ↗ | Out of scope | Out of scope | No | Match |
| CB-0035 ↗ | Out of scope | Out of scope | No | Match |
| CB-0037 ↗ | Out of scope | Out of scope | No | Match |
| CB-0039 ↗ | Out of scope | Out of scope | No | Match |
| CB-0041 ↗ | Out of scope | Out of scope | No | Match |
| CB-0043 ↗ | Out of scope | Out of scope | No | Match |
| CB-0045 ↗ | Out of scope | Out of scope | No | Match |
| CB-0047 ↗ | Out of scope | Out of scope | No | Match |
| CB-0048 ↗ | Out of scope | Out of scope | No | Match |
| CB-0049 ↗ | Abstain | Abstain | Yes | Match |
| CB-0051 ↗ | Abstain | Abstain | Yes | Match |
| CB-0053 ↗ | Abstain | Abstain | Yes | Match |
| CB-0055 ↗ | Abstain | Abstain | Yes | Match |
| CB-0057 ↗ | Abstain | Abstain | Yes | Match |
| CB-0059 ↗ | Abstain | Abstain | Yes | Match |
| CB-0061 ↗ | Abstain | Abstain | Yes | Match |
| CB-0063 ↗ | Abstain | Abstain | Yes | Match |
| CB-0064 ↗ | Abstain | Abstain | Yes | Match |