Evaluation cases
Loading selected evaluation cases…
Leaderboard
Evaluation date: July 19, 2026
1,000Foundational evaluation items
900Research proof variants
9Models in latest comparison
2024-2025Research test articles
| Rank | Model | Grad Prob. | Grad Stat. | Grad ML | Grad Mean | Undergrad Prob. | Undergrad Stat. | Undergrad ML | Undergrad Mean | Overall |
|---|
Benchmark structure
Two datasets span textbook knowledge and research proofs.
Foundational knowledge
- 22,262 tasks from 76 public sources.
- Undergraduate and graduate levels.
- Probability, statistics, and machine learning.
Research reasoning
- 84,179 proof tasks from 6,953 papers (2000–2025).
- Six statistics and machine learning journals.
- Easy, Medium, and Hard variants.
Overall benchmark overview
Foundational composition
Research composition
TRACE pipeline
From papers to self-contained theorem reasoning tasks.
1. Convert
Normalize PDFs and isolate formal components.
2. Harmonize
Consolidate notation and assumptions.
3. Link
Build the dependency graph.
4. Vary
Generate three difficulty levels.
5. Patch
Repair local artifacts without deletion.
6. Validate
Run automated checks and expert review.
TRACE pipeline overview
Distributed proof extraction
Notation harmonization
Context-Aware Patching
Evaluation protocol
Exact matching for closed questions; process-based scoring for derivations.
Logic alignment
Route reference-aligned and alternative solutions.
Reference-based scoring
Score aligned solutions against atomic proof steps.
Independent verification
Verify alternatives for logic, precision, and accuracy.
Scoring pipeline
Evaluation case