StatEval: A Comprehensive Benchmark for Large Language Models in Statistics

Evaluating LLMs across foundational statistics and research-level reasoning.

Evaluation cases

StatEval case browser
Loading selected evaluation cases…

Leaderboard

Evaluation date: July 19, 2026
1,000Foundational evaluation items
900Research proof variants
9Models in latest comparison
2024-2025Research test articles
Rank Model Grad Prob. Grad Stat. Grad ML Grad Mean Undergrad Prob. Undergrad Stat. Undergrad ML Undergrad Mean Overall

Benchmark structure

Two datasets span textbook knowledge and research proofs.

Foundational knowledge

  • 22,262 tasks from 76 public sources.
  • Undergraduate and graduate levels.
  • Probability, statistics, and machine learning.

Research reasoning

  • 84,179 proof tasks from 6,953 papers (2000–2025).
  • Six statistics and machine learning journals.
  • Easy, Medium, and Hard variants.
StatEval overview
Overall benchmark overview
Foundational dataset composition
Foundational composition
Research dataset composition
Research composition

TRACE pipeline

From papers to self-contained theorem reasoning tasks.

1. Convert

Normalize PDFs and isolate formal components.

2. Harmonize

Consolidate notation and assumptions.

3. Link

Build the dependency graph.

4. Vary

Generate three difficulty levels.

5. Patch

Repair local artifacts without deletion.

6. Validate

Run automated checks and expert review.

TRACE pipeline
TRACE pipeline overview
Distributed proof case
Distributed proof extraction
Notation harmonization case
Notation harmonization
Context-aware patching case
Context-Aware Patching

Evaluation protocol

Exact matching for closed questions; process-based scoring for derivations.

Logic alignment

Route reference-aligned and alternative solutions.

Reference-based scoring

Score aligned solutions against atomic proof steps.

Independent verification

Verify alternatives for logic, precision, and accuracy.

Adaptive evaluation pipeline
Scoring pipeline
Evaluation case study
Evaluation case