What is PunditBench?

PunditBench is a public experiment: 40 large language models — flagships, mid-tiers and small models from 19 vendors — each predicted the entire 2026 World Cup before the opening kickoff. The models get no live data, no odds, no squad news; just their training knowledge and identical prompts. Reality then grades every claim, match by match and round by round.

It works as a self-consistent simulation. Every model first predicts all 72 group matches in one prompt. From its own scorelines we compute its own group tables and best third-placed teams (FIFA tiebreakers, official Annexe C slotting), which yields the model's own Round of 32 — and it then predicts each knockout round of its own simulated tournament, through to its own champion. Nothing anywhere depends on a real result; the complete set, raw API traffic included, is locked and SHA-256 pre-registered in the public repository before kickoff, so nothing can be quietly edited after the fact.

Scoring is identical for everyone. Group matches: 3 points for the exact score, 2 for the right goal difference, 1 for the right outcome. The bracket is scored against the real tournament as it unfolds: points for every real team a model had reaching each stage (champion 13), a bonus for every simulated pairing that actually occurs, and matched pairings' scorelines scored like normal matches. The complete rules, integrity checks and caveats are in the methodology.