Ten LLMs Forecast the World Cup. One Picked Spain to Win.
A benchmark had ten LLM assistants forecast the entire 2026 World Cup from identical inputs, then scored them against the real results. Only GPT-5.5 Thinking named champion Spain — and the scoring system rewarded that single bracket call far more than accurate group-stage predictions.
Researchers running a benchmark called AI World Cup 2026 had ten LLM-based assistants each submit one forecast for the entire 2026 FIFA World Cup before the tournament began, working from an identical tournament snapshot, prompt template, JSON schema and scoring procedure. After all 104 matches had been played, the forecasts were scored against the real results. OpenAI's GPT-5.5 Thinking finished first with 744 points — the only one of the ten to name Spain as champion. Spain beat Argentina 1-0 in the July 19 final at MetLife Stadium, on a goal from substitute Ferran Torres in the 106th minute of extra time.
The other nine assistants split across three teams that did not win. Four of the ten — GPT-5.5, DeepSeek, and both of Perplexity's entries — picked Brazil. Three — Gemini, Mistral Medium 3.5 and Grok — picked France. Two, Qwen 3.7 and Claude Sonnet 4.6, picked Argentina, the side Spain actually beat in the final.
The scoring system explains why the champion pick mattered so much more than anything else. Group-stage matches, of which there were 72, paid out on a sliding scale: 5 points for an exact score, 3 for the correct outcome, 2 for picking the winner without the draw, 1 for the correct goal difference. Knockout-stage scoring ran on a separate, steeper track: 2 points per correctly advanced team at the round of 32, climbing to 12 points per correct finalist, plus a flat 20-point bonus for naming the champion outright. Across the ten models, knockout points correlated almost perfectly with final rank (r=0.986). Group-stage points barely correlated with final rank at all (r=0.055).
That split produced a real inversion. Claude Sonnet 4.6 had the field's best group-stage outcome accuracy, at 63.89% of the 72 matches — well ahead of every other model — yet finished sixth of ten overall, because its knockout-round score was 68 points, among the lowest in the field. GPT-5.5 Thinking, by contrast, tied for the field's worst group-stage outcome accuracy at 58.33%, matched by five other models, and converted only 11.11% of group matches into exact scores. It still won, because its knockout score of 242 points was the highest of the ten, built on carrying Spain to the title.
Confidence values submitted alongside each forecast tracked accuracy poorly. Every model reported a numeric confidence for its predictions, and across the field, self-reported confidence correlated with actual accuracy at essentially zero (r=-0.060) — models were not reliably more confident when they were right than when they were wrong, a calibration gap that shows up repeatedly in other LLM benchmark work.
The study, by Jonaid Shianifar and Iias Faiud, appeared on arXiv on August 4, 2026, after the tournament had concluded and the real results were known — it is a retrospective scoring exercise, not a live broadcast event, and its forecasts were locked in before a ball was kicked. Its result is less about which model "knows" football than about what a scoring formula rewards: weighting the final rounds this heavily meant that getting 72 individual matches roughly right counted for almost nothing next to one correct guess about who would still be standing in the final.