Botzone's Mahjong Contest Splits Algorithm Bots From LLM Agents

The sixth International Mahjong AI Competition, organized on the Botzone platform by Peking University's AI Lab together with Weizhiyu Technology, closed both of its divisions in early July 2026, and the results were shown at IJCAI 2026 in Bremen (August 19-21). The Algorithm and LLM tracks played out separately, with separate winners: the team 海中队 topped the sixteen-team Algorithm track with 1315.50 points across duplicate-format matches, while the team Kitagawa won the four-team LLM final with 417.0 points — a gap that reflects two different formats, not a gap in skill.

The game itself is Chinese Official Mahjong (MCR rules), a four-player, imperfect-information contest in which players build a scoring hand from tiles drawn and discarded around the table while reading what the other three are collecting and avoiding the discard that completes an opponent's hand. Unlike chess or Go, a mahjong agent has to reason about hidden information and about three opponents at once, and a single hand can turn on one unlucky or well-read discard — which is why a raw win count across a handful of games says very little about relative skill.

To manage that variance, Botzone runs both divisions in duplicate format: the same shuffled deal is dealt out at multiple tables at once, so an agent's score reflects how it played a given hand rather than which hand it happened to draw. In the Algorithm track, sixteen qualifiers moved through a Swiss-style schedule to a four-team final, where 海中队 finished first with 1315.50 points, ahead of 摸鱼 (1306.50), We Can Play (1291.50) and player team 2 (1206.49) — a spread of under 110 points across the whole podium, tight enough that no single engine dominated.

The LLM track ran on a tighter calendar — registration closed June 9, its own qualifying round closed June 28, and its four-team final was played July 12 — and produced a wider spread: Kitagawa's winning 417.0 points was more than double fourth-placed MahjongTeam's 186.0, with agentic_bot (345.0) and a team entered as "LLM可以算清番吗" (332.0) in between. The numbers are not directly comparable to the Algorithm track's, since fewer duplicate hands were played in the LLM final, but the wider gap between first and last is consistent with a field that has had far less time to converge on a shared standard of play than classical search-based mahjong engines have.

Splitting the divisions is the more interesting decision here than either leaderboard. A hand-coded or reinforcement-learned mahjong engine is optimized end to end for the game: it never has to be prompted, spends no computation producing a natural-language explanation of itself, and can be tuned purely against duplicate-format scoring. An LLM agent is doing something different — turning table state into a natural-language read of hidden information and risk before committing to a discard — and putting both on one leaderboard would mostly measure which approach has had more engineering time invested in it, not which plays the better hand.

The competition's technical committee includes Wenxin Li and Xiaotie Deng of Peking University, Junliang Xing of Tsinghua University, Weinan Zhang of Shanghai Jiao Tong University, and researchers from Google DeepMind and Tencent AI Lab — placing an imperfect-information card game alongside chess and Go as a standard testbed, and placing language models inside that testbed as a category to be measured rather than assumed superior.

The results were already two months old by the time they reached the exhibition floor in Bremen, and Botzone has not said whether the seventh edition will keep the two tracks separate or attempt a merged ranking. For now, the split stands as a concrete answer to a question that keeps coming up in machine-vs-machine competition: once a language model can play a game at all, comparing it to a purpose-built specialist may call for a different contest, not just a different score.