Princeton’s own paper rates the chess engine network inside its new Queen model at 2987 Elo, on chess’s standard rating scale, and Queen itself at 2697. With the engine removed, the language model left behind rates 514, and it lost all 32 of its games.
Queen is a 4-billion-parameter model released on October 2, 2026 by Adithya Bhaskar, Jeffrey Cheng and Danqi Chen of Princeton Language and Intelligence. Inside it, a board-reading network taken from Leela Chess Zero, a long-running open-source chess engine, looks at the position. A small off-the-shelf language model reads what that network sees, then writes out a move and the reasons for it. What Princeton built is a language model that learned to put an engine’s judgment into words. That is a real result. It is also narrower than the abstract’s “4B-parameter chess-language model” playing “at the level of a typical Grandmaster.”
The paper names the parts. The engine side is a 240M-parameter Leela “BT5” encoder with 15 layers. The language side is SmolLM3-3B, a small open model with 36 layers and hidden size 2048. A roughly 470M-parameter “bridge” of cross-attention layers joins the two and lets the language model look up the engine’s internal picture of the board as it writes. According to AlphaSignal’s write-up, embeddings from layer k of the Leela network feed the language model just before its layer 2k.
Princeton’s Table 1 ranks the systems like this, with the frontier models all run at high reasoning effort:
| System | Elo |
|---|---|
| Leela BT5 encoder alone | 2987 |
| Queen (Leela + SmolLM3-3B) | 2697 |
| Gemini-3.1-Pro / GPT-5.6-Sol | 2201 / 2071 |
| GPT-5.6-Luna | 1822 |
| SmolLM3 side without the encoder | 514 |
Taking the engine out costs Queen 2,183 rating points: 2697 with it, 514 without.
The abstract says Queen ends up “substantially surpassing all frontier models on both playing strength and puzzle accuracy, despite containing three orders of magnitude fewer parameters.” That pits a chatbot with a chess engine inside it against chatbots with no engine at all. On the same table, the bare engine network beats Queen by 290 points.
Taking the engine out costs Queen 2,183 rating points: 2697 with it, 514 without.
The language side still does real work. Each training round works the same way, per the abstract. The model analyses the positions after its top candidate moves, consolidates those analyses into an explanation of the current position, and is then retrained on that explanation. Over seven rounds Queen climbed from 1782 to 2697, going 1782, 2024, 2187, 2346, 2539, 2434, 2559, 2697. The paper gives no explanation for the dip in round five.
A second engine was the teacher. Queen’s training data uses Stockfish analysis at 100,000 to 1 million nodes, meaning Stockfish examined that many positions for each one it judged. Queen itself runs no search when it plays. During the first stage only the cross-attention bridge was updated while the Leela encoder stayed frozen, and the paper does not report the encoder changing afterwards. That means much of the 915-point climb came from training the decoder and the bridge to read the engine and say what it sees. The engine sets the ceiling, and getting near it took effort. The authors have a name for the distance that remains:
“The gap between Queen and Leela reflects the challenge of expressing latent expert knowledge in language, which we view as a form of verbalization debt.”
AlphaSignal’s headline described it more accurately than the abstract did: “Queen gives a grandmaster-level chess engine a voice.” The voice is Princeton’s work. The grandmaster is Leela.
How the 2697 was measured
The rating comes from 32 games, four each against eight engine opponents. Those were Stockfish limited to 10, 100, 1,000 and 3,000 nodes per move, Maia-2200, TinyGyal-8, and two Lc0 networks at 10,000 nodes. The paper reports no confidence intervals, and 32 games is a small sample for a rating stated to the point.
None of those games was against a person. Two of the eight opponents were Leela networks. The results were then anchored to the Lichess scale using the known strength of Leela networks, the same engine family that sits inside Queen. Queen’s 2697 was measured against engines on a scale calibrated by the engine it contains. The paper does not say how the Leela encoder’s own 2987 was measured, so that figure is the paper’s own and nothing more.
An AI model graded the explanations, and robotics was never tested
Explaining moves is the reason Queen exists. The authors checked its explanations in two ways.
First, Stockfish at 1 million nodes checked the move lines Queen proposes, counting a mistake as any move that cut expected win rate by 10% or more. Queen does well here. On 1,000 tactical puzzles, its first move avoided a mistake 91.6% of the time, against 81.9% for GPT-5.6-Sol and 82.5% for Gemini-3.1-Pro. Across a whole line the gap narrows to 68.8%, 65.1% and 66.9%.
Second, one judge graded the prose: GPT-5.6-Sol at high effort, on a 1-to-5 scale. Sol was also a contestant, and it gave its own answers the top score on conceptual coherence and on fluency. No human chess players were asked, titled or otherwise.
| Score (1-5) | Queen | GPT-5.6-Sol | Gemini-3.1-Pro |
|---|---|---|---|
| Structural coherence | 3.51 | 3.51 | 3.55 |
| Conceptual coherence | 2.76 | 3.61 | 3.30 |
| Fluency | 4.45 | 4.99 | 4.98 |
The abstract says the explanations “approach GPT-5.6-Sol (high) in coherence.” That holds for structure. Conceptual coherence asks whether the chess ideas in an explanation hang together, and there Queen scored lowest in the table: 2.76, from a judge grading its own competition. The paper’s limitations section offers a likely reason: “hallucinations of motifs and patterns can potentially propagate through the consolidation and keep conceptual coherence low.” Each round retrains Queen on its own consolidated explanations, so an invented idea in one round becomes training material for the next. Queen finds good moves and sometimes explains them with chess ideas that are not on the board.
The claim travelling furthest from the paper is in the abstract’s final sentence, which says the method “suggests a recipe for applying language models to domains where silent expert encoders are available, like games, robotics, and computer use.” The paper runs no robotics or computer-use experiments. For those fields, the recipe is that one sentence.
The code and checkpoints are public, on Hugging Face and GitHub, so anyone can now run the check the paper skipped: titled players reading Queen’s explanations against the board. Two numbers decide whether the recipe is worth anything outside chess: the 290-point verbalization debt and the 2.76 conceptual score. No human has tested either.
Further Reading
- Language Models that Play Chess and Explain Their Moves, The Princeton paper’s arXiv abstract page, with the headline claims and the robotics and computer-use suggestion.
- Language Models that Play Chess and Explain Their Moves (full HTML), Full paper: Table 1 Elo ratings, the 514-Elo ablation, the evaluation setup, judge scores and limitations.
- Princeton’s Queen Teaches a Grandmaster Chess Engine to Explain Its Moves, AlphaSignal’s architecture walkthrough of the Leela-to-SmolLM3 bridge and its training curriculum.
- Princeton Queen 4B chess model 2697 Elo, Pasquale Pillitteri on the release date, the authors’ affiliation, model specs and the frontier comparison.