OpenAI’s GPT-6 Astra edited 0.016% of the weights in Qwen3.5-0.8B, a small open model, and stopped it reasoning in circles. Before the edit, the model repeated itself until it hit a 32,768-token cutoff on 93% of the problems in MATH-500, a 500-question competition math test. After it, 33.1% did, and the score rose from 7.9% to 48.1%.
Run with its long-form “thinking” mode switched off, which is how Qwen ships it by default, the untouched model already scores 44-52% at the same token budget, an independent re-run found. The same post also tried Gemma 3 270M, a Google model that rarely loops. There, Astra’s edits moved the math score from 6.8% to at most 7.7%, a gain Astra itself reported as “not established.”
Four researchers at MIT CSAIL, MIT’s computer science and AI lab, published the Qwen result on 2 November. Headlines turned it into “6× Better at Math” and “sixfold”, and VentureBeat called it “like gene editing for neural networks.”
Anyone who took that at its word and downloaded the edit got a slightly worse model. Jonathan Coletti’s independent replication ran the edited model the way Qwen ships it, with thinking off. It scored 2.2 points lower on MATH-500 (95% confidence interval −4.3 to −0.1). It was also 1.4 points lower on GSM8K, a set of grade-school word problems, and 1.7 points lower on IFEval, an instruction-following test.
The authors’ own table ties the score to the loops. Astra was allowed 10, 25 or 50 forward passes, meaning batches of problems it could actually run the model on. With 10 and 25 passes, 87.2% and 86.4% of answers still hit the limit, and the scores were 12.3% and 14.1%. In the authors’ own practice runs, 85 of the first 96 answers ran to the limit. Only at 50 passes did the loops break, and only then did the score jump. The authors say as much in the body of the post:
“Fixing loops, however, is not the same as making a model better at math: it lets the model finish answers it could already work out.”
The post’s lead does not say this. It goes straight to the 7.9%-to-48.1% figure that every headline reached for.
The edit worked like this. Astra found a single pattern that two internal layers, 15 and 19, write into the model’s working state, and that pattern tipped the model into repeating itself. Astra then subtracted that pattern from those layers’ weights so they could no longer write it. The change touched 131,072 weights. R&D World reports that the run took 80 Python executions over 20.3 hours.
The engineer Sean Goedecke calls it “basically the Heretic abliteration playbook”. That is the same delete-one-pattern trick that Heretic, the open-source guardrail-removal tool, uses to strip out refusals. In his words, it is “exactly what a competent human researcher could have and would have done.”
Nathan Barry recast the edit as a linear program, a standard optimisation method. It matched Astra within noise, 51.7 against 50.9 on MATH-500, while changing about 16,290 weights instead of roughly 130,000. His best result did reuse the layers and the pattern Astra had picked. With a pattern Barry chose himself, it scored 47.8. So Astra’s real contribution was finding the fault, and the fix itself was routine.
Matthew Honnibal, co-founder of the NLP company Explosion, notes that this run had Python and 50 batches of forward passes, so “it’s not a single technique.”
Under Qwen’s own settings, the unedited model already scores 44-52%
On 3 November a user named carlostoledo1891 opened issue #4 on the authors’ own logs repository. He first re-ran their setup and got their result: 7.2%, with 93.1% of answers hitting the limit. He then ran the untouched model with Qwen’s recommended non-thinking settings at the same 32,768-token cap. It scored 44.0-51.7%, depending on how answers are counted. Under the authors’ settings, about 1,860 of his 2,000 answers ran out of room; under Qwen’s, 6 did.
“under the model card’s own recommended settings the unedited model lands where the edited one lands under yours.”
At an 8,000-token cap, the non-thinking run drops to 30.0%, and the opener agreed the number “depends on the token limit as much as on the model.” At the authors’ own budget, though, Qwen run its makers’ way needs no edit to land near 48%.
| Model and setting | MATH-500 score | Answers that hit the token limit |
|---|---|---|
| Qwen3.5-0.8B, unedited, thinking on (the authors’ setup) | 7.9% | 93.0% |
| Qwen3.5-0.8B, edited by Astra, thinking on | 48.1% | 33.1% |
| Qwen3.5-0.8B, unedited, run as Qwen recommends (issue #4) | 44.0% | 0.3% |
| Gemma 3 270M, unedited and best edit | 6.8% and 7.7% | 8.5% (unedited) |
The authors’ defence, in their FAQ, is that they tested “a reasonable setting where the model loops,” and that “turning thinking off raises the base score, but it does not fix the weights.” Set that beside the model card. It warns that in thinking mode the 0.8B model “is more prone to entering thinking loops” than its siblings, and it leaves thinking off by default. The card also recommends a temperature of 1.0 for thinking mode, keeping 0.6 for coding and image tasks, and an 81,920-token cap for competition math. The authors ran thinking mode at 0.6, capped at 32,768. The authors switched on the mode Qwen warns against, then measured their fix against it.
The other benchmarks follow the same pattern. The post reports GSM8K rising from 13.8% to 67.1% and IFEval from 21.0% to 62.3%. In Coletti’s runs, the unedited model with thinking off already scores 57.1 and 52.9. Qwen’s own model card puts the unedited model at 44.0% on IFEval with thinking on, more than double the authors’ 21.0% starting point.
The figures themselves hold up. Ahsen Khaliq at Hugging Face rebuilt the edited model and got 51.0%, with 31.6% of answers hitting the limit. The loop is also a real flaw. Coletti used the presence penalty Qwen recommends against repetition, and the base model still looped on 93% of answers. Against the base model’s best setting, the edited model with thinking on gains 4.3 points on MATH-500 (interval 1.9 to 6.8), 10.5 on GSM8K and 9.2 on IFEval. That margin is probably generous. Coletti capped his thinking-off runs at 8,000 tokens, and 32% of those answers were cut off.
So the edit is worth something: about four points on math, against the model run properly. Coletti’s own summary: “This removes a failure mode; it doesn’t add capability.” For scale, Qwen’s previous-generation 0.6B model scored 77.6 on MATH-500 with thinking on, and 55.2 with it off, at the same cap. The 7.9% starting point was a broken model, not a typical one.
Where there was no loop to fix, and the audit that failed
On Gemma 3 270M, only 8.5% of answers reached the token limit. R&D World and the logs record the scores after editing: 7.0%, 7.6%, 7.7% and 7.0% across budgets of 5 to 50 forward passes. Astra found ordinary arithmetic and algebra errors, with nothing to switch off. The authors conclude that editing “could not add an ability the model lacked.” VentureBeat quoted the X user @1a3orn flagging that this result was missing from the headline claim.
The second test was a test of reading weights, not fixing them. The authors fine-tuned two copies of Gemma, one to answer correctly and one to agree with whatever the user suggested, and asked Astra which was the sycophant. From the weights alone, it failed all four attempts. It succeeded three times out of three only once it could watch the models’ attention heads, the parts that decide which earlier words to look at, while they ran on text Astra wrote.
The post’s closing section still lists “auditing without evaluation” and “recursive self-improvement” among what this work opens up. Several outlets, VentureBeat among them, said in their body text that the gain came from fixing loops. Kingy AI later updated its sixfold piece to note the far higher untouched baseline. It was the only one to correct, which shows the baseline was there for anyone to find.
Astra is OpenAI’s gpt-6-astra, run through Codex CLI, OpenAI’s command-line coding tool. The model has been public in OpenAI’s API since September, so anyone can re-run the experiments.
On 12 November the authors added a “Response to Comments” section to the post. It says the work “is not a method,” counts more than a hundred replications and follow-on projects, and repeats the thinking-off defence. It does not address the 44% re-run, the linear program or the abliteration comparison. As of 10 November, issue #4 had replies from three outside users and none from the authors. The post still opens on 7.9% to 48.1%. And the other use it promises, auditing a model from its weights alone, stands at zero for four.
Further Reading
- Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights, the MIT CSAIL post, with the Qwen table, the Gemma result, the sycophancy test, the FAQ and the 12 November response.
- Independent re-run of the MATH-500 baseline under Qwen’s recommended settings, issue #4 on the authors’ logs repository: the 44-52% baseline and the follow-up tests of token budget and sampling.
- JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit, an independent replication with thinking on and off, with confidence intervals.
- MIT team says GPT-6 Astra can fix an AI model by editing its weights; critics say the method was standard ML engineering, R&D World’s account of every experiment’s figures and the critics.
- Can a Linear Program Do Weight-Space Programming?, Nathan Barry matches Astra’s edit with a standard optimisation method and far fewer weights.