Simon Willison reported on November 3, 2026, that an open-weight language model running on his laptop could identify several games from the weights of small neural networks. In his experiment, Qwen 3.8 27B identified Breakout and Pong, produced a Pac-Man-style description, and failed to identify Freeway. It is an interesting independent test of a narrow task, rather than proof that a model can understand or improve any network simply by looking at numbers.
Willison's exploration followed Sparks of Weight-Space Programming, a post published on November 2 by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab. The authors describe both identifying and modifying weights using GPT-6 Astra. These are different experiments, however: Willison tested identification without computational tools, while the reported improvement to a language model's mathematical performance used Python and feedback from model runs.
What is being read inside a neural network?
Weights are numerical values that determine how signals pass through and combine within a neural network. Conventional training adjusts those values using examples and a performance objective. In this work, researchers asked whether a language model could infer something about another network from its numbers and structure, and sometimes select targeted changes. That is different from explaining code describing the training process.
The game-identification task did not provide numbers without context. According to the researchers' published prompts, the model received a description of a 10 by 10 board with an 8 by 8 playable interior, together with explanations of the input channels. These represented the agent, moving objects, reward targets, obstacles and previous-frame information. The available actions and network architecture were also explained in advance.
Each of the four networks had 17,253 parameters and the same architecture. Weights were presented as numbers rounded to three decimal places. Willison wrote that he checked the prompts were identical apart from the weights. That reduces the possibility of identifying the game from an obvious difference in instructions. It remains a small network with explanations of its inputs, rather than a large unfamiliar model supplied without accompanying information.
What worked on the laptop, and what did not
Willison used Qwen 3.8 27B through LM Studio, without additional training for the task. He said he needed to expand the context window because the default was insufficient. The prompt prohibited code execution and calculators and requested a single response based on the weights. Using software to run a language model does not itself mean that model was given tools to test the network it was being asked about.
In the outputs Willison shared, the model identified Breakout with stated confidence of 88%, Pong with 75%, and a Pac-Man-style game with 72%. Those are the model's assertions of confidence in its answers. They are not success rates measured across a large experiment and do not mean that 88 out of 100 networks would be identified correctly. That distinction matters when the percentages resemble test results.
For Freeway, the model failed and answered that it was a Pac-Man-like game. Willison separately described a run with Kimi K3.1 Pro through OpenRouter that identified Breakout after 9 minutes and 41 seconds. The timing belongs to that particular run, not a general performance specification. Both the successes and failure are needed to understand the demonstration: several persuasive identifications alongside a visible limitation, without a sound estimate of broader accuracy.
The mathematical improvement came from another experiment
The striking number in the original post is a rise from 7.9% to 48.1% on MATH-500 after changes to Qwen3.5-0.8B's weights. That is a result reported by the authors, not measured by Willison. The model names differ too: Qwen3.5-0.8B was the editing target, while Qwen 3.8 27B was one of the models Willison used to identify game networks.
According to the original experiment description, Astra received a Python environment containing the weights, practice problems from the GSM8K and MATH training sets, and a budget of forward passes. The leading result used a budget of 50 passes, each of which could include multiple problems. The researchers prohibited gradient-based training but allowed code and observations of the model's behavior. No conventional training therefore does not mean no tools or no feedback.
The authors describe changes to 131,072 weights, about 0.016% of the model, across two layers. Their explanation is that the edit reduced a tendency to become trapped in repetitive thinking loops. In their table, the share of responses reaching the token limit fell from 93.0% to 33.1%. This gives the mathematical result context: the fix may let the model finish answers that were previously cut off, rather than demonstrate the creation of new ability from nothing.
Why the starting conditions change the interpretation
The experiment enabled thinking mode and allowed up to 32,768 output tokens. Qwen's model card warns that this small version is prone to loops when thinking is enabled, and says its default is non-thinking mode. The 7.9% score should therefore not be treated as a general measure of mathematical ability across reasonable configurations. It is a score under particular conditions, and the comparison needs to stay attached to them.
The authors also report a boundary themselves. In another experiment with Gemma 3 270M, which their work describes as rarely looping, they did not establish a comparable improvement. Its baseline score was 6.8%, and a budget of 50 passes produced 7.0%. They interpret this as a distinction between removing an obstacle that conceals existing ability and adding an ability the model lacks. That, too, is their interpretation of specific results, not a rule proved for every model.
Publishing experiment logs and modification files allows others to inspect the process, but does not turn every outcome into a complete independent reproduction. Broader conclusions would require comparisons under additional generation settings, other networks and repeated runs. For game identification, it also matters what happens when some architectural information is changed or removed. The current results alone do not provide all those comparisons.
What remains when the experiments are separated
Willison wrote that parts of the models' reasoning were difficult to follow and cautioned that some discussion of the work went further than justified. A long reasoning trace is not an independent test of how a correct answer was reached. It can suggest avenues for investigation, but does not by itself prove that the model accurately computed the network's behavior or understood every weight.
The useful distinction is between three actions: identifying what a small network was trained for, editing a network using tools, and evaluating the changed network. Each needs different evidence. Willison's experiment adds limited support for the first; the authors' results describe the other two in a particular setting. Combining everything into a claim about reading and writing weights without tools removes the work's central methodological distinction.
The news remains interesting at its demonstrated scale. A locally running open model produced several useful identifications from numbers and an explained structure, while researchers reported a targeted fix that reduced a failure in another model. The Freeway failure and the starting-condition caveat are not footnotes. They are part of the finding and determine what it can teach us now and what still needs another experiment.