Nov 3, 2026
MIT CSAIL: GPT-6 Astra edited 0.016% of a model's weights and its math score jumped
MIT CSAIL found GPT-6 Astra can read and write raw neural network weights. Editing 0.016% of Qwen3.5-0.8B lifted MATH-500 from 7.9% to 48.1%, and it named a game (Breakout) from raw weights alone.
Sir Toastimir
Frontier Watch · Figures: MIT CSAIL
MIT CSAIL researchers published "Sparks of Weight-Space Programming" on November 2. Their finding: frontier AI models can read and write another neural network's raw weights directly, with no gradients and no training.
It made a small model better at math
Asked to make Qwen3.5-0.8B better at math without gradients, GPT-6 Astra found an issue in its weights causing reasoning loops, which Qwen's own model card warns about. It fixed it by editing 0.016% of the weights. MATH-500 went from 7.9% to 48.1%, and GSM8K from 13.8% to 67.1%.
It read raw weights and named the game
Astra got the raw weights of a 17,253-parameter game-playing policy network as plain text. No game was named and no options were given. It still said Breakout, 99% confident. It went 4 for 4 across Breakout, Pong, Pac-Man and Freeway, all identical 500→32→32→5 networks trained with PPO. Its tell for Breakout: first-layer weights lined up on the bottom row, where the paddle lives.
Game demo: Breakout
The researchers also report that Astra wrote small game-playing networks from scratch, with no environment feedback, that outplayed networks trained with PPO for 100 million steps. The demo is on the blog, linked below.
The limits
It couldn't improve math on the tiny Gemma 3 270M. This is a research blog post from one lab, not an independent replication.
Links
Get your daily slice of AI
One short email with the best new AI tools, robots and AI & tech stock news. Free. Unsubscribe anytime.
We only use your email to send the newsletter.