Code Evolution Cracks ARC-AGI-2
How Imbue set a new record with evolutionary algorithms. And why it matters.
Imagine being able to triple the performance of a low-cost AI model. Without changing the model itself. Simply through clever code design around it. The team at Imbue managed to do just that. Here is the story.
The ARC-AGI-2 benchmark measures whether AI systems truly understand what they are doing. Or whether they just memorize patterns. François Chollet developed this test in 2019. The concept: tasks that are easy for humans, but push AI to its limits.
Imbue has now introduced a method that relies on code evolution. The result: with the open-source model Kimi K2.5, they achieved a 34% success rate. That is a new record for freely available models. With Gemini 3.1 Pro, they even reached 95%.
Results at a Glance
Three runs. Three different base models. Always the same evolution framework.
Kimi K2.5: 2.8x Boost
From 12.1% to 34.0% (+21.9%). Cost: $2.67 per task. This is the best open-source score on ARC-AGI-2. It even outperforms GPT-5.2 in medium-reasoning mode.
Gemini 3 Flash: 1.8x Boost
From 34.0% to 61.4% (+27%). Cost: $2.42 per task. Performance comparable to much more expensive models like Opus 4.6 or GPT-5.2 on X-High.
Gemini 3.1 Pro: 95% Success Rate
From 88.1% to 95.1% (+7%). Cost: $8.71 per task. Significantly cheaper than Gemini 3 Deep Think ($13.62) and other refinement solutions (>$30).
State of the Art
Comparable to Confluence Lab (97.9% for $11.77/task). But: Imbue's solution is fully open source and reproducible.
What Is ARC-AGI Anyway?
A task consists of 2-5 example pairs: input grid, output grid. Your job: find the rule that transforms input into output. Then apply that rule to new test inputs.
Each grid is a colored matrix. Transformation rules vary widely. Sometimes it involves symmetry. Sometimes object movement. Sometimes pattern recognition. Often trivial for humans. For AI? A real challenge.
Early LLMs could not solve ARC-AGI. Zero tasks. It took years before double-digit success rates were reached at all. ARC-AGI-2 was released in 2025 and is significantly more difficult. At the start of 2026, the record stood at 54.2% (GPT-5.2 Pro). Now, just two and a half months later: 84.6% (Gemini 3 Deep Think).
How Code Evolution Works
The concept is biologically inspired. You start with a population of "organisms". Each organism is a Python code snippet trying to solve the task.
Then a loop runs:
1. Sampling
Select a parent organism based on its fitness score. Better solutions are chosen more frequently.
2. Mutation
An LLM generates mutations of the parent code. These are tested. Only mutations that produce new results are accepted.
3. Scoring
Each new organism receives a fitness score. Based on correctness, transfer ability, and code simplicity.
4. Loop
Back to step 1. Up to 16 iterations. The process stops once two good solutions have been found.
The Trick: Fitness Scoring with Three Components
90% correctness. How well does the output prediction match the known examples? Partial credit for every correctly predicted grid cell.
7% transfer score. A reasoning model evaluates how well the solution generalizes to challenge inputs. This catches hard-coded colors or other overfitting issues.
3% simplicity. Fewer hard-coded constants = higher score. Shorter code = higher score. Simpler solutions generalize better.
Prompting Techniques for Better Mutations
Natural Language First: The LLM describes the transformation in natural language first. Only then in code. This aligns better with human visual language.
Differential Formatting: If the input and output grids are the same size, they only display the changed cells. This helps the LLM focus on relevant areas.
Randomized Mutation Strength: Sometimes the prompt requests minor incremental changes. Sometimes radical reinterpretations. This prevents getting stuck in local maxima.
Crossover Mutations: 25% of the time, three parents are combined instead of just one. This allows discoveries from different branches of the population to be merged.
Why This Matters to You
Imbue did not develop the Darwinian Evolver for ARC-AGI. Rather, it was built for Vet (Verify Everything): an agent verifier that checks AI-generated code for quality.
The evolver optimizes the prompts and decision logic within Vet. The result: better code, more reliable agents. And you can download Vet today.
The idea behind it: code quality issues in AI-generated software are constantly changing. Imbue continuously generates new, annotated datasets featuring typical issues. The evolver distills these insights back into Vet's internal prompts. Automatically. Continuously.
The Darwinian Evolver is now open source. Problem-agnostic. You can use it for virtually any code and prompt optimization.
Conclusion: Evolution Beats Brute Force
Code evolution is not hype. It is a fundamental method for getting more out of affordable models. Without touching the model itself.