The ARC Prize organization designs benchmarks which are specifically crafted to demonstrate tasks that humans complete easily, but are difficult for AIs like LLMs, “Reasoning” models, and Agentic frameworks.
ARC-AGI-3 is the first fully interactive benchmark in the ARC-AGI series. ARC-AGI-3 represents hundreds of original turn-based environments, each handcrafted by a team of human game designers. There are no instructions, no rules, and no stated goals. To succeed, an AI agent must explore each environment on its own, figure out how it works, discover what winning looks like, and carry what it learns forward across increasingly difficult levels.
Previous ARC-AGI benchmarks predicted and tracked major AI breakthroughs, from reasoning models to coding agents. ARC-AGI-3 points to what’s next: the gap between AI that can follow instructions and AI that can genuinely explore, learn, and adapt in unfamiliar situations.
You can try the tasks yourself here: https://arcprize.org/arc-agi/3
Here is the current leaderboard for ARC-AGI 3, using state of the art models
- OpenAI GPT-5.4 High - 0.3% success rate at $5.2K
- Google Gemini 3.1 Pro - 0.2% success rate at $2.2K
- Anthropic Opus 4.6 Max - 0.2% success rate at $8.9K
- xAI Grok 4.20 Reasoning - 0.0% success rate $3.8K.
(Logarithmic cost on the horizontal axis. Note that the vertical scale goes from 0% to 3% in this graph. If human scores were included, they would be at 100%, at the cost of approximately $250.)
https://arcprize.org/leaderboard
Technical report: https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf
In order for an environment to be included in ARC-AGI-3, it needs to pass the minimum “easy for humans” threshold. Each environment was attempted by 10 people. Only environments that could be fully solved by at least two human participants (independently) were considered for inclusion in the public, semi-private and fully-private sets. Many environments were solved by six or more people. As a reminder, an environment is considered solved only if the test taker was able to complete all levels, upon seeing the environment for the very first time. As such, all ARC-AGI-3 environments are verified to be 100% solvable by humans with no prior task-specific training
Biased study. Take any average person off the streets and shove this thing in their face. That 100% notion will go down fast.
They didn’t say “100% of humans can solve this benchmark”, they said “humans can solve 100% of this benchmark”.
I couldn’t get past the second level :(
Guy, I found the bot!
I see by your lack of pluralization that you’ve realized there’s only one person here and everyone else is bots. However through inference and deduction, you are therefore also a bot. I have good reason to believe I am the non-bot though I wonder if I could know for certain…
That was a lot of effort for a typo joke…
Cogito, ergo sum
My programming tells me I’m not a bot.
feelsbadman. You need more RAM!
Of the first task? Yikes.
I finished one of the tasks. And, I imagine I could finish at least some of the others. But, I wasn’t being paid, and it wasn’t very entertaining, so I stopped.
EDIT: Got all the way through several other tasks that suddenly looked interesting while watching the replays available of the frontier models. If you paid me $250, I’d finish them all, though probably not optimally. I still don’t know how I’d do against the private sets.
They should ad a “global” and “friends-only” leaderboard (like the Zachtronics games, etc.) and really see the competition (at least human competition) heat up.
“Humans score 100%. Frontier AI scores 0.26%.”
The title deals in absolutes.
Those are the high scores.
🤔 So this is a visual comparison between peak performance of some humans and peak performance of current LLMs in a controlled environment?
Is this a gotcha? Not sure where you got the “visual” from, but yes it is best human performance vs best LLM performance
I don’t know why you assume there has to be a gotcha, maybe it’s the competitive background… Anyway, it’s visual because you look at it to see it. And it’s not the best human performance vs best LLM performance, it’s best controlled performance because the testing is limited to a set of parameters.
That’s what games are? I really don’t see how it is an unfair comparison to you. How would you change it?
Stress test it. Low, average, high, impairment conditions, safeguards off, order, chaos and everything in between.
Pretty defensive there. It’s not even a study
If it studies something, it’s a study. If you feel defensiveness, you consider aggression. If you feel bias in one way, someone can feel bias in another way. If there’s an action, there’s a reaction.
I’m studying these comments, now I am a study
Sort of like how when people outsource all their critical thinking to AI, their ability for critical thinking atrophies?
Aggression as in calling something biased without providing evidence?
As in assuming you are starting with an unbiased point of view.
Of course we all have our biases. But what to do with that lesson? It can be a convenient response whenever someone disagrees with us. But it can also serve as a powerful motivation to find some common ground against all odds. The universe is chaotic. Language is illogical. Yet sometimes we find stuff we can agree on. Isn’t that beautiful?