- Levels
- 51
- Models
- 32
- Best
- 81.4%
- gpt-6-astra (high)
- Field avg
- 62.1%
A suite of 51 individual phone-use levels across seven categories: Primitives, Vision, Memory, IQ, Tempospatial, Games, and Tasks. Each level is inspired by a real failure we encountered while running agents in production.
Agents use the nunu.ai harness to take screenshots, analyze them with code, and interact through taps and swipes. Screen access is pixels only: no structured state, accessibility IDs, or text descriptions.
Primitives
touchscreen control and fine motor accuracy: taps, swipes, complex gestures.
Vision
reading the screen: counting, matching, visual search.
Memory
spotting information that matters and recalling it across a long task.
IQ
reasoning and rule induction, mostly puzzles with no instructions.
Tempospatial
temporal and spatial reasoning tasks.
Games
multi-step games that need a strategy, not just a correct answer.
Tasks
real workflows: driving phone UI, following multi-step instructions.
star lanes
Drag through the numbered stars and planets in order without lifting.
emoji soup
Find and tap three balls hidden among the emoji.
push-box gauntlet
Push every box into a hole without getting stuck.
road rage
Switch lanes to dodge traffic until time runs out.
match three
Match tiles to score points before time runs out.
tiny factory
Build a production line with miners, belts and smelters to fill orders.
- Within a run
- The agent’s best attempt receives a score from 0 to 100%. Each level defines its own criteria: accuracy, game performance, or workflow completion with penalties for mistakes.
- Across runs
- A model’s level score is the average of its published scores across included runs. Repeated runs contribute equally.
- On the leaderboard
- The overall score is the average across scored levels. Category scores average the levels in that category. Each scored level has equal weight, so larger categories contribute more to the overall score. Unscored levels are excluded.
For games, 50% is the human baseline. It represents a human playing the same game for the same duration. A score of 50% does not mean half the games were won.