Independent benchmarking startup Robocurve has published the first head-to-head comparison of OpenAI's GPT-6 Astra and Anthropic's Claude Fable 5.1 driving real robot arms. Running both models under the same Inspect Robots agent policy on bimanual I2RT YAM arms, Astra placed a red block into a bowl 19 out of 20 times against Fable 5.1's 8 out of 20 – and did it in 2.5 minutes per trial at an estimated $0.94, versus 6.8 minutes and $2.12 for Fable.
The puzzle piece is still a wall
On the harder task – inserting a round blue puzzle piece into a matching groove – both models tied at 2 completions in 20. Astra was 1.6× cheaper per attempt, but the failure mode was identical: the model reached the groove and stalled at the final alignment step, exactly where Fable had stalled two days earlier. That suggests the ceiling here is one of perception and fine motor control, not language-model reasoning, and no amount of extra thinking tokens closes the gap.
80% fewer output tokens
The most striking result is efficiency. Astra used about 2,100 output tokens per bowl trial against Fable 5.1's 12,900 – roughly 80% less. On the puzzle it burned 2,700 tokens against 10,500. Robocurve credits OpenAI's tighter action planning and the automatic input caching Astra enjoys, which the benchmark deliberately did not discount. Even at OpenAI's published $10/$50 per million input/output tokens, the model runs the physical task at less than half the marginal cost of its nearest frontier peer.
How the benchmark works
Trials were run with the open-source Inspect Robots harness under MIT license. Each model got 20 blinded, hand-reset trials per task with three camera views, absolute end-effector control, a 20-call LLM budget and medium reasoning effort. A human grader scored each trial on a 0-4 rubric measuring how far the arm progressed – contact, lift, positioning, placement – so failed trials still capture partial credit. Astra's bowl trials ran on a different rig (rig-1) than Fable's (rig-3), a caveat Robocurve flags in its limitations section.
Why it matters
OpenAI's GPT-6 Astra rollout put computer-use and long-context front-and-centre, but the model's real-world robot control numbers were unproven outside OpenAI's own demos. Robocurve's independent run confirms Astra is a genuine step forward on the easy end of manipulation while making a public counter-claim: on tasks that need sub-centimetre coordination, frontier language models are still bottlenecked by the same perceptual limits Lyte and Skild AI have raised money to solve. The benchmark also validates the case for third-party, reproducible robot evaluations at a time when almost every humanoid vendor's marketing videos are cherry-picked.
Reporting based on Robocurve's published GPT-6 Astra report and the Inspect Robots repository on GitHub.