Researchers behind the CaP-X robot-coding benchmark, a group that includes UC Berkeley's Ken Goldberg and Stanford's Fei-Fei Li and Jiajun Wu, posted a preprint on November 12, 2026, showing that programs written by Anthropic's Claude Opus 5.5 succeeded in 221 of 240 physical robot trials, a 92.1% success rate, after the code was frozen with no AI or human present during execution. The study, labeled ACE-CaP-0, ran eight distinct manipulation tasks on a dual-arm robot.
What exactly did the researchers test?
The setup used two Agilex PIPERX robot arms and three Orbbec DaBai DC1 cameras. During a development phase, Claude Opus 5.5 acted as a coding agent: it wrote, ran, inspected and revised its own code based on physical feedback from the robot, with up to 10 human-assisted scene resets allowed per task. Once a program looked solid, the researchers froze it entirely, no more edits, and ran it through 30 fresh evaluation trials per task with neither a person nor the AI intervening. The frozen code had to react to live camera and sensor data on its own.
That's a meaningfully different test than most robot-AI demonstrations, which typically keep a neural network running continuously during execution. Here, the AI's job ended once development stopped.
Which tasks were hardest, and why?
Charger insertion came out perfect at 30 of 30 trials. Bread packing was the weakest performer at 25 of 30. In between, bottle uncapping, flower insertion and shoe packing each hit 27 of 30, cup nesting and parcel reorientation landed at 28 of 30, and chicken plating reached 29 of 30. The pattern suggests that tasks involving deformable or inconsistently shaped objects, bread, flowers, uncapped bottles, are harder for a fixed set of decision rules to handle reliably than tasks with rigid, predictable geometry like a charger and its port.
Why does freezing the code matter?
Most current approaches to robot manipulation rely on vision-language-action models that run inference continuously, frame by frame, during execution. That's computationally expensive and requires the model to be present and running every time the robot acts. The Code-as-Policies approach tested here flips that: a capable AI model is used once, intensively, during development, then steps aside. Execution afterward is just a program running, cheap and deterministic.
The researchers also found that giving the coding agent access to simulation during development mattered a lot. Trials developed with simulation hit 100% success and took an average of 2.6 hours to develop, versus 86.7% success and 3.7 hours without it. Simulation access, in other words, made Claude Opus 5.5 both faster and more reliable at writing working robot code.
What does this mean for the robotics industry's foundation-model push?
Companies like Figure, Physical Intelligence and Nvidia, with its GR00T platform, are pouring resources into end-to-end models meant to generalize across open-ended tasks without task-specific programming. The ACE-CaP-0 results suggest a complementary path for well-defined, repetitive jobs: let a large coding model write a deterministic program once, verify it in simulation, then deploy that program without needing a model running at inference time. That could cut runtime compute costs sharply for the narrow, repeatable tasks that make up much of warehouse and light-manufacturing work, even as general-purpose VLA models remain the better bet for tasks nobody has scripted before.
