Two different bets about how a robot should learn
Classical robotics builds a task by hand: perception modules, a planner, an inverse-kinematics solver, a state machine, and thousands of lines of control code tuned to one robot, one gripper, one narrow set of objects. It offers something valuable — formal guarantees about what the system will and won’t do — but it does not transfer. A new task, or a new robot arm, usually means rewriting much of the stack.
The bet reshaping the field since 2023 is different: the same recipe that produced large language models — huge datasets, a transformer, and scale — should also work for robot control, discovering a general “manifold” for physical skill the way it discovered one for text and images. That is a hypothesis, not a settled fact, and the evidence so far is genuinely mixed. This page lays out what has actually been demonstrated, by whom, and what the people building these systems say are still open problems.
The first evidence: action tokens and cross-robot transfer
Google DeepMind’s RT-2, published in July 2023, was an early, concrete test of the idea. Instead of building a separate action-prediction module, DeepMind trained robot-action tokens directly into a vision-language model’s own output vocabulary, so the same network that describes an image in words can also emit a motor command (arXiv:2307.15818). Across 6,000 evaluation trials, it generalized to unfamiliar objects and settings roughly three times better than prior task-specific baselines — evidence that web-scale visual and language knowledge really does carry some information useful for manipulation.
Three months later, a 34-institution collaboration went further with RT-X and the Open X-Embodiment dataset, pooling data from dozens of different robot types and training one 55-billion-parameter model across all of them (arXiv:2310.08864). It showed materially higher emergent-skill success than a 5-billion-parameter version of the same architecture — an early signal that scale specifically drives transfer across different robot bodies, not just within one.
A crowded field of foundation models for bodies
That result set off a wave of company-built “robot foundation models,” each with a different architecture and each making claims that deserve to be read as company claims rather than independently verified fact.
Physical Intelligence’s π0 (October 2024) combined the PaliGemma vision-language model with a flow-matching action head, trained across eight robot embodiments, and demonstrated tasks like folding laundry, bussing a table, and bagging groceries. Unusually for this field, Physical Intelligence open-sourced the weights and code as “openpi,” making the result independently reproducible rather than taken on faith. Even so, the company’s own assessment was blunt: “generalist robot policies are still in their infancy, and we have a long way to go” (Physical Intelligence blog, Oct. 31, 2024). A newer version, π0.7, claims zero-shot generalization across robot embodiments (arXiv:2604.15483), but as of this writing that claim rests on a preprint that has not been independently replicated.
NVIDIA released GR00T N1 in March 2025 as the first open humanoid foundation model, then GR00T N1.7 in April 2026: a 3-billion-parameter model trained on 20,854 hours of human egocentric video and validated on three different robot platforms, with open weights on Hugging Face. NVIDIA reported that scaling that egocentric training data roughly twentyfold “more than doubles average task completion” on dexterous tasks — a claim worth flagging as NVIDIA’s own, since no independent benchmark accompanies it (Hugging Face blog, Apr. 17, 2026). A successor, GR00T N2, was previewed at GTC in March 2026 as a “world action model” that predicts future states before acting; NVIDIA’s keynote claim that it succeeds at novel tasks “more than twice as often” as leading baselines remains, as of September 2026, an unreleased product claim rather than a tested one.
Google DeepMind’s successor to RT-2, Gemini Robotics 2 (also called ER 2), moved from upper-body demonstrations to whole-body humanoid control on Apptronik’s Apollo 2, including command of a 22-degree-of-freedom five-fingered hand for tasks like tying knots and sealing bags. DeepMind’s own disclosed numbers are the most candid in the field: 45.7–76.3% success on whole-body manipulation and a highly variable 32–92% on multi-finger dexterity, with DeepMind stating plainly that “multi-finger dexterous manipulation remains challenging” (DeepMind blog, Jul. 30, 2026). Bloomberg’s own headline on the release put it more bluntly still: Google’s robots are “struggling with dexterity” (Bloomberg).
The clearest before-and-after: Figure’s Helix 02
The single clearest illustration of what changes under this approach comes from Figure. Its original Helix system (February 2025) used a two-part “System 1 / System 2” architecture: a 7-billion-parameter vision-language model reasoning at 7–9 Hz, feeding an 80-million-parameter transformer running fine motor control at 200 Hz, trained on only around 500 hours of data — Figure says under 5% the size of prior VLA datasets (Figure blog, Feb. 20, 2025).
Helix 02, released in January 2026, added a third layer Figure calls “System 0”: a 10-million-parameter, 1kHz whole-body balance controller trained entirely in simulation across more than 200,000 environments plus more than 1,000 hours of retargeted human motion. Figure’s own description of what that replaced is the sharpest old-versus-new comparison available anywhere in this field: System 0 “replaces 109,504 lines of hand-engineered C++ with a single neural prior for stable, natural motion” (Figure blog, Jan. 27, 2026). That is the classical-robotics paradigm — enumerated, hand-tuned control logic — being replaced wholesale by a learned model, in one company’s own accounting. Figure demonstrated a four-minute, 61-action dishwasher-loading task with no human reset, while cautioning in the same release that “the results are early.”
Boston Dynamics and the Toyota Research Institute report a related but more measured finding from “Large Behavior Model” work on the Atlas humanoid: capabilities that once required hand-programming can now be added without new code, since one model has direct control of the entire robot (Boston Dynamics blog, Aug. 2025). A follow-up 2026 study found that after task-specific fine-tuning, a pretrained behavior model needed roughly 3–5x less task-specific data and was more robust to changing conditions than training from scratch (TRI) — a real, bounded efficiency gain, not evidence of open-ended generality.
Tesla’s Optimus program is the field’s clearest outlier on transparency. Elon Musk and Ashok Elluswamy, who took over Optimus in June 2025, describe the same “photons in, controls out” end-to-end architecture Tesla uses for its driver-assistance software. Unlike DeepMind, NVIDIA, Physical Intelligence, or Figure, Tesla has not published a technical paper on this architecture — only conference remarks and earnings-call commentary (Not a Tesla App, summarizing Tesla’s Q2 2026 earnings call, Aug. 2026).
Narrow records aren’t general competence
The World Humanoid Robot Games in Beijing (August 22–26, 2026) make the distinction between narrow and general capability concrete. Across 2,056 robots from 666 teams, one machine, Tiangong Ultra, ran the 100 meters in 8.64 seconds — faster than Usain Bolt’s 9.58-second human world record. In the same tournament, coverage of the football, boxing, and weightlifting events described robots struggling in unstructured, contact-heavy conditions (Al Jazeera). A repeatable, engineered sprint is not the same achievement as flexible competence in an unpredictable environment, and the Games illustrate that gap in the same week rather than in the abstract.
The grounding view
Vincent Vanhoucke, who heads robotics at Google DeepMind, offers the sharpest reality check available on this whole trend: “most — if not all — robot learning approaches cannot be deployed for any practical task,” because real deployment requires “99.X percent or higher accuracy and reliability” where academic demonstrations report roughly 80% success — in his words, “a fundamentally different beast,” not a scaling problem to be solved with more data (IEEE Spectrum, May 28, 2024).
Rodney Brooks, the MIT roboticist who co-founded iRobot and Rethink Robotics, presses further on why video-imitation training in particular may be the wrong substrate: touch involves a channel current pipelines never capture, with neuroscience research he cites identifying at least fifteen distinct families of touch-sensing neurons in human skin. He calls today’s manipulation demonstrations early and limited, and dismisses claims that humanoids will reach human-level general dexterity within decades as “pure fantasy thinking” (TechCrunch, Sept. 26, 2025).
Both things are true at once. The foundation-model approach has produced real, measured cross-robot transfer that hand-coded control never achieved, and every credible lab publishing numbers — not just skeptics — reports success rates well short of what industrial and household reliability actually requires.