SnackOnAI Engineering | Senior AI Systems Researcher | Technical Deep Dive | November 16, 2026
GR00T N1 started from a vision-language model and bolted on an action head. GR00T 2 starts from a video generator, Cosmos-Predict2.5-2B, and trains it to imagine the next second of camera footage while it writes the motor commands to make that footage happen. Over 91% of its training hours are humans doing things, not robots.
The second story is quieter. The longest, most impressive results don't come from the model alone. Toy assembly goes from 143 to 292 successes out of 300 when you add corrective data and a reasoning agent around it. GR00T 2 is a very good hand. The loop around it is the reason it finishes the job.
The promise: by the end you'll know how one 2B transformer predicts video and actions together, why 280 is the number of tokens it spends per camera frame, and which of NVIDIA's headline numbers measure the model and which measure the system around it.
The fence: we read the GR00T 2 project page and the whitepaper (November 2026, read in sections; some appendix text was cut off), the GR00T N1, DreamZero and SimFoundry papers, and launch coverage from CoRL in Austin on November 9. There is no code or weights yet. The page says "Code Available in December," and no license is stated. Every code block below is a reconstruction from the whitepaper's description, labeled as such. We could not open the Hugging Face dataset card directly. We didn't run anything.
What It Actually Does
GR00T 2 is a world-action model (WAM): a policy that, given camera history, robot state and an instruction, outputs a chunk of motor commands and a prediction of what every camera will see next. A conventional vision-language-action model (VLA) maps image plus text straight to actions. A WAM has to picture the outcome too.
GR00T N1 (March 2025) | GR00T 2 (November 2026) | |
|---|---|---|
Backbone | Eagle-2 vision-language model | Cosmos-Predict2.5-2B video diffusion transformer |
What it predicts | Actions | Future video + actions, jointly |
Language | Inside the VLM | Frozen Cosmos-Reason1, via cross-attention |
Biggest data source | Robot and synthetic trajectories | 402,730 h of egocentric human video |
Adaptation shown | Fine-tune per task | Rank-32 LoRA, 1 to 5 demos, ~30 min on one RTX 5090 |
Weights | Open (N1.x) | "December," license unstated |
There are two models. GR00T 2 (2B) handles bimanual arms and 22-DoF dexterous hands. GR00T 2 Turbo keeps the same 2B transformer but swaps in a video encoder with 16x spatial compression, so it sees a 4x smaller grid and runs onboard a Unitree G1 humanoid's Jetson AGX Thor at 124 ms per plan.
The pretraining mix, in hours:
Source | Hours | Share |
|---|---|---|
Egocentric human video (Human-1M, filtered) | 402,730 | 91.7% |
UMI handheld gripper demos | 10,184 | 2.3% |
Robot teleoperation (YAM, Sharpa, Wuji, G1, Franka) | 23,500 | 5.3% |
In-lab human studio data | 3,000 | 0.7% |
The "436,000+ hours" in NVIDIA's caption is exactly the first three rows: 402,730 + 10,184 + 23,500 = 436,414. Robots contribute about one hour in twenty.
The Architecture, Unpacked

What to notice: the predicted video is never executed. It exists to force the model to understand physics, and at inference the expensive video pass is cached so the action pass can iterate cheaply.
Three decisions define the system.
Inherit physics from a video model. Cosmos-Predict2.5 was pretrained to simulate the world as video. Starting there means the backbone already knows how objects fall, slide and deform before it sees a single robot. The same group's DreamZero proved the idea in February on a 14B Wan2.1 backbone at about 7 Hz. GR00T 2 is the same bet at one seventh the size.
Let humans be the dataset. Human hand-pose trajectories, 21 3D keypoints per hand from head-camera video, serve as action supervision in pretraining. For transfer to robots, gloves plus wrist trackers feed a closed-form inverse kinematics solver that retargets human motion to 7-DoF arms and 22-DoF hands at about 118,000 bimanual solves per second. The video objective also learns from footage with no action labels at all.
Keep the robot-specific part tiny. State encoders, action decoders and one output head are per embodiment. Everything else is shared. That's how one checkpoint drives parallel grippers, two 22-DoF hands and a humanoid, and it's why a rank-32 LoRA is enough to adapt.
The Code, Annotated
Snippet one: the denoising loop. A reconstruction of the whitepaper's "joint passes" and "cached passes." No code is public until December. The function and argument names are ours.
import torch
@torch.no_grad()
def plan_chunk(dit, obs_tokens, text_emb, state_tok, video_steps=1, action_steps=2):
"""One control step: imagine the next frames, write the next 48 actions."""
video = torch.randn(B, N_FUTURE_VIDEO_TOKENS, D) # SnackOnAI: future video starts as pure noise
action = torch.randn(B, 48, ACTION_DIM) # 48-step chunk at 30 Hz, only 30 executed
kv_cache = None
for k in range(max(video_steps, action_steps)):
t = 1.0 - k / max(video_steps, action_steps) # flow matching: integrate noise → data
if k < video_steps:
# JOINT pass: video and action tokens attend to each other in one 2B transformer
v_vel, a_vel, kv_cache = dit(obs_tokens, video, action, state_tok, text_emb, t,
return_kv=True)
video = video + v_vel / video_steps
else:
# CACHED pass: reuse video keys/values, recompute only the 48 action tokens
a_vel = dit.action_only(action, state_tok, kv_cache, t) # ← THIS is the trick
action = action + a_vel / action_steps
return action, video # SnackOnAI: video is returned for monitoring, never sent to motors
The video tokens dwarf the action tokens, so refining actions against a frozen video cache costs a fraction of a full pass. The paper reports 1 video step and 2 action steps in its real-world runs, and TensorRT, CUDA graphs and FP8 cut latency 2.9x to 7.2x.
Snippet two: five-demo adaptation. The recipe from whitepaper Section A.3, written against Hugging Face PEFT. The module names are our assumption.
from peft import LoraConfig, get_peft_model
import torch
for p in model.parameters():
p.requires_grad = False # frozen: text encoder, video VAE, state/action heads, base DiT
cfg = LoraConfig(r=32, lora_alpha=32,
target_modules="all-linear") # paper: rank-32 adapters on EVERY linear layer of the DiT
model.dit = get_peft_model(model.dit, cfg)
# Each of the 1 to 5 demos is cut into 100 evenly spaced windows,
# sampled in proportion to the FROZEN model's loss on that window.
with torch.no_grad():
losses = torch.stack([joint_flow_loss(base_model, w) for w in windows])
weights = losses / losses.sum() # ← THIS is the trick: spend steps where the prior is wrong
sampler = torch.utils.data.WeightedRandomSampler(weights, num_samples=len(windows) * 6)
# Same joint video + action flow-matching loss as pretraining, ~30 min on one RTX 5090.
# Afterwards: model.dit = model.dit.merge_and_unload() → zero extra serving latency
Two choices make five demos enough: the adapter keeps the video objective, so the model still has to predict what happens, and loss-weighted sampling skips the parts of a demo the pretrained prior already handles.
Snippet three: our verification arithmetic.
# Data mix
hours = {"human": 402_730, "umi": 10_184, "teleop": 23_500, "studio": 3_000}
print(sum(hours[k] for k in ("human", "umi", "teleop"))) # 436,414 → NVIDIA's "436,000+"
print(round(100 * hours["human"] / sum(hours.values()), 1)) # 91.7% human video
# Tokens per camera frame: Wan 2.1 VAE downsamples 8x, the DiT patchifies 2x2
lat = (224 // 8) * (320 // 8) # 28 × 40 = 1,120 latent positions
print(lat // 4) # 280 tokens ✓ matches the paper
print((224 // 16) * (320 // 16) // 4) # Turbo's 16x VAE: 70 tokens, the "4x smaller grid"
# Illustrative only: 3 cameras × 2 future latent frames vs one 48-step action chunk
print(round(100 * 3 * 2 * 280 / (3 * 2 * 280 + 48), 1)) # ~97% of predicted tokens are video
# How tight is "95%"? Wilson 95% interval, assuming 4 trials per task × 50 tasks
from statsmodels.stats.proportion import proportion_confint
print(proportion_confint(190, 200, method="wilson")) # ≈ (0.910, 0.973)
280 tokens per frame reproduces exactly from the encoder strides. The roughly 97% video share is our illustration; the number of future latent frames isn't published, but the paper itself admits most tokens go to future video "much of which is not relevant to the next action."
It In Action
Task: "Screw in bulb," one of the 50 unseen Dexter tasks. Dexter is a Tianji robot with two 22-DoF Sharpa hands.
Step one, zero-shot. The pretrained model reads 0.8 s of history from three cameras plus the instruction. Across the Dexter-50 suite it averages 40%, and 40 of 50 tasks succeed at least once. Most misses, the authors say, are precision failures, not misunderstanding the goal.
Step two, collect five demos. Five teleoperated episodes, under 3 minutes of robot data. Against 436,414 hours of pretraining, that's roughly one part in 8.8 million.
Step three, adapt. Rank-32 LoRA, about six epochs over 100 loss-weighted windows per demo, ~30 minutes on one RTX 5090. Merge the adapter.
Step four, run. Each control step takes 153 ms median on the RTX 5090 with three cameras. A chunk predicts 48 steps at 30 Hz and executes 30, about 1 second of motion, so planning uses roughly 15% of the time budget and real-time chunking overlaps the next plan with the current motion.
Output (Dexter-50 averages):
Demos per task | Average success | Tasks perfect in every trial |
|---|---|---|
0 | 40% | n/a (40 of 50 succeed at least once) |
1 | 71% | n/a |
5 | 95% | 37 of 50 |
Dexter-50 average success
zero-shot ████████████████ 40%
1 demo ████████████████████████████▍ 71%
5 demos ██████████████████████████████████████ 95%
If each task ran four trials, 95% means 190 of 200, with a 95% confidence interval of about 91% to 97%. The paper's own open-problems section concedes that 20 trials per task can't separate 95% from 99%.
Why This Design Works (and what it trades away)
It works because video is the one data source that scales. Robot teleop costs operator hours per episode. Egocentric human video comes from 51,500+ collectors across 123,000 scenes. The scaling ablation is the most important chart in the paper: going from 0% to 100% of human video lifts real-robot success on 72 Dexter tasks from 18% to 41%, and "none of the metrics levels off at 100%."
It works because predicting video is a dense training signal. An action label is a few dozen numbers per step. A future frame is 280 tokens per camera that must be physically consistent. The model can't fake it.
It trades away compute. Most predicted tokens are future video that never moves a motor. The paper names this as its first open problem.
It trades away memory. The model sees 0.8 s of history. Anything longer, which step am I on or what did I already try, is delegated to an outside agent.
It trades away speed at the top of the stack. That agent's median turn takes 7.5 s, against 0.6 s per policy call, and the robot waits during a median 38% of an episode.
It trades away touch. Force is predicted as an output for DexUMI tasks, but tactile sensing isn't an input. Safety of physical actions is listed as unestablished.
Technical Moats
Human-1M. About 1 million hours of sensorized human labor, filtered to 402,730 for training with strict rules: both hands visible, head moving under 20°/s, wrist under 1 m/s, a 95% human-review pass rate. Only 100,000 hours are open.
The capture hardware. GEAR UMI, DexUMI exoskeletons, glove-plus-tracker rigs with robot-matched cameras, and a retargeting stack fast enough to process all of it. The hardware designs are open; the data volume isn't.
Training at rack scale. Pretraining ran on 1,024 GB300 GPUs at about 12,000 sample windows per second with 94.5% packing efficiency in 32,768-token rows. That's a global batch of 16,384 and roughly 0.73 optimizer steps per second.
The evaluation stack. SimFoundry turns one real video into a simulated digital twin, and the team's 25-task YAM sim benchmark correlates with real results at Pearson 0.92. Cheap, trustworthy evaluation compounds faster than any single model.
Contrarian Insights
One: the long-horizon wins belong to the system, not the model. On real toy assembly over 100 trials each, supervised GR00T 2 scored 69, 51 and 23, or 143 of 300. With DAgger corrections and a reasoning agent it scored 98, 97 and 97, 292 of 300. Charger insertion went from 2/20 to 20/20 only after real-world RL. Long-horizon visual steering went from 1.7% to 53.3% because a planner drew marks on the camera image. These are excellent engineering results, but they describe a stack of model, RL, corrective data and agent. Anyone comparing raw policies against "GR00T 2" needs to know which one they're comparing against.
Two: 95% is a five-demo number on NVIDIA's own suite, and the field's independent suites tell a much harder story. The Dexter-50 tasks are named in the whitepaper's figures, but trial counts per task aren't stated, and the authors themselves say 30 to 35% of head-to-head comparisons are ties. Meanwhile the independent RoboDojo benchmark, run by 44 authors over 30 policies, found the best real-world score at 12.8%, by π0.5. GR00T 2's agentic RoboDojo entry reports 75.8%, which is a big claim worth independent replication. Until weights ship in December and other labs run it, treat 95% as "how fast it adapts in NVIDIA's lab."
Surprising Takeaway
GR00T 2 learned more from people than from robots: 91.7% of its pretraining hours are human video, and its own ablation shows real-robot success still climbing at 100% of that data. The robots contribute about one hour in twenty. If the curve keeps rising, the bottleneck for robot dexterity isn't robot fleets. It's head cameras on people doing ordinary jobs, which is a data-collection business, not a hardware one.
TL;DR For Engineers
GR00T 2 is a 2B world-action model built on Cosmos-Predict2.5-2B: one transformer denoises future camera video and a 48-step action chunk together with flow matching.
Pretraining is 436,414 hours, 91.7% egocentric human video. Human video still improves real-robot success at full scale (18% to 41% on 72 tasks).
Adaptation is a rank-32 LoRA on every linear layer, loss-weighted windows, ~30 min on one RTX 5090: 40% zero-shot to 95% with five demos on Dexter-50.
Latency is 153 ms per chunk on an RTX 5090, about 15% of a 1 s chunk. Turbo runs on a G1's Jetson Thor at 124 ms.
The biggest long-horizon numbers include RL, DAgger and a reasoning agent. No code, weights or license until December.
Explain It Like I'm New
Teaching a robot to use its hands has always been expensive. The usual method is to have a person steer the robot through a task again and again, recording every movement, and then train a model to copy those recordings. Every new task, and every new robot, meant starting a big new round of recording.
GR00T 2 takes a different route. It starts from an AI model that was originally built to generate realistic videos, so it already has a rough sense of how the physical world behaves. Then it's trained on hundreds of thousands of hours of ordinary people working with their hands, filmed from head-mounted cameras, plus a much smaller amount of robot footage.
The clever part is what the model is asked to do. Instead of only deciding "move the thumb here," it also has to imagine what the cameras will see a moment later if that move works. Imagining the outcome forces it to understand cause and effect, a bit like a chess player picturing the board after a move.
Because so much is learned in advance, a new task needs only a handful of examples, a few minutes of robot recording and about half an hour on one gaming graphics card.
If that holds up outside NVIDIA's lab, the hard part of building capable robots shifts from collecting robot data to recording how people already work.
See It In Action
GR00T 2 project page videos, NVIDIA GEAR. Watch the rollouts. Uncut task videos for Dexter, YAM and the G1, including the GB300 memory-module seating and human-to-robot transfer demos. The closest thing to the real result until code ships.
Human-1M Dataset Explorer, NVIDIA on Hugging Face Spaces. Browse the data. Scroll through the egocentric clips with hand keypoints and language segments to see exactly what "human video as action data" looks like.
SimFoundry, NVIDIA GEAR and UT Austin. See the real-to-sim pipeline. Shows one real video becoming an interactive simulated scene, the tool behind GR00T 2's 0.92 sim-to-real evaluation correlation.
We found no full talk on GR00T 2 as of November 10. The CoRL presentations are this week, and recordings should follow.
Community Conversation
Tech Times, critique of the 95% claim. Argues "succeeded at least once" is a weak bar and rollout counts are missing, and contrasts RoboDojo's 12.8% best real-world score. One correction: it describes GR00T 2 as based on the 14B DreamZero, but the whitepaper says Cosmos-Predict2.5-2B.
NVIDIA GEAR (DreamZero authors), World Action Models are Zero-shot Policies. The direct ancestor: joint video and action denoising on a 14B backbone, cut from 5.7 s to about 150 ms per chunk, and showing human video improves unseen tasks by over 42%.
Interesting Engineering, human-to-robot skill transfer coverage. Highlights the X-Adapt results: a five-fingered robot doing a lab funnel task at 65% from human demos alone, with zero teleoperation.
SimFoundry authors (NVIDIA, Georgia Tech, Stanford, UT Austin, Toronto), SimFoundry paper. Simulated "cousins" of real scenes improve real-world success by 17 to 40%, which is why NVIDIA can afford to evaluate dozens of policies per week.
Conclusion: The Right Foundation for Robot Hands, Measured by the People Who Built It
GR00T 2 makes the strongest case yet that robot manipulation should be built on video models and trained mostly on human footage. The architecture is clean: one 2B transformer, joint video and action flow matching, a cached action pass for speed, and small per-robot heads. The scaling ablation is the result that matters most, because it says the cheapest data source hasn't run out. The adaptation recipe, a rank-32 LoRA and 30 minutes on a consumer GPU, is genuinely practical. But the most impressive long-horizon numbers come from a system of model, RL, corrective data and agent. Every number so far comes from NVIDIA's own lab, and nothing ships until December. Read it as the right blueprint, then wait for the weights and an outside lab's evaluation before you plan a deployment around 95%.
References
GR00T N1: An Open Foundation Model for Generalist Humanoid Robots, NVIDIA. The VLM-plus-action-head predecessor GR00T 2 departs from.
World Action Models are Zero-shot Policies (DreamZero), NVIDIA. The 14B proof of concept for joint video-action denoising.
SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluation, Ranawaka et al. Real-to-sim digital twins and cousins used for training and evaluation.
Flow Matching for Generative Modeling, Lipman et al. The objective behind both the video and action losses.
Universal Manipulation Interface, Chi et al. The handheld-gripper data collection method behind the 10,184 UMI hours.
LoRA: Low-Rank Adaptation of Large Language Models, Hu et al. The adapter used for five-demo task adaptation.
GR00T 2 is a 2B world-action model built on Cosmos-Predict2.5 that predicts future camera video and robot actions together, pretrained on 436,414 hours that are 91.7% human video. It adapts to new tasks with five demos and a 30-minute LoRA, but its biggest long-horizon results come from RL and a reasoning agent around it, and no code ships until December.
Sponsored Ad If you enjoy practical AI insights, check out SnackOnAI and support the newsletter by subscribing, sharing, and exploring our sponsored ad—it helps us keep building and delivering value 🚀
Frontier models perform better in Attio’s agent harness.
We gave GPT-6 Astra the same job twice: analyze 100 sales calls and draft the follow-ups, once in Ask Attio and once in ChatGPT.
Ask Attio finished in 4 minutes 49 seconds. ChatGPT took 26 minutes 11 seconds, about 5.4 times as long.
The most ambitious GTM work requires more than frontier intelligence. It requires architecture that can put that intelligence to work at scale.



