DreamGen

NVIDIA, United States 路 May 2025

NVIDIA (GEAR Lab)

World model 路 Open weights, NVIDIA Open Model 路 Also written GR00T Dreams, GR00T-Dreams, Isaac GR00T-Dreams blueprint

Parameters
14 B
Size of the released Cosmos-Predict2-14B GR00T Dreams checkpoints, as named on Hugging Face.
GPU memory
26 GB
weights at bf16, computed. Fits a 40 GB card
Inference latency
not published
per action step
Weights
NVIDIA Open Model
NVIDIA Open Model License for the released Cosmos-Predict2-14B GR00T Dreams checkpoints (GR1 and DROID), which ask for licence...

What it is

DreamGen is NVIDIA's recipe for training robot policies on videos that a world model has imagined. It fine-tunes an image to video model on the target robot, generates videos of new behaviours from one image and a text instruction, labels them with pseudo-actions from an inverse dynamics or latent action model, and trains a policy such as GR00T N1 on these neural trajectories. From teleoperation data of a single pick-and-place task in one environment, a Fourier GR1 humanoid learned 22 new behaviours: on 14 new behaviours in a seen environment success rose from 11.2 to 43.2 percent, and on 13 tasks in 10 unseen environments from 0 to 28.5 percent. The released workflow, GR00T Dreams, uses Cosmos-Predict2 as the world model and ships inverse dynamics models for Franka, GR1, SO-100 and RoboCasa.

Architecture

Backbone
An image to video world model fine-tuned on the target robot: WAN2.1 for most experiments in the paper, Cosmos-Predict2-14B in the released workflow, with checkpoints post-trained on GR1 humanoid and DROID video.
Action head
None of its own. Pseudo-actions for the generated videos come from an inverse dynamics model or a latent action model, and a separate policy (GR00T N1, Diffusion Policy or 蟺0 in the paper) is trained on them.
Parameters
Size of the released Cosmos-Predict2-14B GR00T Dreams checkpoints, as named on Hugging Face. DreamGen itself is a pipeline: the paper mostly fine-tunes WAN2.1, and the inverse dynamics model and the policy are separate networks.
Pretraining data
Inherits the video pretraining of the base world model. Robot fine-tuning uses teleoperation video of the target robot; the humanoid study starts from a single pick-and-place task in one environment and generates 50 neural trajectories per new task.
Embodiments
Fourier GR1 humanoid, Franka Emika Panda, SO-100 arm, RoboCasa simulation

What hardware it needs

This is the section most people came for, so it is worth being precise about which numbers are measured and which are arithmetic.

Computed from 14 B parameters. Weights only, so treat it as a floor: activations, image encoder intermediates and the CUDA context come on top, in practice 30 to 50 percent more for inference.
PrecisionWeightsNote
fp3252 GBTraining master weights, rarely used for inference
bf16 or fp1626 GBThe usual way these checkpoints are served
int813 GBQuantised, expect some loss of precision on fine motions
int46.5 GBAggressive quantisation, verify success rate before trusting it

Smallest real card that fits the bf16 weights plus a 40 percent runtime allowance: 40 GB. That is an estimate for inference, not for fine tuning. Full fine tuning also holds optimiser state and gradients, which typically costs several times the weights unless you use LoRA or a comparable adapter.

No memory or speed figure is published for the GR00T Dreams checkpoints or for the pipeline.

The authors publish no memory or latency figure for this model. Everything above is computed from the parameter count.

Reported results

Grouped by what the evaluation actually asked. Values from different suites are not comparable with each other, so each one keeps its suite and split.

Simulation

Reproducible benchmark suites. Everyone runs the same episodes, so the numbers can be compared inside a suite.

  • RoboCasa 24 tasks, 300 demonstrations plus neural trajectories, GR00T N1
    57.61%
    average success rateAuthors' own evaluation. Without neural trajectories 49.59 percent. With 30 and 100 demonstrations the neural trajectories lift 17.44 to 23.32 and 32.07 to 39.94 percent; neural trajectories alone reach 20.55 percent.source

Real world

Rollouts on physical hardware. The setups differ, so read these as evidence, not as a ranking.

  • Real robot, SO-100 arm (DreamGen paper protocol) 2 tasks, 10 and 13 real demonstrations plus neural trajectories, GR00T N1
    45.5%
    average success rateAuthors' own evaluation, 10 trials per task. Without neural trajectories 21 percent.source
  • Real robot, Fourier GR1 humanoid (DreamGen paper protocol) 14 new behaviours in a seen environment, GR00T N1 with neural trajectories, trained from one pick-and-place task
    43.2%
    success rateAuthors' own evaluation, 10 rollouts per checkpoint. Baseline trained only on the pick-and-place data: 11.2 percent, partly partial credit for grasping.source
  • Real robot, Fourier GR1 humanoid (DreamGen paper protocol) 13 tasks in 10 unseen environments, GR00T N1 with neural trajectories
    28.5%
    success rateAuthors' own evaluation. Baseline 0 percent.source

Fine tuned tasks

No results in this category are published for this model.

On real hardware

Humanoid study: 43.2 percent on 14 new behaviours in a seen environment (baseline 11.2) and 28.5 percent on 13 tasks in 10 unseen environments (baseline 0). Data augmentation with neural trajectories for GR00T N1: GR1 four tasks 37 to 46.4 percent (46.0 in the paper's table), Franka three tasks 23 to 37 percent, SO-100 two tasks 21 to 45.5 percent, with about 10 rollouts per checkpoint.

Fine tuning it yourself

The GR00T-Dreams repository walks through the steps: post-train Cosmos-Predict2 on your robot's videos, generate videos from an image and an instruction, extract actions with an inverse dynamics model into LeRobot format, and fine-tune GR00T N1. Inverse dynamics models are provided for Franka, GR1, SO-100 and RoboCasa; another robot needs its own, trained from a few real trajectories.

Where it helps, where it does not

Strengths

  • Teaches behaviours that were never demonstrated: 22 new behaviours on a humanoid from one pick-and-place dataset.
  • Works with an SO-100: the repository ships an inverse dynamics model for the arm, and the paper reports 21 to 45.5 percent on two SO-100 tasks.
  • Gains grow with the amount of generated data, log-linearly up to 333 times the original demonstrations in RoboCasa.
  • Not tied to one policy: the paper reports gains for GR00T N1, Diffusion Policy and 蟺0.

Limits

  • A pipeline, not one model: world model, inverse dynamics model and policy are trained separately.
  • Absolute success stays modest: 43.2 percent on new behaviours and 28.5 percent in new environments.
  • Real-world numbers rest on about 10 rollouts per checkpoint, so single values carry wide uncertainty.
  • Released checkpoints are under the NVIDIA Open Model License and need licence acceptance on Hugging Face.

Sources

Everything on this page was taken from these documents. Where they disagree with what you read here, they win.

Entry last checked 2026-11-12.

Try a policy on a real arm

A real SO-100 runs on /live and can be driven from the browser when it is online. Five of the models in this arena can be fine tuned on a dataset you record with your own arm.