Diffusion Policy Explained for Robot Manipulation

Behaviour cloning in its simplest form is regression: look at the camera, look at the joint angles, predict the next action. It works until the demonstrations disagree with themselves. A human pushing a block can go around it on the left or on the right, and both are correct. Regression averages them and drives straight into the block. Diffusion Policy fixes that by changing what the network outputs: instead of one action, it denoises a whole future action sequence out of Gaussian noise, conditioned on the last couple of camera frames.

This article works through the paper (arXiv 2303.04137), the reference implementation, and the lerobot port you can actually run on an SO-100. It covers what receding horizon control buys you, why the action execution horizon is the one hyperparameter that decides whether the arm is smooth or hesitant, and what the denoising loop costs per control step on real hardware. It also says plainly where AY-Robots does not help: there is no diffusion trainer on this platform.

The problem single-action regression cannot solve

An imitation learning dataset recorded by teleoperation is not a function. The same observation appears many times with different labels, because the operator made a different choice on Tuesday than on Monday. The paper names three consequences that a plain regression head handles badly, and each of them shows up on a real arm.

The earlier fixes were all about the output representation: mixtures of Gaussians, categorical bins over quantised actions, or energy-based implicit policies. Implicit Behavioural Cloning gets multimodality right in theory but needs negative sampling for an intractable normalisation constant, and the paper documents the resulting training instability: its training loss falls smoothly while its action prediction accuracy does not improve, and its evaluation success rate oscillates, which makes checkpoint selection guesswork. For the real Push-T comparison the authors had to select the IBC checkpoint by minimum training-set action MSE instead of by the last checkpoint they used for everything else. Diffusion sidesteps the normalisation constant entirely: it models the score, the gradient of the log density, and the constant differentiates away.

What the denoising loop actually computes

At time step t the policy takes the last To observation steps, calls them Ot, and produces Tp future actions At. Training is the standard DDPM objective with one change: the observation is a condition, not part of the thing being denoised. Add noise to a ground-truth action chunk, ask the network to predict the noise it added, take the MSE. That is the whole loss.

Keeping observations out of the denoised output matters more than it looks. The vision encoder runs once per inference, not once per denoising iteration, which is what makes the method fast enough to close a loop at all. The paper states this explicitly as the reason the formulation accommodates real-time control and makes end-to-end vision training feasible.

# 1. encode observations ONCE -> global conditioning vector
global_cond = encode(state_queue, image_queue)     # ResNet18 + spatial softmax

# 2. start from pure noise shaped (batch, horizon, action_dim)
sample = torch.randn(batch_size, config.horizon, action_dim)

# 3. denoise. num_inference_steps is None by default -> num_train_timesteps = 100
noise_scheduler.set_timesteps(num_inference_steps)
for t in noise_scheduler.timesteps:
    model_output = unet(sample, t, global_cond=global_cond)   # <- one U-Net call
    sample = noise_scheduler.step(model_output, t, sample).prev_sample

# 4. keep n_action_steps of the chunk, starting at the CURRENT observation,
#    not at the start of the horizon. Everything else is thrown away.
start = config.n_obs_steps - 1
actions = sample[:, start : start + config.n_action_steps]

Step 3 is the entire cost story. The U-Net is called once per denoising iteration, and with lerobot's defaults that loop runs 100 times before a single action reaches the servos. Step 4 is the entire behaviour story: most of what the network predicted is discarded on purpose.

Receding horizon control: predict a lot, commit to a little

The idea is borrowed straight from model predictive control, and the paper cites Mayne and Michalska 1988 for it. Predict a long chunk, execute a short prefix, throw the rest away, re-plan from a fresh observation. You get temporal consistency from the long prediction and reactivity from the short commitment. lerobot's own docstring draws the timeline better than prose does.

(legend: o = n_obs_steps, h = horizon, a = n_action_steps)
|timestep            | n-o+1 | n-o+2 | ..... | n     | ..... | n+a-1 | n+a   | ..... | n-o+h |
|observation is used | YES   | YES   | YES   | YES   | NO    | NO    | NO    | NO    | NO    |
|action is generated | YES   | YES   | YES   | YES   | YES   | YES   | YES   | YES   | YES   |
|action is used      | NO    | NO    | NO    | YES   | YES   | YES   | NO    | NO    | NO    |

constraint: n_action_steps <= horizon - n_obs_steps + 1

Two things follow. First, the horizon is measured from the first observation, not from now, so with n_obs_steps = 2 you lose a step at the front. Second, this is action chunking with a re-plan rule, which is exactly the family ACT belongs to. The difference is that ACT produces its chunk in one forward pass while Diffusion Policy produces it in a hundred.

The execution horizon is the knob that decides how the arm feels. The paper's ablation (Figure 5, left) shows the trade-off directly and lands on 8 steps for most tasks. At 10 Hz on their UR5 that is 0.8 s of committed motion, or 0.6 s for real Push-T where they used Ta = 6. At lerobot's recording default of 30 fps, the current n_action_steps = 32 commits you to roughly 1.07 s of open-loop motion before the policy looks at a camera again.

There is a second, less obvious payoff. Because the chunk extends into the future, a policy that is a few control steps behind reality can still hand over a valid action. The paper measured this: with position control, Diffusion Policy holds peak performance with a simulated latency of up to 4 steps. Velocity control degrades faster, which the authors attribute to compounding error. That is the technical reason the paper insists on end-effector position control rather than velocity commands.

The numbers the paper actually reports

Two backbones are offered for the noise prediction network. A 1D temporal CNN, a U-Net over the time axis adapted from Janner et al., conditioned with FiLM on both the observation embedding and the denoising iteration. And a time-series diffusion transformer built on minGPT, with noisy actions as input tokens and the iteration index as a prepended token. The recommendation in the paper is blunt: start with the CNN, move to the transformer only if the task has fast and sharp action changes, and expect to tune more when you do. Both variants have entries in the arena, so you can read their published benchmark numbers next to the modern VLAs.

The headline is a 46.9% average relative improvement over the best baseline per task across 15 tasks and 4 benchmarks. That number is a mean of per-task relative improvements, computed as 0.46858, not a success-rate delta, so read it as a ranking claim rather than an absolute one. The long-horizon results are more interesting: 32% improvement on the Block Push two-block metric and 213% on the Franka Kitchen four-object metric, both cases where the order of sub-goals is arbitrary and a policy that cannot commit falls apart. None of it is a claim about how many training steps you personally will need, which depends on your data.

• Real UR5 Push-T, 136 demonstration episodes, a fixed 12 hours of training per method: 95% success and 0.8 average IoU against 0.84 for the human operator. The best IBC variant scored 0% and the best LSTM-GMM 20%.
• Sauce pouring and spreading on a Franka: 50 demonstrations per task, 90% used for training, 1000 epochs, last checkpoint evaluated.
• Bimanual shirt folding: 284 demonstrations, 75% success over 20 trials. Reported failure modes are missed grasps on the sleeves and collar, and the policy being unable to stop adjusting the shirt at the end.
• Vision encoder ablation on robomimic square (Table 5), CNN backbone, 500 epochs, 50 initial conditions. The full numbers are in the callout below; the short version is that finetuning a pretrained encoder at a lower learning rate wins in all three architectures tested.
• Real-robot loop on the UR5 station: the policy commands at 10 Hz, the controller linearly interpolates to 125 Hz. Five RealSense D415 cameras record the scene, two of them feed the policy, downsampled to 320x240 at 10 fps. Demonstrations came from a 3Dconnexion SpaceMouse at 10 Hz.
• Training length in the reference repo's real-robot config: 600 epochs, AdamW at lr 1e-4 with betas 0.95 and 0.999, weight decay 1e-6, EMA enabled, a checkpoint every 50 epochs. The config comments say EMA destroys performance with BatchNorm, which is why the encoder uses GroupNorm.

What it costs at inference on a real arm

The paper is honest about this in its limitations section: Diffusion Policy has higher computational cost and higher inference latency than simpler methods like LSTM-GMM, and action sequence prediction only partially mitigates it. The one hard measurement they give is 0.1 s per chunk on an Nvidia 3080, using DDIM with 100 training iterations and 10 inference iterations.

Running it yourself with lerobot on an SO-100

The reference repo targets a UR5 with RealSense cameras and a SpaceMouse. For a low-cost arm the practical route is lerobot's port, which registers Diffusion Policy under the config name diffusion and trains it with the same lerobot-train entry point as ACT or SmolVLA. Record first, then train, then roll out.

1. 1
   Install lerobot with the diffusion extra

   Diffusion Policy depends on Hugging Face diffusers for its schedulers, which is an optional extra: _make_noise_scheduler calls require_package("diffusers", extra="diffusion"), so without it the policy raises at construction time. On main the pin is diffusers>=0.38.0,<0.40.0. The recording and rollout CLIs live behind a second extra, core_scripts.

   bash

   git clone https://github.com/huggingface/lerobot.git
   cd lerobot
   pip install -e ".[diffusion,core_scripts]"
   lerobot-train --help

2. 2
   Record a dataset from teleoperation

   Defaults on main: fps=30, episode_time_s=60, reset_time_s=60, num_episodes=50. Vary the start pose every episode. A diffusion policy is exactly as multimodal as your demonstrations and no more.

   bash

   lerobot-record \
     --robot.type=so100_follower \
     --robot.port=/dev/tty.usbmodem58760431541 \
     --robot.id=black \
     --robot.cameras="{front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
     --teleop.type=so100_leader \
     --teleop.port=/dev/tty.usbmodem58760431551 \
     --teleop.id=blue \
     --dataset.repo_id=${HF_USER}/so100_pickplace \
     --dataset.num_episodes=50 \
     --dataset.single_task="Pick up the cube and drop it in the bin" \
     --display_data=true

3. 3
   Train with the diffusion policy type

   lerobot's global defaults are seed=1000, batch_size=8, steps=100000, save_freq=20000. Batch 8 is small for this policy; the paper used 64 for image-based tasks and the published lerobot/diffusion_pusht checkpoint used 64 as well.

   bash

   lerobot-train \
     --dataset.repo_id=${HF_USER}/so100_pickplace \
     --policy.type=diffusion \
     --policy.device=cuda \
     --batch_size=64 \
     --steps=200000 \
     --output_dir=outputs/train/so100_diffusion \
     --job_name=so100_diffusion \
     --wandb.enable=true

4. 4
   Set the horizons explicitly instead of inheriting them

   The defaults changed between v0.5.1 and v0.6.0. Pin them in the command so your run is reproducible against a written-down number rather than against whatever main did last month.

   bash

   # the pre-0.6.0 defaults, which match the paper
     --policy.horizon=16 \
     --policy.n_action_steps=8 \
     --policy.n_obs_steps=2 \
     --policy.drop_n_last_frames=7
   
   # __post_init__ validates exactly one of these two rules:
   #   horizon % (2 ** len(down_dims)) == 0, i.e. a multiple of 8
   # the other one is only written down in the select_action docstring:
   #   n_action_steps <= horizon - n_obs_steps + 1   <- nothing checks this

5. 5
   Cut the denoising steps and measure what it costs you

   Train with DDPM at 100 steps, then evaluate the same checkpoint at several inference budgets. DDIM decouples training and inference iteration counts, which is the whole reason it is in the config.

   bash

     --policy.noise_scheduler_type=DDIM \
     --policy.num_inference_steps=10

6. 6
   Roll it out on the arm with lerobot-rollout

   This is the step most old tutorials get wrong. On lerobot main lerobot-record is a pure data-collection tool with no policy inference: it has no --policy.path field, and it rejects dataset names starting with eval_ with an error telling you to use lerobot-rollout. The rollout CLI takes a --strategy.type of base, sentry, highlight, dagger or episodic; episodic is the one with reset phases between episodes. Keep a hand on the power switch for the first episode.

   bash

   lerobot-rollout \
     --strategy.type=episodic \
     --policy.path=outputs/train/so100_diffusion/checkpoints/last/pretrained_model \
     --robot.type=so100_follower \
     --robot.port=/dev/tty.usbmodem58760431541 \
     --robot.id=black \
     --robot.cameras="{front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
     --dataset.repo_id=${HF_USER}/rollout_so100_diffusion \
     --dataset.num_episodes=10 \
     --dataset.single_task="Pick up the cube and drop it in the bin"

For a sanity check that the pipeline works before you spend a day on your own data, the published lerobot/diffusion_pusht checkpoint was trained on the lerobot/pusht dataset with --batch_size=64 --steps=200000, and the weights that are actually published are the checkpoint at 175k steps. Its model card reports 65.4% success over 500 evaluation episodes with an average maximum overlap ratio of 0.955, against 64.2% and 0.957 for an equivalent model trained in the original repo. Note that the card's commands still use the old python lerobot/scripts/train.py form, which predates the console entry points. Translate them before you paste.

Where Diffusion Policy sits next to what you can train today

Three families descend from the same observation that a robot policy should output a sequence. ACT predicts a chunk in one pass with a transformer and a CVAE. Diffusion Policy denoises the chunk over many passes. Flow matching policies like Pi0.5 learn a continuous velocity field to the same end with far fewer integration steps, which is why the family moved in that direction. The platform numbers below come from the AY-Robots catalog; the Diffusion Policy row comes from the paper.

Read the latency column and the action-representation column together. The 20 ms, 152 ms, 245 ms and 485 ms figures are the platform's own measured per-action-step numbers for the models it trains. Diffusion Policy has no row there because it is not trained here, and its cost has a different shape: one U-Net call multiplied by num_inference_steps, per chunk, before any action reaches the servos. That is the entire trade. You pay N forward passes to buy a properly multimodal action distribution, and whether the purchase is worth it depends on whether your demonstrations are actually multimodal, which is a property of how you recorded them rather than of the model. The data collection guide covers how to record demonstrations that are varied without being inconsistent.

Honest limits, including on this platform

Two of these are properties of the method and two are properties of where you run it. None of them are fixable by tuning.

• No language, no multi-task. lerobot's diffusion config accepts observation.state plus at least one observation.image* key or observation.environment_state. There is no task string. One checkpoint per task, exactly like ACT, and unlike every VLA on this platform.
• No pretraining to borrow. There is no diffusion policy foundation checkpoint to fine-tune. The vision encoder can start from ImageNet weights, the denoiser cannot start from anything.
• Inference latency is the deployment constraint. Remote inference over the public internet is viable for slow pick-and-place and not for fast reactive motion, whatever the model. Adding round trips to a policy that already spends 100 U-Net calls per chunk makes it worse, not better.
• AY-Robots will not train this for you. The five trainable policies are fixed. What the platform contributes to a diffusion project is the arm, the teleoperation, the recording client and the public dataset directory, not the trainer.

What came after, and what to read next

The paper's own limitations section points at the fix: fewer inference steps through better noise schedules, better solvers, or consistency models. That work happened. Consistency Policy (Prasad, Lin, Wu, Zhou and Bohg, May 2024) distills a trained Diffusion Policy by enforcing self-consistency along its own denoising trajectories, and reports an order of magnitude speedup over the fastest alternative method while demonstrating inference on a laptop GPU. 3D Diffusion Policy (Ze et al., March 2024) swaps the image encoder for a point-cloud encoder and reports 85% success on four real tasks from 40 demonstrations each. Universal Manipulation Interface (Chi et al., February 2024, the same first author) attacks the data side instead, collecting demonstrations with a handheld gripper rather than a robot.

The larger movement was toward flow matching, which reaches a multimodal action distribution with a straighter path and far fewer integration steps than a 100-step denoising loop. That is what Pi0 and Pi0.5 do. If you are choosing a model to deploy rather than to understand, start from the VLA overview, then the ACT versus SmolVLA comparison for the cheap end and the pricing page for what a run costs. If you have no arm yet, /live streams a physical SO-100 with no signup.