SAI

Dita

Shanghai AI Laboratory, China 路 March 2025

Shanghai AI Laboratory, Zhejiang University, SenseTime Research, CUHK MMLab, Peking University, Tsinghua University and HKISI CAS

Robot foundation model 路 Open weights, Apache-2.0 路 Also written Diffusion Transformer Policy, RoboDita

Parameters
334 M
334M total, 221M trainable (paper appendix).
GPU memory
0.6 GB
weights at bf16, computed. Fits an 8 GB card
Inference latency
not published
per action step
Weights
Apache-2.0
Apache-2.0 according to the repository's LICENSE file; the README badge says MIT.

What it is

Dita is a 334M parameter diffusion transformer policy pretrained on Open X-Embodiment, from Shanghai AI Laboratory and partners (ICCV 2025). Instead of a small diffusion head on top of a fused embedding, the transformer itself denoises the action chunk in context, attending to raw visual tokens from a DINOv2 encoder and to a frozen CLIP text embedding. The authors call it a vision language action policy, but there is no vision language model inside, which is why it is listed here. Fine-tuned, it reaches 82.4 percent average on LIBERO and an average length of 3.61 on CALVIN ABC to D, and it adapts to a real Franka from 10 demonstrations per task with a single third-person camera.

Architecture

Backbone
No VLM backbone. DINOv2 image encoder fine-tuned end to end, a Q-Former with FiLM text injection that compresses each image to 32 tokens, a frozen CLIP text encoder, and a LLaMA2-style causal transformer with 12 blocks of width 768 trained from scratch.
Action head
In-context diffusion: the causal transformer denoises the action tokens directly (DDPM training with 1,000 steps, DDIM with 20 steps at evaluation). 7-dimensional end-effector actions.
Action chunk
16 steps per forward pass
Control rate
3 Hz
Parameters
334M total, 221M trainable (paper appendix). LoRA fine-tuning leaves about 11M trainable. Control rate above is for the real Franka.
Pretraining data
A subset of Open X-Embodiment selected following the Octo and OpenVLA recipes; a separate checkpoint pretrained on DROID is also released.
Embodiments
Franka Emika Panda with Robotiq gripper (real, 10-shot fine-tuning), Google robot (SimplerEnv), LIBERO and CALVIN simulation

What hardware it needs

This is the section most people came for, so it is worth being precise about which numbers are measured and which are arithmetic.

Computed from 334 M parameters. Weights only, so treat it as a floor: activations, image encoder intermediates and the CUDA context come on top, in practice 30 to 50 percent more for inference.
PrecisionWeightsNote
fp321.2 GBTraining master weights, rarely used for inference
bf16 or fp160.6 GBThe usual way these checkpoints are served
int80.3 GBQuantised, expect some loss of precision on fine motions
int40.2 GBAggressive quantisation, verify success rate before trusting it

Smallest real card that fits the bf16 weights plus a 40 percent runtime allowance: 8 GB. That is an estimate for inference, not for fine tuning. Full fine tuning also holds optimiser state and gradients, which typically costs several times the weights unless you use LoRA or a comparable adapter.

Published figures. These come from the model authors, not from this site.
Control rate3 Hz

No memory or latency figure. Pretraining used 32 A100 GPUs at batch size 8,192 for 100,000 steps; real-robot inference ran on one A100.

Reported results

Grouped by what the evaluation actually asked. Values from different suites are not comparable with each other, so each one keeps its suite and split.

Simulation

Reproducible benchmark suites. Everyone runs the same episodes, so the numbers can be compared inside a suite.

  • LIBERO average over Spatial, Object, Goal and Long, fine-tuned per suite, third-person camera
    82.4%
    success rateAuthors' own evaluation. Per suite: Spatial 84.2, Object 96.3, Goal 85.4, Long 63.8. Octo fine-tuned 75.1 and OpenVLA fine-tuned 76.5 in the same table.source
  • CALVIN ABC to D
    3.61 tasks (out of 5)
    average length (tasks completed in sequence)Authors' own evaluation without image augmentation; 3.78 with augmentation. Five tasks in a row succeed in 50.0 percent of sequences.source

Real world

No results in this category are published for this model.

Fine tuned tasks

No results in this category are published for this model.

On real hardware

Fine-tuning from 10 demonstrations per task on a Franka Emika Panda with a third-person RealSense camera, covering pick and place, pouring, stacking, rotation, drawer and long-horizon tasks. With full fine-tuning the policy still completed 20 percent of long-horizon tasks under extreme variations, where LoRA fine-tuning failed; trial counts are not stated.

Fine tuning it yourself

Fine-tune from the Open X-Embodiment checkpoint with LoRA or fully; the authors report that full fine-tuning is clearly better for robustness and long tasks. Scripts for 10-shot real-robot fine-tuning, CALVIN and LIBERO are in the repository.

Where it helps, where it does not

Strengths

  • Compact at 334M parameters, 221M of them trainable.
  • Adapts to a new real setup from 10 demonstrations per task with one third-person camera.
  • Open code and checkpoints, including a DROID-pretrained variant.

Limits

  • Licence is unclear between the LICENSE file (Apache-2.0) and the README badge (MIT).
  • Checkpoints sit on Google Drive rather than on a model hub.
  • Real-robot results come as figures without trial counts.
  • 3 Hz control on the real robot.

Sources

Everything on this page was taken from these documents. Where they disagree with what you read here, they win.

Entry last checked 2026-11-12.

Try a policy on a real arm

A real SO-100 runs on /live and can be driven from the browser when it is online. Five of the models in this arena can be fine tuned on a dataset you record with your own arm.