Dita
Shanghai AI Laboratory, China 路 March 2025
Shanghai AI Laboratory, Zhejiang University, SenseTime Research, CUHK MMLab, Peking University, Tsinghua University and HKISI CAS
Robot foundation model 路 Open weights, Apache-2.0 路 Also written Diffusion Transformer Policy, RoboDita
- Parameters
- 334 M
- 334M total, 221M trainable (paper appendix).
- GPU memory
- 0.6 GB
- weights at bf16, computed. Fits an 8 GB card
- Inference latency
- not published
- per action step
- Weights
- Apache-2.0
- Apache-2.0 according to the repository's LICENSE file; the README badge says MIT.
What it is
Dita is a 334M parameter diffusion transformer policy pretrained on Open X-Embodiment, from Shanghai AI Laboratory and partners (ICCV 2025). Instead of a small diffusion head on top of a fused embedding, the transformer itself denoises the action chunk in context, attending to raw visual tokens from a DINOv2 encoder and to a frozen CLIP text embedding. The authors call it a vision language action policy, but there is no vision language model inside, which is why it is listed here. Fine-tuned, it reaches 82.4 percent average on LIBERO and an average length of 3.61 on CALVIN ABC to D, and it adapts to a real Franka from 10 demonstrations per task with a single third-person camera.
Architecture
- Backbone
- No VLM backbone. DINOv2 image encoder fine-tuned end to end, a Q-Former with FiLM text injection that compresses each image to 32 tokens, a frozen CLIP text encoder, and a LLaMA2-style causal transformer with 12 blocks of width 768 trained from scratch.
- Action head
- In-context diffusion: the causal transformer denoises the action tokens directly (DDPM training with 1,000 steps, DDIM with 20 steps at evaluation). 7-dimensional end-effector actions.
- Action chunk
- 16 steps per forward pass
- Control rate
- 3 Hz
- Parameters
- 334M total, 221M trainable (paper appendix). LoRA fine-tuning leaves about 11M trainable. Control rate above is for the real Franka.
- Pretraining data
- A subset of Open X-Embodiment selected following the Octo and OpenVLA recipes; a separate checkpoint pretrained on DROID is also released.
- Embodiments
- Franka Emika Panda with Robotiq gripper (real, 10-shot fine-tuning), Google robot (SimplerEnv), LIBERO and CALVIN simulation
What hardware it needs
This is the section most people came for, so it is worth being precise about which numbers are measured and which are arithmetic.
| Precision | Weights | Note |
|---|---|---|
| fp32 | 1.2 GB | Training master weights, rarely used for inference |
| bf16 or fp16 | 0.6 GB | The usual way these checkpoints are served |
| int8 | 0.3 GB | Quantised, expect some loss of precision on fine motions |
| int4 | 0.2 GB | Aggressive quantisation, verify success rate before trusting it |
Smallest real card that fits the bf16 weights plus a 40 percent runtime allowance: 8 GB. That is an estimate for inference, not for fine tuning. Full fine tuning also holds optimiser state and gradients, which typically costs several times the weights unless you use LoRA or a comparable adapter.
| Control rate | 3 Hz |
No memory or latency figure. Pretraining used 32 A100 GPUs at batch size 8,192 for 100,000 steps; real-robot inference ran on one A100.
Reported results
Grouped by what the evaluation actually asked. Values from different suites are not comparable with each other, so each one keeps its suite and split.
Simulation
Reproducible benchmark suites. Everyone runs the same episodes, so the numbers can be compared inside a suite.
- LIBERO average over Spatial, Object, Goal and Long, fine-tuned per suite, third-person camera82.4%success rateAuthors' own evaluation. Per suite: Spatial 84.2, Object 96.3, Goal 85.4, Long 63.8. Octo fine-tuned 75.1 and OpenVLA fine-tuned 76.5 in the same table.source
- CALVIN ABC to D3.61 tasks (out of 5)average length (tasks completed in sequence)Authors' own evaluation without image augmentation; 3.78 with augmentation. Five tasks in a row succeed in 50.0 percent of sequences.source
Real world
No results in this category are published for this model.
Fine tuned tasks
No results in this category are published for this model.
On real hardware
Fine-tuning from 10 demonstrations per task on a Franka Emika Panda with a third-person RealSense camera, covering pick and place, pouring, stacking, rotation, drawer and long-horizon tasks. With full fine-tuning the policy still completed 20 percent of long-horizon tasks under extreme variations, where LoRA fine-tuning failed; trial counts are not stated.
Fine tuning it yourself
Fine-tune from the Open X-Embodiment checkpoint with LoRA or fully; the authors report that full fine-tuning is clearly better for robustness and long tasks. Scripts for 10-shot real-robot fine-tuning, CALVIN and LIBERO are in the repository.
Where it helps, where it does not
Strengths
- Compact at 334M parameters, 221M of them trainable.
- Adapts to a new real setup from 10 demonstrations per task with one third-person camera.
- Open code and checkpoints, including a DROID-pretrained variant.
Limits
- Licence is unclear between the LICENSE file (Apache-2.0) and the README badge (MIT).
- Checkpoints sit on Google Drive rather than on a model hub.
- Real-robot results come as figures without trial counts.
- 3 Hz control on the real robot.
Sources
Everything on this page was taken from these documents. Where they disagree with what you read here, they win.
- https://arxiv.org/abs/2503.19757
- https://arxiv.org/html/2503.19757v2
- https://github.com/RoboDita/Dita
- https://robodita.github.io/
- https://github.com/RoboDita/Dita#model-checkpoints
Entry last checked 2026-11-12.
Try a policy on a real arm
A real SO-100 runs on /live and can be driven from the browser when it is online. Five of the models in this arena can be fine tuned on a dataset you record with your own arm.