UniVLA (task-centric latent actions)
OpenDriveLab, China · May 2025
- Parameters
- 7.5 B
- Built on the Prismatic-7B VLM (TRI-ML prism-dinosiglip-224px+7b).
- GPU memory
- 14 GB
- weights at bf16, computed. Fits a 24 GB card
- Inference latency
- not published
- per action step
- Weights
- Apache-2.0
- downloadable checkpoint
What it is
UniVLA from OpenDriveLab attacks the cost side of VLA training rather than the accuracy side. Its mechanism is a latent action model built in DINO feature space and conditioned on language, which extracts task-centric latent actions from ordinary video while suppressing task-irrelevant motion, so that internet video and human video become usable training data. A Prismatic-7B backbone predicts in that discrete latent space (codebook size 16), and a per-embodiment decoder of only about 12M parameters converts latent actions into robot commands. The economics are the headline: about 960 A100-hours of pretraining against OpenVLA's 21500, with 95.2 percent LIBERO average and 92.0 percent on LIBERO-Long. The clearest weakness is SimplerEnv WidowX at 47.9 percent overall, where Stack Block reaches only 2.8 percent success despite a 66.7 percent grasp rate, which shows placement precision is not solved by latent actions.
Architecture
- Backbone
- Prismatic-7B (TRI-ML prism-dinosiglip-224px+7b): fused SigLIP and DINOv2 visual encoder, projection layer and a LLaMA-2 language model
- Action head
- Two-stage latent action design. A latent action model in DINO feature space, conditioned on language to suppress task-irrelevant dynamics, produces a discrete task-centric latent action space with a codebook of size 16. A lightweight per-embodiment action decoder of about 12M parameters maps latent actions to robot commands.
- Parameters
- Built on the Prismatic-7B VLM (TRI-ML prism-dinosiglip-224px+7b). The action decoder is extremely lightweight at roughly 12M parameters (10.8M to 12.6M depending on configuration), and with LoRA rank 32 the total trainable parameter count is about 123M. So the 7B is the frozen-plus-LoRA backbone, not the trained policy.
- Pretraining data
- Internet-scale video across embodiments and viewpoints: Bridge-V2, manipulation and navigation subsets of Open X-Embodiment, GNM navigation data, and Ego4D human videos. Separate checkpoints exist for Bridge-only and human-video-only pretraining so the contribution of each source can be isolated.
- Embodiments
- manipulation arms (LIBERO, CALVIN, SimplerEnv WidowX, real-robot setup), navigation agents (R2R VLN-CE), human video (Ego4D) as a training source
What hardware it needs
This is the section most people came for, so it is worth being precise about which numbers are measured and which are arithmetic.
| Precision | Weights | Note |
|---|---|---|
| fp32 | 28 GB | Training master weights, rarely used for inference |
| bf16 or fp16 | 14 GB | The usual way these checkpoints are served |
| int8 | 7.0 GB | Quantised, expect some loss of precision on fine motions |
| int4 | 3.5 GB | Aggressive quantisation, verify success rate before trusting it |
Smallest real card that fits the bf16 weights plus a 40 percent runtime allowance: 24 GB. That is an estimate for inference, not for fine tuning. Full fine tuning also holds optimiser state and gradients, which typically costs several times the weights unless you use LoRA or a comparable adapter.
No inference VRAM figure is published. Training used a 32-GPU A100 cluster. The relevant practical fact is that adapting to a new embodiment trains only the roughly 12M parameter action decoder plus LoRA rank 32, about 123M trainable parameters, rather than the 7B backbone.
The authors publish no memory or latency figure for this model. Everything above is computed from the parameter count.
Reported results
Grouped by what the evaluation actually asked. Values from different suites are not comparable with each other, so each one keeps its suite and split.
Simulation
Reproducible benchmark suites. Everyone runs the same episodes, so the numbers can be compared inside a suite.
- LIBERO average over Spatial, Object, Goal and Long95.2%success rateAuthors' own evaluation, reported as 95.2 +/- 0.3 percent. Per-suite: Spatial 96.5, Object 96.8, Goal 95.6, Long 92.0. OpenVLA scores 76.5 percent average on the same protocol.source
- LIBERO Long suite, 10 percent of the training data62.4%success rateAuthors' own evaluation. OpenVLA scores 11.6 percent and OpenVLA-OFT 43.0 percent at the same data budget. This is the strongest evidence for the data-efficiency claim.source
- SimplerEnv (WidowX / Bridge) overall average across four tasks, three seeds47.9%success rateAuthors' own evaluation, 47.9 +/- 1.0 percent. Best prior baseline RoboVLM scores 31.3 percent. Stack Block is only 2.8 percent despite a 66.7 percent grasp rate, which is the model's clearest failure mode.source
Real world
Rollouts on physical hardware. The setups differ, so read these as evidence, not as a ranking.
- Real robot (UniVLA paper setup) average over four manipulation tasks75%success rateAuthors' own evaluation. Average step-wise score 2.49 against 1.36 for LAPA and 0.98 for OpenVLA.source
- R2R VLN-CE (vision-and-language navigation) validation, oracle success47.1%oracle success rateAuthors' own evaluation. OpenVLA scores 17.5 percent. Demonstrates the framework transfers beyond manipulation.source
Fine tuned tasks
No results in this category are published for this model.
On real hardware
75.0 percent average success rate across four real manipulation tasks with an average step-wise score of 2.49, against 1.36 for LAPA and 0.98 for OpenVLA, which the authors report as a 36.7 percent improvement over the prior best.
Fine tuning it yourself
This is the model's strongest practical property. Full-scale pretraining took about 960 A100 GPU-hours, which the authors put at 5 percent of OpenVLA's 21500 A100-hours, and the Bridge and human-video subsets took only 200 GPU-hours. Downstream adaptation trains the roughly 12M parameter action decoder with LoRA rank 32 on the backbone, about 123M trainable parameters. Eight checkpoints are released covering the latent action model, three pretraining variants and four task-specific fine-tunes (LIBERO, CALVIN, R2R, SimplerEnv-Bridge). Apache-2.0, commercially usable.
Where it helps, where it does not
Strengths
- Pretraining cost of about 960 A100-hours against OpenVLA's 21500, roughly 1/20, with published GPU-hour figures rather than a vague efficiency claim
- Adapting to a new embodiment trains only about 12M action-decoder parameters plus LoRA, so downstream cost is low even though the backbone is 7B
- Best LIBERO-Long result in this comparison at 92.0 percent, and it degrades gracefully under data scarcity (62.4 percent on LIBERO-Long with only 10 percent of the data, against 11.6 percent for OpenVLA)
- Covers both manipulation and navigation with one framework, and reports R2R VLN-CE oracle success of 47.1 percent against 17.5 percent for OpenVLA
- Apache-2.0 with eight released checkpoints including separate Bridge-only and human-video-only pretraining variants, which makes the data ablation reproducible
- Learns from action-free human video (Ego4D), and the human-only pretraining variant still reaches 88.7 percent LIBERO average
Limits
- SimplerEnv WidowX overall average is only 47.9 percent, and the Stack Block subtask collapses to 2.8 percent despite a 66.7 percent grasp rate, so precise placement after a successful grasp is a clear failure mode
- No published VRAM, latency, control frequency or action chunk size
- The 7B Prismatic backbone must still be loaded at inference even though only 123M parameters are trained, so the memory footprint is that of a 7B model
- Real-robot evaluation covers four tasks on one setup, which is thin next to the simulation coverage
- Repository news stops at May 2025 with UniVLA v1.0 and no later update was found, so the project may be dormant
- Name collision: a different model also called UniVLA was published by BAAI in June 2025, and benchmark numbers for the two are routinely confused
Sources
Everything on this page was taken from these documents. Where they disagree with what you read here, they win.
- https://arxiv.org/abs/2505.06111
- https://arxiv.org/html/2505.06111v3
- https://github.com/OpenDriveLab/UniVLA
- https://raw.githubusercontent.com/OpenDriveLab/UniVLA/main/README.md
- https://huggingface.co/qwbu/univla-7b
Entry last checked 2026-08-11.
Try a policy on a real arm
A physical SO-100 is online and free to drive in the browser, and five of the models in this arena can be fine tuned on a dataset you record with your own.