V-JEPA 2-AC
Meta, United States 路 June 2025
Meta FAIR
World model 路 Open weights, MIT 路 Also written V-JEPA 2 action-conditioned, vjepa2-ac-vitg
- Parameters
- not published
- No total is published.
- GPU memory
- not published
- weights at bf16, computed
- Inference latency
- 16000 ms
- a single NVIDIA RTX 4090, per action while planning
- Weights
- MIT
- MIT.
What it is
V-JEPA 2-AC is the action-conditioned world model Meta post-trained from V-JEPA 2 for robot planning. A predictor of about 300M parameters learns, on top of the frozen 1B ViT-g encoder, how 7-dimensional end-effector actions change the features of the next frame, from less than 62 hours of unlabeled DROID video. At test time it plans with the cross-entropy method in a model predictive control loop towards an image goal, without task-specific training, reward or data from the test labs. On Franka arms in two labs it averaged 80 percent on pick-and-place with a cup and 65 percent with a box, where Octo reached 15 and 10 percent. Planning takes 16 s per action on one RTX 4090, against 4 minutes for a Cosmos baseline.
Architecture
- Backbone
- Frozen V-JEPA 2 ViT-g/16 video encoder (about 1B parameters). Not a vision language model.
- Action head
- No action head. A block-causal transformer predictor (24 layers, 16 heads, width 1024, about 300M parameters) predicts future features from 7-dimensional end-effector actions, and actions come from planning: the cross-entropy method searches action sequences whose predicted features match a goal image, executes the first action and plans again.
- Parameters
- No total is published. The action-conditioned predictor has about 300M parameters (paper) and runs on the frozen ViT-g encoder, which the README lists at 1B and Hugging Face counts at 1,034,555,264.
- Pretraining data
- Encoder: more than 1 million hours of internet video (V-JEPA 2). Predictor: less than 62 hours of unlabeled robot video from the DROID dataset, without rewards or task labels.
- Embodiments
- Franka arm (DROID setup), deployed in two labs
What hardware it needs
This is the section most people came for, so it is worth being precise about which numbers are measured and which are arithmetic.
| Inference latency, reported | 16000 ms on a single NVIDIA RTX 4090, per action while planning source |
The paper plans on a single NVIDIA RTX 4090 (16 s per action) but publishes no memory figure.
Reported results
Grouped by what the evaluation actually asked. Values from different suites are not comparable with each other, so each one keeps its suite and split.
Simulation
No results in this category are published for this model.
Real world
No results in this category are published for this model.
Fine tuned tasks
What the model reaches after being adapted to a new task or a new robot, usually from a small number of demonstrations. This is the number that matters if you bring your own data.
- Real robot, Franka arms in two labs (V-JEPA 2 paper) pick-and-place cup, zero-shot from image goals, mean of 2 labs with 10 trials each80%success rateAuthors' own evaluation. Lab 1 80, lab 2 80. Octo on the same protocol 15 percent. A Cosmos world model planner reached 0 percent in lab 2.source
- Real robot, Franka arms in two labs (V-JEPA 2 paper) pick-and-place box, zero-shot from image goals, mean of 2 labs with 10 trials each65%success rateAuthors' own evaluation. Lab 1 80, lab 2 50. Octo 10 percent. A Cosmos world model planner reached 0 percent in lab 2.source
- Real robot, Franka arms in two labs (V-JEPA 2 paper) grasp cup, zero-shot from image goals, mean of 2 labs with 10 trials each65%success rateAuthors' own evaluation. Lab 1 70, lab 2 60. Octo 15 percent. A Cosmos world model planner reached 0 percent in lab 2.source
- Real robot, Franka arms in two labs (V-JEPA 2 paper) grasp box, zero-shot from image goals, mean of 2 labs with 10 trials each25%success rateAuthors' own evaluation. Lab 1 30, lab 2 20. Octo 0 percent. A Cosmos world model planner reached 20 percent in lab 2.source
On real hardware
Zero-shot on Franka arms in two labs, 10 trials per task and lab, planning from image goals. Averages: reach 100, grasp cup 65, grasp box 25, reach with object 75 (cup) and 75 (box), pick-and-place cup 80, pick-and-place box 65 percent, against 100, 15, 0, 15, 70, 15 and 10 percent for Octo. In lab 2 a Cosmos world model planner reached 80, 0, 20, 0 and 0 percent on reach, grasp cup, grasp box and the two pick-and-place tasks.
Fine tuning it yourself
Post-training starts from the frozen pretrained ViT-g encoder with the DROID config in the repository (configs/train/vitg16/droid-256px-8f.yaml). The reported tasks need no task-specific fine-tuning; goals are given as images, not as language.
Where it helps, where it does not
Strengths
- Plans pick-and-place in labs it never saw data from: 80 percent with a cup and 65 percent with a box, against 15 and 10 percent for Octo on the same protocol.
- Needs little robot data: under 62 hours of unlabeled video and no demonstrations of the tasks.
- Planning is 15 times faster than with a Cosmos video world model on the same GPU: 16 s against 4 minutes per action.
- Open code and checkpoint under MIT.
Limits
- Slow: 16 s of planning per action on an RTX 4090.
- Goals must be images, not language instructions.
- Grasping a box stays weak at 25 percent on average.
- Evaluated on one robot type (Franka) with 10 trials per task and lab.
Sources
Everything on this page was taken from these documents. Where they disagree with what you read here, they win.
- https://arxiv.org/abs/2506.09985
- https://arxiv.org/html/2506.09985v1
- https://github.com/facebookresearch/vjepa2
- https://ai.meta.com/blog/v-jepa-2-world-model-benchmarks/
- https://github.com/facebookresearch/vjepa2#v-jepa-2-ac
Entry last checked 2026-11-12.
Try a policy on a real arm
A real SO-100 runs on /live and can be driven from the browser when it is online. Five of the models in this arena can be fine tuned on a dataset you record with your own arm.