V-JEPA 2-AC

Meta, United States 路 June 2025

Meta FAIR

World model 路 Open weights, MIT 路 Also written V-JEPA 2 action-conditioned, vjepa2-ac-vitg

Parameters
not published
No total is published.
GPU memory
not published
weights at bf16, computed
Inference latency
16000 ms
a single NVIDIA RTX 4090, per action while planning
Weights
MIT
MIT.

What it is

V-JEPA 2-AC is the action-conditioned world model Meta post-trained from V-JEPA 2 for robot planning. A predictor of about 300M parameters learns, on top of the frozen 1B ViT-g encoder, how 7-dimensional end-effector actions change the features of the next frame, from less than 62 hours of unlabeled DROID video. At test time it plans with the cross-entropy method in a model predictive control loop towards an image goal, without task-specific training, reward or data from the test labs. On Franka arms in two labs it averaged 80 percent on pick-and-place with a cup and 65 percent with a box, where Octo reached 15 and 10 percent. Planning takes 16 s per action on one RTX 4090, against 4 minutes for a Cosmos baseline.

Architecture

Backbone
Frozen V-JEPA 2 ViT-g/16 video encoder (about 1B parameters). Not a vision language model.
Action head
No action head. A block-causal transformer predictor (24 layers, 16 heads, width 1024, about 300M parameters) predicts future features from 7-dimensional end-effector actions, and actions come from planning: the cross-entropy method searches action sequences whose predicted features match a goal image, executes the first action and plans again.
Parameters
No total is published. The action-conditioned predictor has about 300M parameters (paper) and runs on the frozen ViT-g encoder, which the README lists at 1B and Hugging Face counts at 1,034,555,264.
Pretraining data
Encoder: more than 1 million hours of internet video (V-JEPA 2). Predictor: less than 62 hours of unlabeled robot video from the DROID dataset, without rewards or task labels.
Embodiments
Franka arm (DROID setup), deployed in two labs

What hardware it needs

This is the section most people came for, so it is worth being precise about which numbers are measured and which are arithmetic.

Published figures. These come from the model authors, not from this site.
Inference latency, reported16000 ms on a single NVIDIA RTX 4090, per action while planning source

The paper plans on a single NVIDIA RTX 4090 (16 s per action) but publishes no memory figure.

Reported results

Grouped by what the evaluation actually asked. Values from different suites are not comparable with each other, so each one keeps its suite and split.

Simulation

No results in this category are published for this model.

Real world

No results in this category are published for this model.

Fine tuned tasks

What the model reaches after being adapted to a new task or a new robot, usually from a small number of demonstrations. This is the number that matters if you bring your own data.

  • Real robot, Franka arms in two labs (V-JEPA 2 paper) pick-and-place cup, zero-shot from image goals, mean of 2 labs with 10 trials each
    80%
    success rateAuthors' own evaluation. Lab 1 80, lab 2 80. Octo on the same protocol 15 percent. A Cosmos world model planner reached 0 percent in lab 2.source
  • Real robot, Franka arms in two labs (V-JEPA 2 paper) pick-and-place box, zero-shot from image goals, mean of 2 labs with 10 trials each
    65%
    success rateAuthors' own evaluation. Lab 1 80, lab 2 50. Octo 10 percent. A Cosmos world model planner reached 0 percent in lab 2.source
  • Real robot, Franka arms in two labs (V-JEPA 2 paper) grasp cup, zero-shot from image goals, mean of 2 labs with 10 trials each
    65%
    success rateAuthors' own evaluation. Lab 1 70, lab 2 60. Octo 15 percent. A Cosmos world model planner reached 0 percent in lab 2.source
  • Real robot, Franka arms in two labs (V-JEPA 2 paper) grasp box, zero-shot from image goals, mean of 2 labs with 10 trials each
    25%
    success rateAuthors' own evaluation. Lab 1 30, lab 2 20. Octo 0 percent. A Cosmos world model planner reached 20 percent in lab 2.source

On real hardware

Zero-shot on Franka arms in two labs, 10 trials per task and lab, planning from image goals. Averages: reach 100, grasp cup 65, grasp box 25, reach with object 75 (cup) and 75 (box), pick-and-place cup 80, pick-and-place box 65 percent, against 100, 15, 0, 15, 70, 15 and 10 percent for Octo. In lab 2 a Cosmos world model planner reached 80, 0, 20, 0 and 0 percent on reach, grasp cup, grasp box and the two pick-and-place tasks.

Fine tuning it yourself

Post-training starts from the frozen pretrained ViT-g encoder with the DROID config in the repository (configs/train/vitg16/droid-256px-8f.yaml). The reported tasks need no task-specific fine-tuning; goals are given as images, not as language.

Where it helps, where it does not

Strengths

  • Plans pick-and-place in labs it never saw data from: 80 percent with a cup and 65 percent with a box, against 15 and 10 percent for Octo on the same protocol.
  • Needs little robot data: under 62 hours of unlabeled video and no demonstrations of the tasks.
  • Planning is 15 times faster than with a Cosmos video world model on the same GPU: 16 s against 4 minutes per action.
  • Open code and checkpoint under MIT.

Limits

  • Slow: 16 s of planning per action on an RTX 4090.
  • Goals must be images, not language instructions.
  • Grasping a box stays weak at 25 percent on average.
  • Evaluated on one robot type (Franka) with 10 trials per task and lab.

Sources

Everything on this page was taken from these documents. Where they disagree with what you read here, they win.

Entry last checked 2026-11-12.

Try a policy on a real arm

A real SO-100 runs on /live and can be driven from the browser when it is online. Five of the models in this arena can be fine tuned on a dataset you record with your own arm.