MolmoAct2
Allen Institute for AI, USA · May 2026
Allen Institute for AI (Ai2)
- Parameters
- 5.5 B
- The allenai/MolmoAct2 checkpoint reports 5,485,309,424 total parameters (5,442,196,272 in F32) in its safetensors index, and...
- GPU memory
- 10 GB
- weights at bf16, computed. Fits a 16 GB card
- Inference latency
- 180 ms
- Single NVIDIA H100, LIBERO, action horizon 10, with CUDA Graph replay enabled
- Weights
- Apache-2.0
- downloadable checkpoint
What it is
MolmoAct2 is Ai2's May 2026 rebuild of the Action Reasoning Model line, aimed squarely at deployment rather than benchmarks. The architecture grafts a discrete-token VLM into a DiT-style flow matching action expert with per-layer KV conditioning, so each layer of the action expert is conditioned on the keys and values of the corresponding VLM layer. The backbone is Molmo2-ER, a 4.85B embodied-reasoning VLM trained on a 3.3M-sample corpus, which scores 63.8 average across 13 embodied reasoning benchmarks and beats GPT-5 (57.9) and Gemini Robotics-ER 1.5 Thinking (61.3). Measured on a single H100 with action horizon 10, MolmoAct2 reaches 55.79 Hz amortized control rate with CUDA Graph replay, about 180 ms per action against 6,700 ms for the original MolmoAct. Ai2 also released the MolmoAct2-BimanualYAM Dataset with 34.5k demonstrations totalling over 720 hours, the largest open bimanual dataset to date. Action chunk length is not fixed but derived from the source dataset control frequency, since one chunk always corresponds to one second of robot motion.
Architecture
- Backbone
- Molmo2-ER, an embodied-reasoning VLM built on Molmo2-4B (4,850,869,200 parameters, Apache-2.0)
- Action head
- DiT-style continuous action expert trained with a flow matching objective, attached by per-layer KV conditioning: keys and values from each VLM layer are reused as the key and value inputs to the corresponding action expert cross-attention layer. Ai2 also released MolmoAct2-FAST-Tokenizer, an open reimplementation of Physical Intelligence's FAST action tokenizer, used in the discrete pre-training stage.
- Action chunk
- 10 steps per forward pass
- Parameters
- The allenai/MolmoAct2 checkpoint reports 5,485,309,424 total parameters (5,442,196,272 in F32) in its safetensors index, and the Hugging Face collection labels it 5B (checked 11.08.2026). The Molmo2-ER backbone alone reports 4,850,869,200 parameters. Checkpoints are approximately 22 GB each.
- Pretraining data
- 3.3M-sample embodied reasoning corpus across six capability pillars for the Molmo2-ER backbone, trained with a specialize-then-rehearse recipe (components include Video Pointing 370K and Video Embodied QA 703k). Action pre-training uses millions of action sequences across five embodiments. Three action datasets were released: MolmoAct2-BimanualYAM Dataset with 34.5k robot demonstrations totalling over 720 hours collected over two months, plus SO-100/101 and DROID datasets, with a broader corpus of 38,059 robot datasets curated from 1,222 public LeRobot datasets contributed by 377 users. Data collection was supported by Cortex AI.
- Embodiments
- Bimanual YAM arms (720-hour dataset, fine-tuned checkpoint MolmoAct2-BimanualYAM), Franka Emika Panda via DROID with absolute joint-pose control (MolmoAct2-DROID), SO-100/101 single-arm (MolmoAct2-SO100_101), Bimanual Franka Emika Panda in RoboEval, Franka Panda in LIBERO simulation
What hardware it needs
This is the section most people came for, so it is worth being precise about which numbers are measured and which are arithmetic.
| Precision | Weights | Note |
|---|---|---|
| fp32 | 20 GB | Training master weights, rarely used for inference |
| bf16 or fp16 | 10 GB | The usual way these checkpoints are served |
| int8 | 5.1 GB | Quantised, expect some loss of precision on fine motions |
| int4 | 2.6 GB | Aggressive quantisation, verify success rate before trusting it |
Smallest real card that fits the bf16 weights plus a 40 percent runtime allowance: 16 GB. That is an estimate for inference, not for fine tuning. Full fine tuning also holds optimiser state and gradients, which typically costs several times the weights unless you use LoRA or a comparable adapter.
| Inference latency, reported | 180 ms on Single NVIDIA H100, LIBERO, action horizon 10, with CUDA Graph replay enabled source |
| Checkpoint download | 22 GB |
bfloat16 inference of the YAM checkpoint stays under 16 GB VRAM, so a single 16 GB to 24 GB card is workable. float32 for the DROID checkpoint needs roughly 96 GB of free VRAM. Inference has also been validated on Intel XPU without code changes. Pre-training used 2 nodes times 8 H100 GPUs at a global batch size of 64 for the stage documented in the paper.
Reported results
Grouped by what the evaluation actually asked. Values from different suites are not comparable with each other, so each one keeps its suite and split.
Simulation
Reproducible benchmark suites. Everyone runs the same episodes, so the numbers can be compared inside a suite.
- LIBERO Object100%
- LIBERO Average over 4 suites97.2%success ratePaper Table 8, highest in table. Per split: Spatial 97.8, Object 100.0, Goal 97.8, Long 93.2. Improves 10.6 points over MolmoAct-7B-D.source
Real world
Rollouts on physical hardware. The setups differ, so read these as evidence, not as a ranking.
- RoboEval average across 8 bimanual Franka tasks44.3%success ratePaper Section 5. Surpasses baselines but shows bimanual coordination remains the hard case.source
Fine tuned tasks
What the model reaches after being adapted to a new task or a new robot, usually from a small number of demonstrations. This is the number that matters if you bring your own data.
- Real-world zero-shot (Franka DROID setup) 5 tasks, 15 trials per task, no per-task fine-tuning87.1%average success ratePaper Section 5, MolmoAct2-DROID. Per task: apple-on-plate 100.0, pipette-in-tray 86.7, red-cube-in-tape-roll 93.3, knife-in-box 93.3, multi-object bowl 62.0. 38.7 points above the runner-up MolmoBot.source
- Real-world zero-shot (SO-100/101) second zero-shot embodiment56.7%success ratePaper Section 5, MolmoAct2-SO100/101. An 11.4 point gain over the prior best on this embodiment, and far below the Franka DROID result, so low-cost arms remain harder.source
Embodied reasoning
Offline benchmarks that test understanding rather than control. No robot moves in these, so a high score says nothing about grasping.
- Embodied reasoning suite (13 benchmarks including BLINK and SAT) average, backbone only63.8 scoreaverage scorePaper table for Molmo2-ER, the VLM backbone rather than the full policy. Beats GPT-5 (57.9), Gemini Robotics-ER 1.5 Thinking (61.3), Gemini 2.5 Pro (57.1) and Qwen3-VL-8B (61.0).source
On real hardware
Zero-shot real-world evaluation on a Franka DROID setup across 5 tasks with 15 trials each averages 87.1 percent: apple-on-plate 100.0, pipette-in-tray 86.7, red-cube-in-tape-roll 93.3, knife-in-box 93.3, multi-object bowl 62.0. This is 38.7 percent above the runner-up MolmoBot. On the second zero-shot embodiment, SO-100/101, MolmoAct2-SO100/101 reaches 56.7 percent, an 11.4 percent gain. A third-party evaluation by Cortex AI scores MolmoAct2 at 0.51 average and ranks it first on 7 of 8 tasks.
Fine tuning it yourself
Fine-tuned checkpoints are published per embodiment (DROID, BimanualYAM, SO100_101, LIBERO), and the base MolmoAct2 checkpoint is explicitly described as a foundation checkpoint for further robot fine-tuning rather than a ready-to-run policy, so embodiment-specific adaptation is required before deployment. Datasets are released in LeRobot v3.0 format with language annotations. Action chunk length follows the source dataset control frequency because one chunk equals one second of motion; LIBERO uses relative end-effector control at 10 Hz and therefore a 10-step chunk. Each action is padded to 32 dimensions. Serving is via HTTP servers (host_server_droid.py, host_server_yam.py) that return actions plus dt_ms.
Where it helps, where it does not
Strengths
- Roughly 37 times faster than the original MolmoAct, about 180 ms per action versus 6,700 ms on the same H100 setup, which moves it from demo-only into deployable territory
- Runs on affordable hardware: bfloat16 inference under 16 GB VRAM, and validated on Intel XPU as well as CUDA
- Highest LIBERO average among compared methods at 97.2 percent, with 100 percent on LIBERO-Object
- 87.1 percent zero-shot real-world success on a Franka DROID setup without per-task fine-tuning, 38.7 percent above the runner-up
- Molmo2-ER backbone beats GPT-5 and Gemini Robotics-ER 1.5 Thinking on a 13-benchmark embodied reasoning average (63.8 versus 57.9 and 61.3)
- 720-hour bimanual YAM dataset is the largest open bimanual tabletop manipulation dataset released, roughly 30 times the robot data used for MolmoAct
- Fully open under Apache-2.0: weights, code, datasets, plus an open reimplementation of the FAST action tokenizer
- Independently evaluated by Cortex AI rather than self-reported only
Limits
- The base checkpoint is a foundation model, not a ready-to-run policy, and requires embodiment-specific fine-tuning before deployment
- The multi-object bowl task drops to 62.0 percent, well below the other four real-world tasks, so cluttered multi-object scenes remain the failure mode
- bfloat16 loading is not officially supported by the upstream modeling code; the repository applies two runtime patches to modeling_molmoact2.py at startup to make it work
- float32 inference for the DROID checkpoint needs roughly 96 GB of free VRAM, so the cheap path depends on the bfloat16 patches
- Action chunk length is not a free parameter but is tied to the source dataset control frequency, which constrains how datasets can be mixed
- MolmoAct2-Think is more accurate on LIBERO (98.1 versus 97.2) but roughly 4.4 times slower, so accuracy and latency cannot both be maximized
- RoboEval success rate of 44.3 percent shows bimanual coordination is still far from solved
Sources
Everything on this page was taken from these documents. Where they disagree with what you read here, they win.
- https://arxiv.org/abs/2605.02881
- https://arxiv.org/pdf/2605.02881v2
- https://allenai.org/blog/molmoact2
- https://github.com/allenai/molmoact2
- https://raw.githubusercontent.com/allenai/molmoact2/main/README.md
- https://huggingface.co/allenai/MolmoAct2
- https://huggingface.co/allenai/Molmo2-ER
- https://huggingface.co/collections/allenai/molmoact2-models
Entry last checked 2026-08-11.
Try a policy on a real arm
A physical SO-100 is online and free to drive in the browser, and five of the models in this arena can be fine tuned on a dataset you record with your own.