RoboMamba
Peking University, China · June 2024
Peking University, with AI2Robotics
- Parameters
- 3.2 B
- The paper states 'RoboMamba, with only 3.2B parameters'.
- GPU memory
- 6.0 GB
- weights at bf16, computed. Fits a 12 GB card
- Inference latency
- not published
- per action step
- Weights
- closed
- MIT for the GitHub repository lmzpai/roboMamba.
What it is
RoboMamba is a NeurIPS 2024 vision language action model that replaces the transformer language backbone with a Mamba state space model, giving linear rather than quadratic inference complexity. A CLIP or SigLIP ViT-Large encoder is aligned with a 2.8B Mamba model through LLaVA style co-training, after which a 3.7M parameter head is trained for a few dozen minutes on one A100 to predict SE(3) end effector poses. It reports 63 percent on seen and 53 percent on unseen SAPIEN object categories along with competitive general VQA scores, so it functions as a reasoning model as much as a policy. Two things limit its usefulness for a deployment decision: it emits a single pose rather than an action chunk, and its entire efficiency claim is relative (3x and 7x faster) with no absolute latency number published. Checkpoints sit in a test branch and the training code is only released on email request.
Architecture
- Backbone
- CLIP or SigLIP ViT-Large vision encoder combined with a Mamba state space language model at 2.8B parameters (the abstract cites 2.7B), with a 1.4B alternative. Input resolution 224 or 336 pixels.
- Action head
- simple MLP policy head predicting a single SE(3) end effector pose rather than a chunk of continuous actions
- Parameters
- The paper states 'RoboMamba, with only 3.2B parameters'. The language model is described as the 2.8/1.4B Mamba model in the method section while the abstract says 2.7B, so the paper is internally inconsistent about this figure. The manipulation policy head adds only 3.7M parameters, about 0.1 percent of the model and roughly 7 MB on disk.
- Pretraining data
- Aligned and co-trained on general vision language data following the LLaVA recipe (one epoch alignment plus two epochs instruction co-training), then eight epochs of manipulation fine-tuning. The paper reports no robot demonstration count for a pretraining corpus, which is a meaningful gap when comparing against Open X-Embodiment trained policies.
- Embodiments
- SAPIEN articulated object manipulation (simulation), Franka Emika Panda with suction gripper (real)
What hardware it needs
This is the section most people came for, so it is worth being precise about which numbers are measured and which are arithmetic.
| Precision | Weights | Note |
|---|---|---|
| fp32 | 12 GB | Training master weights, rarely used for inference |
| bf16 or fp16 | 6.0 GB | The usual way these checkpoints are served |
| int8 | 3.0 GB | Quantised, expect some loss of precision on fine motions |
| int4 | 1.5 GB | Aggressive quantisation, verify success rate before trusting it |
Smallest real card that fits the bf16 weights plus a 40 percent runtime allowance: 12 GB. That is an estimate for inference, not for fine tuning. Full fine tuning also holds optimiser state and gradients, which typically costs several times the weights unless you use LoRA or a comparable adapter.
Fine-tuning the manipulation head takes 'a few dozen minutes' on a single A100 and updates only 3.7M parameters. No VRAM figure and no absolute inference latency is published. The paper reports only relative speedups on an A100 without any baseline timing, so no per step latency can be derived.
The authors publish no memory or latency figure for this model. Everything above is computed from the parameter count.
Reported results
Grouped by what the evaluation actually asked. Values from different suites are not comparable with each other, so each one keeps its suite and split.
Simulation
Reproducible benchmark suites. Everyone runs the same episodes, so the numbers can be compared inside a suite.
- SAPIEN manipulation seen object categories (20)63%success rateAuthors' own evaluation. ManipLLM scored 56 in the same table. Per category results range from 26 to 91 percent.source
- SAPIEN manipulation unseen object categories (10)53%success rateAuthors' own evaluation. ManipLLM scored 51. Per category results range from 19 to 93 percent.source
Real world
No results in this category are published for this model.
Fine tuned tasks
No results in this category are published for this model.
Embodied reasoning
Offline benchmarks that test understanding rather than control. No robot moves in these, so a high score says nothing about grasping.
- VQAv2 general visual question answering79.6%accuracyAuthors' own evaluation, 2.7B model. Related scores: OKVQA 63.3, GQA 64.2, VizWiz 57.1, POPE 86.3, MME 1297.2, MMBench 60.9, MM-Vet 29.4.source
- RoboVQA robotic reasoning42.8 scoreBLEU-4Authors' own evaluation, 2.7B model. BLEU-1 was 62.7. This is a language benchmark, not a manipulation benchmark.source
On real hardware
Qualitative demonstrations on a Franka Emika Panda with a suction gripper, covering task planning, long horizon planning, affordance generation and discrimination, past and future action prediction and 6-DoF pose prediction on household objects. The paper publishes no real robot success rate table, so there is no measured real world number to compare against other models here.
Fine tuning it yourself
Manipulation is learned by attaching a 3.7M parameter policy head and training it for eight epochs, which the authors report takes a few dozen minutes on a single A100. Only the head is updated, about 0.1 percent of parameters, and the resulting artefact is about 7 MB. The training code is not in the public repository and must be requested from the authors by email.
Where it helps, where it does not
Strengths
- Linear inference complexity from the Mamba state space backbone rather than quadratic attention, which is the structural argument for its speed claim.
- Manipulation skill is acquired by training a 3.7M parameter head for a few dozen minutes on one A100, the cheapest adaptation path in this comparison.
- Strong general and robotic reasoning scores for its size (VQAv2 79.6, POPE 86.3, RoboVQA BLEU-4 42.8), so it can serve as a planner as well as a policy.
- Small gap between seen and unseen object categories in SAPIEN (63 against 53 percent) suggests the reasoning pretraining does transfer.
- MIT licensed repository.
Limits
- The efficiency claim is purely relative. The paper says '3 times faster than previous robotic VLA models' and '7 times faster than LLaMA-AdapterV2 and ManipLLM' but gives no absolute millisecond or Hz figure anywhere, so the speed cannot be compared against any other model on this page.
- It predicts a single SE(3) pose rather than a continuous action chunk, so it is not directly comparable to closed loop policies like pi0 or SmolVLA and cannot simply be dropped into a 30 Hz control loop.
- No LIBERO, SimplerEnv, Meta-World or CALVIN results, which makes cross model comparison with the rest of this field impossible.
- Real robot evidence is qualitative only, with no success rates reported.
- Training code is released only on email request and no checkpoint is published on Hugging Face, so reproduction is gated.
- The paper is internally inconsistent about the language model size, citing 2.7B in the abstract and 2.8B in the method section.
- No update since the December 2024 revision, making it the oldest model in this comparison.
Sources
Everything on this page was taken from these documents. Where they disagree with what you read here, they win.
- https://arxiv.org/abs/2406.04339
- https://arxiv.org/html/2406.04339v2
- https://github.com/lmzpai/roboMamba
- https://neurips.cc/virtual/2024/poster/95690
- https://openreview.net/forum?id=JxOQeg1NkH
- https://sites.google.com/view/robomamba-web
Entry last checked 2026-08-11.
Try a policy on a real arm
A physical SO-100 is online and free to drive in the browser, and five of the models in this arena can be fine tuned on a dataset you record with your own.