SAI

SpatialVLA

Shanghai AI Laboratory, China · January 2025

Shanghai AI Laboratory and Fudan University

Foundation VLAOpen weights, MITAlso written spatialvla-4b-224-pt, spatialvla-4b-mix-224-pt
Parameters
3.5 B
The official Hugging Face model card lists 4B parameters in BF16, built on google/paligemma2-3b-pt-224.
GPU memory
8.5 GB
reported by the authors
Inference latency
not published
per action step
Weights
MIT
downloadable checkpoint

What it is

SpatialVLA is a 4B parameter generalist policy built on PaliGemma2-3B and pretrained on 1.1 million real robot episodes from Open X-Embodiment and RH20T. Its two contributions are Ego3D Position Encoding, which injects 3D information into the visual input, and Adaptive Action Grids, which discretise the action space with finer resolution where motions are frequent and can be re-discretised to fit a new robot. It reaches 71.9 percent zero-shot on SimplerEnv Google Robot visual matching and 78.1 percent on LIBERO after fine-tuning, but only 34.4 percent zero-shot on WidowX Bridge, so its strength is tied to how close the target arm is to the pretraining distribution. The model card states plainly that it does not zero-shot generalise to new embodiments, and an independent evaluation by the NORA authors measured it at 11.1 percent on their own WidowX arm. It is one of the few models here with a documented VRAM figure, 8.5 GB for inference.

Architecture

Backbone
PaliGemma2-3B-pt-224 (google/paligemma2-3b-pt-224)
Action head
autoregressive discrete action tokens over Adaptive Action Grids, which statistically discretise the action space and allocate finer resolution to frequently occurring motions. Ego3D Position Encoding injects 3D information into the visual input.
Control rate
20 Hz
Parameters
The official Hugging Face model card lists 4B parameters in BF16, built on google/paligemma2-3b-pt-224. Released variants are spatialvla-4b-224-pt (base pretrained), spatialvla-4b-mix-224-pt and spatialvla-4b-224-sft-bridge, plus community adapters. Accepted at RSS 2025.
Pretraining data
1.1 million real world robot episodes from Open X-Embodiment and RH20T. Training used 64 A100 GPUs for approximately 10 days at batch size 2048.
Embodiments
Google Robot (SimplerEnv), WidowX / BridgeData V2, LIBERO (simulation), cross embodiment from Open X-Embodiment and RH20T

What hardware it needs

This is the section most people came for, so it is worth being precise about which numbers are measured and which are arithmetic.

Computed from 3.5 B parameters. Weights only, so treat it as a floor: activations, image encoder intermediates and the CUDA context come on top, in practice 30 to 50 percent more for inference.
PrecisionWeightsNote
fp3213 GBTraining master weights, rarely used for inference
bf16 or fp166.5 GBThe usual way these checkpoints are served
int83.3 GBQuantised, expect some loss of precision on fine motions
int41.6 GBAggressive quantisation, verify success rate before trusting it

Smallest real card that fits the bf16 weights plus a 40 percent runtime allowance: 12 GB. That is an estimate for inference, not for fine tuning. Full fine tuning also holds optimiser state and gradients, which typically costs several times the weights unless you use LoRA or a comparable adapter.

Published figures. These come from the model authors, not from this site.
Inference memory, reported8.5 GB source
Control rate20 Hz

The authors run inference on a single NVIDIA RTX 4090 at about 8.5 GB and roughly 20 Hz.

Reported results

Grouped by what the evaluation actually asked. Values from different suites are not comparable with each other, so each one keeps its suite and split.

Simulation

Reproducible benchmark suites. Everyone runs the same episodes, so the numbers can be compared inside a suite.

  • LIBERO average over spatial, object, goal and long, fine-tuned
    78.1%
    success rateAuthors' own evaluation. Per suite: spatial 88.2, object 89.9, goal 78.6, long 55.5.source
  • SimplerEnv Google Robot, visual matching, zero-shot
    71.9%
    success rateAuthors' own evaluation. Per task: pick coke can 81.0, move near 69.6, open/close drawer 59.3. Rises to 75.1 after fine-tuning.source
  • SimplerEnv Google Robot, variant aggregation, zero-shot
    68.8%
    success rateAuthors' own evaluation. The visually shifted split. Drawer manipulation falls hardest, from 59.3 to 36.2.source
  • LIBERO long, fine-tuned
    55.5%
    success rateAuthors' own evaluation. Weakest suite by a wide margin.source
  • SimplerEnv WidowX Bridge, zero-shot overall
    34.4%
    success rateAuthors' own evaluation. Rises to 42.7 after fine-tuning. Roughly half the Google Robot score, so performance is strongly embodiment dependent.source

Real world

No results in this category are published for this model.

Fine tuned tasks

What the model reaches after being adapted to a new task or a new robot, usually from a small number of demonstrations. This is the number that matters if you bring your own data.

  • Real-world WidowX (NORA authors' setup) 9 tasks, 10 trials each, overall average
    11.1%
    success rateTHIRD PARTY EVALUATION, not run by the SpatialVLA authors. Reported by the NORA authors, who measured NORA at 56.7 and OpenVLA at 40 on the same nine tasks. Treat as evidence that SimplerEnv scores do not transfer to an arbitrary real arm without fine-tuning, not as a verdict on the model.source

On real hardware

The paper reports real robot evaluations alongside simulation, showing in distribution multi-task generalisation and out of distribution adaptation after re-discretising the Adaptive Action Grids for the new setup. The official model card is explicit that the model does not zero-shot generalise to new robot embodiments or settings absent from pretraining, and that fine-tuning on collected demonstrations is required instead. An independent evaluation by the NORA authors on their own WidowX arm placed zero-shot SpatialVLA at 11.1 percent across nine tasks, which is consistent with that caveat.

Fine tuning it yourself

Adaptation to a new robot works by re-discretising the pre-learned Adaptive Action Grids to that robot's action statistics, which is the mechanism the authors propose instead of retraining the action space. Full parameter and LoRA fine-tuning are both supported and the authors ran their fine-tuning on 4 to 8 A100 GPUs. Requires transformers >= 4.47.0. The model card recommends fine-tuning rather than zero-shot use for real world datasets of limited scale.

Where it helps, where it does not

Strengths

  • Pretrained on 1.1 million real robot episodes from Open X-Embodiment and RH20T, the largest pretraining corpus of any model in this comparison.
  • 71.9 percent zero-shot on SimplerEnv Google Robot visual matching without any task specific fine-tuning.
  • Documented 8.5 GB inference memory on the official model card, one of the few explicit VRAM figures in this field.
  • Ego3D Position Encoding and Adaptive Action Grids give an explicit mechanism for adapting the action space to a new robot instead of retraining from scratch.
  • MIT licensed with several released variants and full Hugging Face transformers integration.
  • Peer reviewed at RSS 2025.

Limits

  • WidowX Bridge zero-shot at 34.4 percent is less than half the Google Robot score of 71.9 percent, so performance depends heavily on how close the target embodiment is to the pretraining distribution.
  • LIBERO-Long at 55.5 percent is a weak long horizon result, roughly 34 points behind NORA-1.5.
  • The official model card states outright that the model does not zero-shot generalise to new embodiments and that fine-tuning is required.
  • In the NORA authors' independent real WidowX evaluation it averaged 11.1 percent across nine tasks, far below its simulation figures, which is the sharpest available warning that SimplerEnv scores do not transfer unmodified.
  • No published inference latency and no control frequency, so real time suitability has to be measured.
  • Last paper revision is v5 from May 2025 and no successor version has appeared since.

Sources

Everything on this page was taken from these documents. Where they disagree with what you read here, they win.

Entry last checked 2026-08-11.

Try a policy on a real arm

A physical SO-100 is online and free to drive in the browser, and five of the models in this arena can be fine tuned on a dataset you record with your own.