What goes in and what comes out
- In: one or more camera streams, typically a wrist view and a scene view.
- In: the current joint state of the arm, six values on an SO-100 sized robot.
- In: one task sentence, the same one you typed while recording.
- Out: joint targets, emitted as a block of future actions rather than one next command.
The language half is what separates a VLA from a plain vision policy. Because the text encoder was pretrained on language long before a robot was involved, it tolerates phrasings it never saw during your run: a policy recorded on "pick up the red cube and place it in the bowl" generally still answers to "put the red cube in the bowl". What it cannot do is invent behavior. The sentence selects among things the model was shown, it does not describe new ones.
The four VLAs here, and the one model that is not one
| Model | Parameters | Reads a task sentence | Inference per action step |
|---|---|---|---|
| GR00T N1.7 | about 3 billion | yes | about 152 ms |
| GR00T N1.5 | about 3 billion | yes | about 165 ms |
| Pi0.5 | about 3 billion, PaliGemma backbone | yes | about 485 ms |
| SmolVLA | about 450 million | yes | about 245 ms |
| ACT | about 80 million | no | about 20 ms |
Pretraining is the other half of the definition in practice. A VLA arrives as a checkpoint that has already seen a large amount of robot and image data, so your run adapts it rather than building it. That is why SmolVLA lists about 30 episodes as a floor and the three billion parameter models about 50, while a from-scratch model has to learn everything from your dataset alone.
Half your episodes saying "pick up the red cube" and the other half saying "grab cube" teaches the model that the sentence carries no information. Write one sentence per task, use it for every episode of that task, and use the same one again at inference.
The per-model pages under /policies go through what each architecture does with these inputs. /policies/pi0-5 is the deepest one on the output side, because flow matching is where the differences become visible on hardware.