Small VLA Models: TinyVLA, SmolVLA and the Sub-1B Case

Almost every vision-language-action model that gets written about assumes a datacentre card. OpenVLA is 7 B parameters. GR00T N1.7 and Pi0.5 are around 3 B and want an A100 80 GB to fine-tune. If the card on your desk is a 4090, that class of model is out of reach.

Two papers argue you do not need it. TinyVLA (arXiv 2409.12514, accepted by IEEE Robotics and Automation Letters in February 2025) builds a VLA from a 400 M to 1.3 B backbone plus a diffusion action head. SmolVLA (arXiv 2506.01844, submitted 2 June 2025) ships 450 M parameters with open weights, open data and a script that runs on one consumer card. Here is what both measured, and where the sub-1 B argument stops holding.

What actually counts as a small VLA

The parameter count on a model card is not one number. It can mean total weights, weights loaded at inference, or weights that receive gradients during fine-tuning. TinyVLA reports both, and the gap is large: TinyVLA-H is 1.3 B total but 143 M trainable, because the vision tower and most of the language model stay frozen while LoRA adapters and the action head move. The paper puts the trainable share at about 5 percent.

Two rows are worth arguing about. ACT at roughly 80 M is smaller than everything else here but has no language model, so it is not a VLA: it learns one task and cannot be told to do another. Octo-Small at 27 M is conditioned through a T5-Base text encoder, not an LLM backbone. Good baselines, neither gives you the instruction following the label implies.

TinyVLA: the speed comes from the head, not the backbone

The obvious reading is that they made the language model smaller, so it got faster. The paper's ablation says otherwise. Swapping OpenVLA's 7 B backbone for a roughly 1 B one cut per-action prediction from 292 ms to 140 ms, a 2x gain. TinyVLA-H, at a comparable parameter count, runs at 14 ms on the same card. The other 10x is the action head.

OpenVLA emits actions as discretised tokens through the language model head, one per action dimension, autoregressively. TinyVLA attaches a diffusion decoder that produces the whole chunk in one shot. Table V holds the architecture at TinyVLA-H and swaps only the head, on the same five real-robot tasks. It is the cleanest evidence in either paper for the argument flow matching policies make, and the reason Pi0 moved away from token decoding.

What TinyVLA scored

The bimanual row has a boring explanation the paper gives itself: OpenVLA is pretrained on Open X-Embodiment, which consists entirely of single-arm data, so a two-arm action space is out of distribution and it scores zero. The diffusion policy baseline beats TinyVLA-H on two of those three tasks. Background in the Open X-Embodiment write-up.

The TinyVLA repo, as of August 2026

The paper is good. The code is a research drop. The repository is github.com/liyaxuanliyaxuan/TinyVLA; its README dates the code release to 17 February 2025 and the last commit is 11 March 2025. Fine for reproducing a paper, a problem if you wanted a maintained library.

git clone https://github.com/liyaxuanliyaxuan/TinyVLA
conda create -n tinyvla python=3.10 -y
conda activate tinyvla
pip install --upgrade pip
pip install -r requirements.txt
cd policy_heads && pip install -e .
cd ../llava-pythia && pip install -e .

TinyVLA also does not read LeRobot datasets. Its format is ACT-style HDF5: action, language_raw, and an observations group with multi-view images, joint_positions, qpos and qvel. The repo ships data_utils/rlds_to_h5py.py for RLDS input, and each task is registered by hand in aloha_scripts/constants.py with its dataset_dir, episode_len and camera_names. If your episodes came from lerobot or the desktop client, you own the conversion.

# scripts/train.sh, the real flags from the repository
ACTION_HEAD=droid_diffusion

deepspeed --master_port 29600 --num_gpus=8 --num_nodes=1 ./train_tinyvla.py \
  --deepspeed scripts/zero2.json \
  --lora_enable True \
  --lora_module 'vit llm' \
  --lora_r 64 \
  --lora_alpha 256 \
  --non_lora_lr 2e-5 \
  --task_name "example_task_config" \
  --model_name_or_path /path/to/pretrained_vlm \
  --freeze_vision_tower True \
  --freeze_backbone True \
  --bf16 True \
  --max_steps 10000 \
  --per_device_train_batch_size 32 \
  --gradient_accumulation_steps 1 \
  --learning_rate 2e-4 \
  --lr_scheduler_type "cosine" \
  --warmup_ratio 0.005 \
  --save_steps 1000 \
  --action_head_type $ACTION_HEAD \
  --use_state True \
  --window_size 6

• --num_gpus=8 assumes an eight-card node. Set it to 1 and drop the batch size well below 32. No single-GPU variant ships upstream, so the command cannot be run verbatim on a 4090.
• --deepspeed scripts/zero2.json points at a file that is not in the top-level scripts directory. The DeepSpeed configs ship at llava-pythia/scripts/zero2.json. Fix the path before your first run.
• The README requires the output directory name to contain llava_pythia, plus lora if LoRA is enabled.
• The backbone is a separate download: lesjie/Llava-Pythia-400M, -700M or -1.3B. There is no single base checkpoint you point a policy at the way you point at lerobot/smolvla_base.

SmolVLA: the sub-1 B model you can run this afternoon

SmolVLA makes the same argument inside a maintained library. It is 450 M parameters on a SmolVLM2-500M-Video-Instruct backbone, and the efficiency comes from settings you can read out of configuration_smolvla.py rather than from a claim about being small.

Pretraining used 481 community LeRobot datasets, 22.9 K episodes and 10.6 M frames on 4 GPUs, 200000 steps at a global batch of 256; the paper puts the whole project at about 30 K GPU hours. One number to watch: the Hugging Face launch blog says 487 curated datasets where the paper's table says 481. We use the paper's figure and flag the disagreement.

One accident helps: the SmolVLA paper's Meta-World table carries TinyVLA's own published numbers as a baseline, so for once both models sit in one table, SmolVLA-0.45B at 57.3 against TinyVLA-H at 31.6. It also reports SmolVLA training around 40 percent faster than Pi0 on 6x less memory. All of it comes from the SmolVLA authors; the arena entry links each value to its source.

Where SmolVLA spends its time

AY-Robots lists SmolVLA at 245 ms per action step and ACT at 20 ms in the policy catalog. TinyVLA reports 14 ms per action prediction. Those numbers are not comparable, and treating them as if they were is the most common mistake in this topic: some time one forward pass, some divide that pass across a chunk once action chunking amortises it.

• SmolVLA emits 50 actions per forward pass and executes all 50. At the 30 fps the paper uses on real robots, one inference covers about 1.7 seconds of motion.
• Asynchronous inference computes the next chunk while the current one still executes: 9.7 s average task completion against 13.75 s synchronous, roughly 30 percent faster, and 19 pick-and-place cycles in a fixed 60-second window against 9.
• Success rates were comparable rather than better, 78.3 synchronous against 73.3 asynchronous. Async buys throughput, not accuracy.
• Chunking hides compute latency, not network latency, and it makes the policy less reactive because it is committed to 50 actions.

Fine-tuning SmolVLA on one consumer card

The manual path, on lerobot 0.6.1, the current PyPI release as of 23 August 2026. Everything below comes from the Hugging Face lerobot SmolVLA documentation, fetched the same day; the guided version is gentler.

1. 1
   Install lerobot with the smolvla extra

   The SmolVLA dependencies are an optional extra, not part of the base install. Installing without it and then wondering why the policy type is unknown is a common half hour lost.

   bash

   git clone https://github.com/huggingface/lerobot.git
   cd lerobot
   pip install -e ".[smolvla]"

2. 2
   Get a dataset with enough episodes

   The docs recommend around 50 episodes and state that the same task with 25 was not enough. AY-Robots sets the minimum at 30. Record your own, or start from the dataset directory.

   bash

   # any LeRobotDataset on the Hub works as --dataset.repo_id
   # lerobot/svla_so100_pickplace is the paper's own 50-episode set:
   # 5 cube positions, 10 episodes each

3. 3
   Start the fine-tune

   The command from the lerobot guide, unmodified. The docs put 20000 steps at roughly 4 hours on one A100. On a 24 GB card, expect longer and measure it.

   bash

   cd lerobot && lerobot-train \
     --policy.path=lerobot/smolvla_base \
     --dataset.repo_id=${HF_USER}/mydataset \
     --batch_size=64 \
     --steps=20000 \
     --output_dir=outputs/train/my_smolvla \
     --job_name=my_smolvla_training \
     --policy.device=cuda \
     --wandb.enable=true

4. 4
   Drop the batch size until it fits

   batch_size=64 is a documented example, not a promise about your card. The docs advise starting small and increasing while loading times stay short.

   bash

   lerobot-train --help

5. 5
   Roll the checkpoint out on the arm

   Same library, one command. The real-time chunking flags are commented out in the docs and are the ones to reach for on low-power hardware.

   bash

   lerobot-rollout \
     --strategy.type=base \
     --robot.type=so101_follower \
     --robot.port=/dev/ttyACM0 \
     --robot.id=my_blue_follower_arm \
     --robot.cameras="{ front: {type: opencv, index_or_path: 8, width: 640, height: 480, fps: 30}}" \
     --task="Grasp a lego block and put it in the bin." \
     --policy.path=HF_USER/FINETUNE_MODEL_NAME

Two ways to get a small VLA onto an arm

When a small model is the wrong choice

Both papers are more careful here than the summaries are. TinyVLA's failure analysis is specific: the 0.4 B variant failed three times by misreading the instruction, which the authors attribute to limited language comprehension in the smaller VLM, and that mode disappeared at 1.3 B. SmolVLA shows the same curve from the other end: the 2.25 B version of the identical architecture scores 88.75 on LIBERO and 68.24 on Meta-World against 87.3 and 57.3 for the 0.45 B model.

The comparison worth running is not TinyVLA against SmolVLA. It is SmolVLA against ACT on your own task, and the SmolVLA paper is the argument for why. Trained single-task with no robotics pretraining, SmolVLA averaged 40.0 across three SO-100 tasks where single-task ACT managed 48.3. What lifts it to 78.3 is community pretraining plus multi-task training, not the architecture alone. On the SO-101 lego task, both single-task, SmolVLA does win: 90 against 70 in distribution. Then SmolVLA against Pi0.5 if the budget allows.

A decision table you can act on

For the last row, the live arm streams a physical SO-100 you can drive from the browser without an account, and the three ways to start covers the rest. The SO-100 is the reference arm here, roughly 110 to 150 EUR in parts, with a full setup guide.

What comes after the sub-1 B models

Shrinking the parameter count is one way to make a VLA cheap and not obviously the winning one. OpenVLA-OFT (arXiv 2502.19645) keeps the 7 B backbone and changes only how actions are decoded, reporting a 26x increase in action generation throughput and LIBERO rising from 76.5 to 97.1 percent. BitVLA (arXiv 2506.07530) makes every weight of a 1-bit BitNet b1.58 2B4T backbone ternary, reporting 11.0x less memory and 4.4x lower end-to-end latency while matching full-precision OpenVLA-OFT. X-VLA-0.9B (arXiv 2510.10274) sits on the 1 B line with flow matching.

The common thread: the action decoder, not the language model, is where the latency lives. Ask how a policy emits actions before you ask how many parameters it has. The arena has 85 models and 332 results, each linked to its source; there is a broader overview in our VLA introduction.