OpenVLA on a Low-Cost Arm: 7B Params, LoRA, and the SO-100

What OpenVLA actually is

OpenVLA is what made vision-language-action models reproducible outside a large industrial lab. Before it the strong results belonged to RT-2-X, a closed 55 B-parameter model. OpenVLA published weights, training code and data mixture, and beat RT-2-X by 16.5 points of absolute task success across 29 tasks and multiple embodiments, with 7x fewer parameters.

Architecturally it is a vision-language model with the text head repurposed. An image and an instruction go in, and the model autoregressively emits action tokens. The trick is the tokenizer: each action dimension is discretized into 256 bins, with edges at the 1st and 99th quantile rather than the min and max, so outlier demonstrations cannot blow out the resolution of the action space. Those 256 values overwrite the 256 least-used tokens in the Llama vocabulary. No new head, no new loss.

What it was trained on, and what that excludes

The pretraining mixture decides whether OpenVLA has ever seen anything like your arm. It starts from Open X-Embodiment, which at the time held more than 70 robot datasets and over 2 M trajectories, then filters hard.

• Manipulation only. No navigation, no locomotion.
• At least one third-person camera in the dataset.
• Single-arm end-effector control. This is the filter that matters: joint-space datasets were excluded by construction.
• Mixture weights borrowed from Octo, which down-weights low-diversity datasets.
• DROID went in at 10% weight, then was pulled entirely for the final third of training because action token accuracy on it stayed low.

The fine-tuning bill, from the paper's own table

The paper compares five adaptation strategies on Franka-Tabletop tasks. It is the most useful table in the paper if you are budgeting a fine-tuning run. VRAM is at batch size 16.

Two conclusions. Freezing the vision encoder is the one thing you must not do: it costs 22 points against full fine-tuning while still burning 156 GB, because the visual features genuinely need to adapt to your scene. And LoRA at rank 32 lands within 1.5 points of full fine-tuning while training 1.4% of the model. Full fine-tuning needs 8 A100s for 5 to 15 hours per task; LoRA needs one A100 for 10 to 15 hours, an 8x compute reduction.

Where OpenVLA and an SO-100 disagree

This section decides the question in the title, and it has nothing to do with parameter counts. Put a real public LeRobot SO-100 dataset schema next to what OpenVLA emits.

# lerobot/svla_so100_pickplace -> meta/info.json  (codebase_version v3.0)
action                    [6]  main_shoulder_pan, main_shoulder_lift,
                               main_elbow_flex,  main_wrist_flex,
                               main_wrist_roll,  main_gripper
observation.state         [6]  the same six joints
observation.images.top    [480, 640, 3]
observation.images.wrist  [480, 640, 3]

# openvla/openvla-7b
action                    [7]  dx, dy, dz, droll, dpitch, dyaw, gripper
                               (normalized end-effector deltas)
image                     one 224 x 224 third-person view, and only one

None of these are impossible. Together they mean you are not fine-tuning a model, you are porting one: convert every episode to RLDS, either relabel your LeRobot dataset into end-effector deltas or redefine the action space to six joints and accept that the pretrained action tokens no longer mean what they meant, drop the wrist camera, then write the end-effector to joint-command bridge for inference. That last piece has to be right or the policy will look broken when it is fine.

The imitation learning signal in your data is fine. The mismatch is plumbing and representation, which is why this takes weeks rather than an afternoon.

Running OpenVLA yourself, end to end

Here is the honest path, with the repo's real flags and defaults. It pins versions and means it: PyTorch 2.2.0, torchvision 0.17.0, transformers 4.40.1, tokenizers 0.19.1, timm 0.9.10, flash-attn 2.5.5.

1. 1
   Build the pinned environment

   Flash Attention 2 has to be installed without build isolation or it will fail to compile against your torch. The LoRA script additionally requires peft==0.11.1.

   bash

   conda create -n openvla python=3.10 -y
   conda activate openvla
   conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia -y
   
   git clone https://github.com/openvla/openvla.git
   cd openvla
   pip install -e .
   
   pip install packaging ninja
   pip install "flash-attn==2.5.5" --no-build-isolation
   pip install "peft==0.11.1"

2. 2
   Get a dataset into RLDS

   The README example uses BridgeData V2, a 124 GB download. Renaming is not optional; the loader finds the dataset by name.

   bash

   cd <PATH TO BASE DATASETS DIR>
   wget -r -nH --cut-dirs=4 --reject="index.html*" \
     https://rail.eecs.berkeley.edu/datasets/bridge_release/data/tfds/bridge_dataset/
   
   # Required. Skipping this produces runtime errors later.
   mv bridge_dataset bridge_orig

3. 3
   Register your own data

   Convert to RLDS with kpertsch/rlds_dataset_builder, then edit two files: a dataset config and a transform function mapping your columns onto the model's action layout. This is where the 6-versus-7 problem becomes code you write.

   text

   prismatic/vla/datasets/rlds/oxe/configs.py     # add your dataset config
   prismatic/vla/datasets/rlds/oxe/transforms.py  # add your transform fn

4. 4
   Launch LoRA fine-tuning

   These are the repo's documented values. finetune.py defaults that are easy to miss: max_steps 200000, save_steps 5000, image_aug True, shuffle_buffer_size 100000, lora_dropout 0.0, use_quantization False.

   bash

   torchrun --standalone --nnodes 1 --nproc-per-node 1 vla-scripts/finetune.py \
     --vla_path "openvla/openvla-7b" \
     --data_root_dir <PATH TO BASE DATASETS DIR> \
     --dataset_name bridge_orig \
     --run_root_dir <PATH TO LOG/CHECKPOINT DIR> \
     --adapter_tmp_dir <PATH TO TEMPORARY ADAPTER DIR> \
     --lora_rank 32 \
     --batch_size 16 \
     --grad_accumulation_steps 1 \
     --learning_rate 5e-4 \
     --image_aug True \
     --save_steps 5000

5. 5
   Serve the checkpoint

   deploy.py exposes a REST endpoint; defaults are host 0.0.0.0 and port 8000. Your client posts an image plus an instruction and gets seven normalized deltas back, which you un-normalize and convert to joint commands yourself.

   bash

   python vla-scripts/deploy.py \
     --openvla_path openvla/openvla-7b \
     --host 0.0.0.0 \
     --port 8000

6. 6
   Un-normalize with the right key

   Actions come back normalized against a specific dataset's statistics. Pass the unnorm_key for the dataset you fine-tuned on, or the numbers will be silently wrong rather than obviously wrong.

   python

   from transformers import AutoModelForVision2Seq, AutoProcessor
   import torch
   
   processor = AutoProcessor.from_pretrained("openvla/openvla-7b", trust_remote_code=True)
   vla = AutoModelForVision2Seq.from_pretrained(
       "openvla/openvla-7b",
       attn_implementation="flash_attention_2",
       torch_dtype=torch.bfloat16,
       low_cpu_mem_usage=True,
       trust_remote_code=True,
   ).to("cuda:0")
   
   action = vla.predict_action(**inputs, unnorm_key="bridge_orig", do_sample=False)

Two routes to the same goal

How OpenVLA compares to what this platform trains

Latency first, because it is the number people get wrong. At bfloat16 on an RTX 4090 OpenVLA runs at roughly 6 Hz, which inverts to about 167 ms per action. That lands just above both GR00T figures, 152 ms for N1.7 and 165 ms for N1.5, and well ahead of Pi0.5 at 485 ms. OpenVLA is not slow for its class. It is memory-hungry and format-hostile, which is a different complaint.

OpenVLA-OFT changed the answer

If you are evaluating OpenVLA today you are evaluating two things, and the README says so. Since a note dated 2025-03-03 it recommends the Optimized Fine-Tuning recipe. OFT keeps the base model and changes four things: parallel decoding instead of autoregressive, action chunking, continuous actions instead of the 256-bin tokens, and an L1 regression objective instead of cross-entropy.

The effect is large. Average LIBERO success goes from 76.5% to 97.1%, and action generation throughput improves 26-fold. On a bimanual ALOHA setup the recipe beat Pi0 and RDT-1B fine-tuned with their own defaults, and beat Diffusion Policy and ACT trained from scratch by up to 15 points absolute. Three of those four changes are things the models on this platform's policies page already do by design: chunked, continuous actions with a regression or flow objective.

Read that honestly. These numbers are Appendix E.2, over 500 trials per suite and three seeds. OpenVLA takes both the best average success rate and the best average rank, 1.5, but Diffusion Policy from scratch beats it outright on LIBERO-Object, and every method is near 50% on the long-horizon suite. The paper attributes the narrow margins to OpenVLA being pretrained purely on real-world data with no simulation. It also lists the cleaning required first: no-op actions filtered out, images rotated 180 degrees, failed demonstrations removed, including 121 of 500 in LIBERO-Long, and wrist-camera images discarded for every method so the comparison matches OpenVLA's single third-person input.

The verdict for an SO-100 owner

OpenVLA is an important model and a good codebase. For a low-cost joint-space arm it is also the wrong tool for getting a task working, not because 7 B is too big or 167 ms too slow, but because the whole interface, from dataset format to action space to camera count, was built for a different family of robots.

• Studying VLA adaptation or tokenization: use OpenVLA, starting from the OFT recipe.
• Franka or WidowX data already in RLDS with end-effector control: OpenVLA is a strong initialization and LoRA rank 32 is cheap.
• An SO-100 and a LeRobot dataset, and you want a working policy: use a model that reads it natively. Start with training your first policy.
• Fewer than 50 episodes: SmolVLA accepts 30, and collecting better data beats changing models.
• Want data rather than an opinion: the Arena carries 85 models and 332 benchmark results.

For background, read vision-language-action models and the complete SO-100 setup guide. What a run costs is on pricing, and the training form is documented in the training docs.