How to train SmolVLA on an SO-100

Why SmolVLA is the run to start with

At about 450 million parameters, SmolVLA fine-tunes inside 24 GB of VRAM. The other vision-language-action models on this platform are roughly 3 billion parameters and need an 80 GB A100, which the pool rents at 1.20 to 2.00 USD per hour against 0.30 to 0.60 USD for a 24 GB card. The interesting part is not the money, it is what the money does to your habits. At a couple of dollars per run you will try four versions of a dataset over a weekend. At ten dollars a run you will try one and then spend the weekend arguing with yourself about whether it was worth repeating.

The second reason is the data floor. SmolVLA becomes useful at around 30 episodes. GR00T N1.7 and Pi0.5 want 50 or more before a fine-tune has anything to hold on to, and so does ACT, which is far smaller than any of them. Thirty deliberate demonstrations is one focused evening. That gap decides what your first weekend with the arm looks like.

The third reason is that SmolVLA reads the task string. Every frame it trains on is paired with the sentence you typed when you recorded that episode, and at inference you pass a sentence too. One policy can therefore cover two or three related jobs in the same scene, which is something ACT cannot do at all.

What you need before you start

• A calibrated SO-100 with all six STS3215 servos responding. /learn/so-100-getting-started covers assembly, pairing and calibration from the box.
• At least one camera, fixed so it cannot be nudged. A scene camera plus a wrist camera is noticeably better than a scene camera alone.
• The desktop client or the CLI, paired to your account. The robot driver ships inside it, so there is no Python environment to build and no dependency resolution to lose an afternoon to.
• Thirty or more reviewed episodes of one task, recorded as LeRobot v2.1 and uploaded.
• Billing set up on the account, because the run rents a GPU by the hour. Current rates are on /pricing.

curl -fsSL https://www.ay-robots.com/install.sh | sh
ay-robots --version

# Pairing code comes from the Robots page in the dashboard
ay-robots pair 4QD2-8HRM-1XV6

# Six servos and your cameras should both show up here
ay-robots devices

# Once per physical arm, before you record anything
ay-robots calibrate

The dataset: thirty episodes and one sentence

Language conditioning is a real feature and a real trap in equal measure. Used deliberately, one policy answers to several phrasings of the same job. Used carelessly, you split a 30 episode dataset into three sets of ten that the model treats as three unrelated tasks, and none of the three has enough behind it to learn anything.

The rule while recording is one task, one sentence, typed identically every time. Write the sentence down once and paste it. Do not retype it, because you will not retype it the same way at eleven at night on episode 27.

SmolVLA carries a pretrained language encoder, so a policy trained on "pick up the red cube and place it in the bowl" will often still respond to "put the red cube in the bowl". Do not lean on that with 30 episodes. Tolerance to rewording is something you measure during evaluation, not something you assume during recording.

# meta/episodes.jsonl, one line per recorded episode
{"episode_index": 0, "tasks": ["pick up the red cube and place it in the bowl"], "length": 388}
{"episode_index": 1, "tasks": ["pick up the red cube and place it in the bowl"], "length": 402}
{"episode_index": 2, "tasks": ["pick up red cube, place in bowl"], "length": 377}
{"episode_index": 3, "tasks": ["pick up the red cube and place it in the bowl"], "length": 395}

Concretely: thirty episodes of putting the cube in the bowl plus thirty of putting it on the plate, same scene and same cameras, train into one policy you steer with the string you pass at inference. Plan for that during the first recording session, because retrofitting a second task into an existing scene is much harder once the tripod has moved.

Start the run

1. 1
   Check the dataset format

   Open Datasets and look at the format on your upload. SmolVLA reads LeRobot v2.1, and so does GR00T, which rejects v3.0 outright. Recording v2.1 costs you nothing at SO-100 dataset sizes and keeps the upgrade path open.

2. 2
   Open Training and select the dataset

   The picker only lists uploads that finished processing. If yours is missing, ingest is still running, and the archive step on a two-camera dataset takes a few minutes.

3. 3
   Pick SmolVLA as the model

   The form redraws for the model you choose. For SmolVLA you get batch size, learning rate, steps, gradient accumulation, seed and log frequency. Chunk size and action steps belong to ACT and are not shown here.

4. 4
   Name the policy so you can read it later

   Arm, task, model, version. Something like so100-cube-bowl-smolvla-v1. Six weeks from now the run list is the only record of what you tried, and three entries called test will not help you.

5. 5
   Leave the defaults alone, but fix the seed

   Batch 2, learning rate 1e-4, 20 000 steps, gradient accumulation 8. Set the seed to a number you write down. Without a fixed seed you cannot tell a genuine data improvement from run-to-run noise on your next attempt.

6. 6
   Press Start, then watch the first few hundred steps

   The pool buys a 24 GB card on the spot market, so there is a wait before the first log line. Once loss is printing, the run page shows loss, learning rate, gradient norm and GPU telemetry. If loss is still flat after several hundred steps, stop the run rather than pay for the remaining four hours.

The defaults, and when to change them

Seed and log frequency exist for the lerobot family, which is ACT, SmolVLA and Pi0.5. Both GR00T entry points have no seed field at all. If you plan to graduate to GR00T later, do not build a workflow that depends on bit-for-bit reproducibility, because it stops being available the moment you get there.

Cost, duration and the agent route

A SmolVLA run sits on the 24 GB tier: roughly 0.30 to 0.60 USD per hour, and a full 20 000 step run on a dataset of this size lands in the 2 to 5 hour range. Call it 1 to 3 USD start to finish. The A100 tier that GR00T and Pi0.5 require is 1.20 to 2.00 USD per hour over 3 to 6 hours, so about 4 to 12 USD for the equivalent run.

Spot pricing moves daily and the pool selects by VRAM rather than by GPU name, so the card you get is whichever 24 GB device was cheapest that minute. This changes the wall clock, not the result.

curl -sS https://www.ay-robots.com/api/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -H "MCP-Protocol-Version: 2025-06-18" \
  -H "Authorization: Bearer $AY_ROBOTS_API_KEY" \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "tools/call",
    "params": {
      "name": "start_training",
      "arguments": {
        "dataset_id": "ds_8f3a91c4",
        "policy_name": "so100-cube-bowl-smolvla-v1",
        "model_id": "smolvla",
        "max_steps": 20000,
        "confirm_spend": true
      }
    }
  }'

start_training refuses to do anything unless confirm_spend is true, because renting a GPU on a vague instruction is not a thing an agent should be able to do. The full tool list is on /mcp.

Run the policy on the arm

Deployment does not care that the policy is a VLA. Pick a checkpoint, load it in the client, pass a task string, and the arm moves. Two things are specific to SmolVLA.

The first is latency. SmolVLA reports about 245 ms per action step. Next to ACT at about 20 ms that is a different feel: the arm advances in visible increments rather than flowing. For pick and place it is fine. For anything that has to react to a moving object it is not, and no amount of tuning changes that.

The second is that you have to supply a task string, and on the first evaluation it should be exactly the string you recorded with. Testing a paraphrase is a good experiment, it is just a separate one.

1. Put the cameras back in the exact positions used during recording. The policy is largely reading pixel locations, and a moved tripod is still the leading cause of confident, wrong behavior.
2. Run twenty attempts with the object in different positions and write each outcome down as it happens, not afterwards from memory.
3. Repeat with two or three checkpoints from the back half of the run. The last checkpoint is not automatically the best one.
4. Then run the same twenty attempts with a paraphrased task string and note the drop. That number tells you how much language robustness your dataset actually bought.

If the whole loop is new to you, /learn/run-your-first-policy walks the deployment side in more detail, and /live lets you drive a real SO-100 in the browser without owning one.

Use the cheap run as a data-quality probe

The most useful output of a SmolVLA run is often not the policy. It is the verdict on your dataset. A run that cannot learn the task at all is nearly always telling you something about the data rather than something about the model, and it tells you for a dollar instead of twelve.

Read it like this. If loss falls normally, the policy loads, and the arm makes recognizably task-shaped attempts even at a poor success rate, the dataset has signal, and a GR00T fine-tune on the same episodes is a reasonable next purchase. If the arm does something unrelated to the task on every single attempt, six hours on an 80 GB card will buy you a more expensive version of the same disappointment.

Troubleshooting

When to move up, and where

SmolVLA has a lower ceiling than the 3 billion parameter models on anything needing fine positioning. The signal that you have hit it is a success rate that stops responding to more data. At that point the next step is GR00T N1.7: same v2.1 dataset, an 80 GB card, batch size 32, about 152 ms per action step, and no seed field. /train/groot-n1-7-on-so-100 is that run end to end, and /policies/groot-n1-7 has the numbers.

If what you need is speed rather than generalization, the move is sideways instead of up. ACT runs at roughly 20 ms per step on the same 24 GB tier, learns your one task very well and nothing else, and needs a 100 000 step schedule to get there. /compare/act-vs-smolvla lays out that trade honestly.