SmolVLA: the model to train first

Why this is the model to run first

Every vision-language-action model here except SmolVLA is a roughly 3 billion parameter network that the pool has to put on an 80 GB card. SmolVLA is about 450 million parameters and fits on a 24 GB consumer GPU. That one difference decides what a run costs, how long you wait for an answer, and how many times you can afford to be wrong before the dataset is right.

It is a real VLA, not a toy: camera frames, arm state and a natural-language task string in, actions out. It arrives as a pretrained checkpoint, so your job is a fine-tune rather than a fit from random weights, which is why about 30 good episodes already get somewhere useful.

What 450 million parameters can and cannot do

On one tabletop task recorded with a single SO-100 and fixed cameras, SmolVLA reaches a genuinely useful policy: it picks the object up from placements around the ones you demonstrated, and it absorbs small approach errors. What it will not do is beat a 3 billion parameter model on precision, and it carries less pretraining, so its tolerance for unfamiliar scenes is narrower.

• Realistic: one task, or a few related tasks, in the scene you recorded
• Realistic: object placements spread across the area you covered
• Optimistic: millimeter-accurate insertion, or anything decided by contact forces
• Not going to happen: a task you never demonstrated, described in words at inference time

Language conditioning, and what it buys you

The task string you type before each recording is not metadata for SmolVLA, it is an input. The model encodes the sentence alongside the images, so one checkpoint can hold several behaviors and pick between them based on what you ask for.

That buys two things. One policy for several phrasings of one task: train on "pick up the red cube and place it in the bowl" and asking for "put the red cube in the bowl" generally still works, because the text side was pretrained and does not need an exact match. And one policy for several tasks, if you recorded them into one dataset with distinct strings. What it does not buy is new behavior: language selects among things the model has seen.

Why a 24 GB card changes the economics

GPUs are rented by the hour on a spot market, and the pool selects them by VRAM rather than by name. The two tiers are far enough apart that they support different working styles, not just different invoices.

The interesting number is not the total, it is the threshold at which you stop deliberating. At one to three dollars a run, the cheapest way to answer "would more episodes help" is to record them and train again. At four to twelve, people reason about their dataset instead of testing it, which is guesswork with extra steps. Spot rates move daily, so check /pricing before planning a series.

SmolVLA is also not cloud-only. The trained checkpoint runs on your own machine at inference, so you rent a GPU while training and nothing at all while the arm works.

The defaults, and when to change them

SmolVLA starts at batch size 2 with 8 gradient accumulation steps, a learning rate of 1e-4 and 20,000 steps. The batch is small because 24 GB of VRAM bounds it; accumulation puts it back, giving an effective batch of 16 without a bigger card. The form also exposes seed and logFreq, both belonging to the lerobot family.

Every trainer here is step-based, not epoch-based. Twenty thousand steps at an effective batch of 16 is 320,000 sample presentations, and that number does not move when your dataset grows. Double the episodes and each is seen half as often, which is the most common reason a second, larger dataset trains worse than the first.

Leave the learning rate at 1e-4 unless the curve misbehaves. Flat from step one, or spiking and staying high, is a learning rate or data-loading problem, and /learn/train-your-first-policy has the full table of curve shapes.

# Start a SmolVLA run through the hosted MCP server.
# The web UI does the same thing: open Training, pick the dataset, press Start.
curl -s https://www.ay-robots.com/api/mcp \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AY_ROBOTS_API_KEY" \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "tools/call",
    "params": {
      "name": "start_training",
      "arguments": {
        "dataset_id": "ds_7f3a91c4",
        "policy_name": "cube-pick-smolvla-v1",
        "model_id": "smolvla",
        "max_steps": 40000,
        "confirm_spend": true
      }
    }
  }'

Use the first run as a data-quality probe

The most valuable output of a first SmolVLA run is usually not the policy, it is a verdict on the dataset. If SmolVLA cannot learn the task at all on the defaults, a 3 billion parameter model on the same data will almost never rescue it, because what kills an imitation dataset is a property of the data rather than of the network.

Contradictory demonstrations, a camera that moved between sessions, kept failures, an action stream that does not line up with the frames: none of that is fixed by capacity. A larger model fits contradictions more confidently, it does not resolve them.

1. 1
   Train once on the defaults, changing only the seed

   Open Training, pick the dataset, choose SmolVLA, set a seed and write it down. A tuned first run that fails tells you nothing about which change caused it.

   text

   Training form, SmolVLA on an SO-100 dataset
   
     Model              SmolVLA
     Batch size         2         default
     Grad accumulation  8         default, effective batch 16
     Learning rate      1e-4      default
     Steps              20000     default
     Seed               1000      your choice, but write it down

2. 2
   Evaluate on the arm, twenty attempts, written down

   Cameras exactly where they were during recording, same lighting, same reset. Varied placement, and write down every attempt, including the embarrassing ones.

3. 3
   Classify the failure before changing anything

   Failures cluster, and the cluster is the diagnosis. Map what you saw onto the table below.

4. 4
   Change exactly one thing, then run again

   Either the data or the model, never both. Fix the dataset and switch to GR00T in one step and you pay 80 GB rates to learn nothing.

When to graduate to GR00T N1.7 or Pi0.5

Move up when the data has stopped being the limiting factor: you have a dataset SmolVLA learns from, an evaluation protocol you trust, and a specific complaint that more episodes will not fix.

1. At the ceiling: SmolVLA mostly succeeds and the rest is precision, not confusion. GR00T N1.7 is the default next step, and at about 152 ms it is faster at inference too.
2. Generalization you never demonstrated: GR00T N1.7 carries pretraining on a large multi-robot corpus, which buys tolerance for unseen object positions.
3. Contact-rich work: Pi0.5, at about 485 ms per action step, visible as hesitation on fast motions.

Both jump you to the 80 GB tier, so budget four to twelve dollars per run and expect to iterate less. /compare/groot-n1-7-vs-smolvla puts the two side by side.

Where to go next

• /train/smolvla-on-so-100 runs the whole thing end to end on an SO-100
• /learn/train-your-first-policy for loss curves and checkpoint selection
• /compare/act-vs-smolvla is the decision most first projects face
• /compare/groot-n1-7-vs-smolvla is the one you face once the data is good
• /live drives a real SO-100 in the browser, no signup
• /download has the desktop client, driver included