ACT: the specialist

What ACT is, in one paragraph

ACT is a transformer of about 80 million parameters that maps camera frames and joint states onto sequences of joint commands. It came out of the Stanford ALOHA work and it is the smallest model this platform trains. Unlike the other four it starts from random weights: no foundation model underneath, no corpus of other people’s robots, no text encoder. Everything it knows arrived in your dataset, and a model with nothing to unlearn has nothing to fight.

Action chunking, explained properly

A plain behavior-cloning policy works one frame at a time: look, predict one action, execute, look again. The problem is compounding error. Each slightly wrong action puts the arm in a state a little further from the training data, where the next prediction is worse, and the drift accelerates. Halfway through an episode the policy is reasoning about a situation it has never seen, and it put itself there.

ACT changes the unit of prediction. From one observation it emits a whole chunk of future actions, a fixed-length sequence covering the next chunkSize timesteps. The runtime executes the first nActionSteps of that chunk, then takes a fresh observation and predicts again.

# What the runtime does with an ACT checkpoint, in pseudocode.
while not done:
    obs = read_cameras_and_joints()   # one observation
    chunk = policy(obs)               # chunk_size actions, predicted jointly
    for action in chunk[:n_action_steps]:
        send_to_arm(action)           # executed open loop, no new observation

Two separate things follow, and they get conflated constantly. Decision points per episode drop by a factor of nActionSteps, so there are fewer chances to drift and less time for an error to feed on itself. And the actions inside a chunk were generated together rather than independently, so they form one coherent trajectory. That is what produces smoothness: not filtering after the fact, but a model committing to a path instead of averaging possibilities at every frame.

chunkSize and nActionSteps

ACT is the only model whose training form exposes these two knobs, and the lerobot default is 100 for both. The hard constraint is that nActionSteps must be less than or equal to chunkSize: the runtime cannot execute more actions than the model predicts. They are not symmetric. chunkSize is architectural and baked in at training time, so raising it later means training again. nActionSteps only governs how much of each prediction gets used.

Replanning often is nearly free: an inference costs about 20 ms while the arm runs at tens of hertz. The real cost is at the seams, because a fresh chunk need not continue the previous one smoothly, so every boundary can kink the trajectory. Long execution windows give smooth motion and slow reactions, short ones the reverse.

Configure a run

1. 1
   Confirm the dataset is one task

   Check the task strings. If two tasks are mixed in, split the dataset first: ACT cannot tell them apart and will average them.

2. 2
   Pick ACT in the training form

   Open Training, select the dataset, choose ACT. Batch size 8, learning rate 1e-5 and 100,000 steps are prefilled, with no gradient accumulation because at this size there is no need for it.

   text

   Training form, ACT on an SO-100 dataset
   
     Model          ACT
     Batch size     8         default
     Learning rate  1e-5      default
     Steps          100000    default
     Chunk size     100       lerobot default, what the model predicts
     Action steps   100       lerobot default, must be <= chunk size
     Seed           1000      set it so the run is reproducible

3. 3
   Leave chunkSize at 100 unless episodes are short

   If a typical episode is shorter than the chunk, the model spends capacity predicting past the end of the task. That is the one case for lowering it before a first run.

4. 4
   Start it and let it finish

   Do not kill a run at 30,000 steps because the loss looks flat. A from-scratch model at 1e-5 spends a long time looking unimpressive.

5. 5
   Compare checkpoints on the arm

   Keep two or three from the second half of the run and evaluate each physically. The final one is not automatically the best, and the loss gap will not tell you which is.

From scratch, and what that costs you

• One policy per task. There is no task string to switch behavior with, so a second task means a second dataset and a second checkpoint.
• No generalization past your demonstrations. Move a camera, change the lighting, swap in a differently colored cube, and performance drops with no prior to cushion it.
• It wants more of your data than SmolVLA does, about 50 episodes as a working minimum, because nothing else carries the load.
• Whatever it does is what you showed it. Two approach strategies produce a policy that averages them into one that suits neither.

The flip side is diagnostic clarity. When an ACT policy fails, the cause is in your dataset or your setup, never in an interaction with pretraining you cannot inspect. That is why /learn/train-your-first-policy points beginners at ACT before anything larger.

Why 100,000 steps, and why that is still cheap

The default schedule is 100,000 steps at batch size 8 with no accumulation, so 800,000 sample presentations. Next to GR00T N1.7 finishing at 20,000 that looks alarming, but it is the wrong comparison: those 20,000 steps adjust a pretrained network, these 100,000 build one. Cutting the schedule short is how people talk themselves into a policy that almost works.

The step count is not the price. A 24 GB card rents for 0.30 to 0.60 USD per hour, a typical run takes 2 to 5 hours, and the whole thing lands around 1 to 3 USD. The learning rate of 1e-5 is low because the schedule is long: small, patient updates across many steps rather than large ones across few.

Twenty milliseconds, and what it unlocks

About 20 ms per action step puts ACT roughly an order of magnitude below every vision-language model here. At a 30 Hz control loop the budget per step is about 33 ms, so ACT fits inside it with room left. The arm moves continuously instead of pausing to think, which changes which tasks are feasible at all.

Latency and chunking compound usefully. Because inference is cheap you can lower nActionSteps for reactivity without the arm stalling at chunk boundaries, a trade the slower models cannot make: a 485 ms model replanning every few steps would spend most of its time thinking. ACT is also not cloud-only, and an 80 million parameter checkpoint does not need a special machine to run it.

When ACT wins, and when it is the wrong tool

The honest failure mode is that ACT does not degrade gracefully. A vision-language model handed a slightly unfamiliar scene often does something approximately sensible, because its pretraining has seen many scenes. ACT does something arbitrary. On a fixed workstation you control that is a non-issue. If your scene varies in ways you cannot enumerate, take the pretraining and accept the latency.

Where to go next

• /train/act-on-so-100 for the full run on an SO-100, chunk settings in context
• /compare/act-vs-smolvla is the specialist against the generalist
• /compare/groot-n1-7-vs-smolvla before you conclude that bigger is the answer
• /learn/train-your-first-policy for loss curves and checkpoint selection
• /live gives you a real SO-100 to drive in the browser, no signup
• /download has the desktop client with the driver built in