How to train ACT on the SO-101

What is different, and what is not

The SO-101 is the successor to the SO-100 and the revision is in the hardware. Feetech STS3215 servos, 7.4 V, five joints plus a gripper for six actuators, LeRobot v2.1 recordings, the trainer key act: all identical. Every screen you touch and every number on this page is the same as on the SO-100 guide. The wrist is the part that was redesigned, and a redesigned wrist changes the joint trajectories your teleoperation actually produces.

For a pretrained vision-language-action model that would barely register. For ACT it is worth a paragraph, because ACT starts from random weights and has no prior about arms in general. Whatever your wrist does during a demonstration is not a variation on something the model already knows, it is the entire definition of the task as far as the model is concerned.

Do not merge SO-100 and SO-101 episodes

This is the one genuinely SO-101 specific decision in the whole workflow, and it comes up constantly because people upgrade mid-project. Servo zero points already differ between two physical units of the same model, which is why calibration exists. Add a revised wrist on top of that and the same recorded joint value corresponds to a meaningfully different pose on the two arms.

A pretrained model can absorb some of that: it has seen enough embodiments that a small systematic offset reads as noise. ACT reads it as a contradiction, because it is fitting your data and only your data. Two arms in one ACT dataset is the same failure mode as two grasping strategies in one ACT dataset, and it produces the same result, a policy that splits the difference and satisfies neither.

• Keep one dataset per physical arm, not per arm model. Two SO-101 units still deserve separate calibration and a check that they behave the same before you pool them.
• If you already merged and cannot re-record, train on the majority arm and evaluate only on that arm. A mixed policy evaluated on a mixed fleet tells you nothing about either.
• If you must pool, pool for a pretrained model rather than for ACT, and read /compare/act-vs-smolvla before you commit the GPU time.
• Recalibrate after any mechanical work on the wrist. The zero point moves and every episode you record afterward disagrees with the ones before.

Prerequisites

• A calibrated SO-101 with its 7.4 V supply and a working leader or teleoperation setup
• The desktop client paired to your account, which carries the driver for both SO-family arms
• A fixed scene camera, and a wrist camera if the revised wrist mount allows one on your build
• 50 or more reviewed episodes of one task, all recorded on this arm
• An account with billing enabled, since the run rents a GPU per hour

ay-robots devices

# SERIAL PORTS
#   /dev/cu.usbmodem58FA0821911   SO-101  (6 servos responding)
# CAMERAS
#   0  Scene camera    1920x1080
#   1  Wrist camera    1280x720

# Recalibrate after any mechanical work on the wrist
ay-robots calibrate

Recording an ACT dataset on the SO-101

The rule for ACT everywhere is consistency ahead of volume, and on the SO-101 the wrist is where consistency tends to slip. A revised joint has a different feel under teleoperation, so the first ten episodes after an upgrade are often recorded with a slightly different wrist habit than the fiftieth. Record ten, watch them back, then record the rest once your hands have settled.

1. Check the wrist camera clears the full wrist range before episode one. A cable that catches at the extreme of the new joint changes the view mid-episode, and later you will read that as a policy failure.
2. Fix the approach direction and the grasp height and hold them for the whole session.
3. Move the object between episodes, keep your technique still.
4. Review each episode and delete the failures. ACT has no prior that lets it discount a bad demonstration.
5. Stop recording the moment the task is complete, so the model does not learn to idle in frame.

Action chunking in short

ACT does not predict the next action, it predicts a block of future actions in one pass and then plays part of that block back before looking again. Fewer predictions per episode means fewer opportunities for a small error to knock the arm into a state no demonstration covered, which is the failure mode that kills naive behavior cloning. And because the actions in a block were planned together, they do not argue with each other, so the SO-101 moves in one continuous motion rather than correcting itself thirty times a second.

chunkSize is how far ahead the model predicts. nActionSteps is how much of that prediction gets executed. Both default to 100, and nActionSteps can never exceed chunkSize. Cut nActionSteps and the policy replans more often, reacts better to a scene that moved, and shows a slight twitch where one chunk hands over to the next. Leave it at 100 and you get the smoothest possible motion at the price of three seconds of blindness per prediction. It is a runtime setting, so you can test both values against the same checkpoint in the same afternoon.

Starting the run

1. 1
   Upload and verify the dataset

   Datasets, upload, wait for processing, then open the episode preview and scrub through two of them. The preview is also where you confirm the arm was recorded as an SO-101 with six action dimensions.

2. 2
   Open Training and pick the dataset

   Only one dataset per run. If you have SO-100 and SO-101 recordings in your account with similar names, check the robot type on the dataset card before you select it.

3. 3
   Select ACT

   The form redraws with the lerobot parameters plus the two chunk fields, which appear for ACT and nothing else.

4. 4
   Keep the defaults, add a seed

   Batch 8, learning rate 1e-5, 100 000 steps, gradient accumulation 1, chunk size 100, nActionSteps 100. The seed is the only field worth filling in on run one, because it makes run two comparable.

5. 5
   Name it after the arm as well as the task

   so101-cube-pick-v1 rather than cube-pick-v1. Six months in, a policy trained on the wrong arm’s data is a real and annoying way to lose an afternoon.

6. 6
   Start and watch the loss for the first thousand steps

   A curve that is flat from the first step usually means the data did not load the way you assumed. That is worth catching in two minutes rather than in four hours.

Defaults for an SO-101 run

Cost and duration

ACT asks for a 24 GB card, which the pool buys on the spot market by VRAM rather than by model name. A 100 000 step run over a 50 episode SO-101 dataset takes 2 to 5 hours at 0.30 to 0.60 USD per hour, so about 1 to 3 USD. The A100 tier that GR00T and Pi0.5 need costs 1.20 to 2.00 USD per hour instead, which is the practical reason people validate a new arm with ACT before spending anything larger on it.

Deploy and evaluate

The checkpoint runs locally in the client at roughly 20 ms per action step, comfortably inside a 30 fps loop, so the SO-101 executes the trajectory rather than stepping through it. Restore the camera positions and the lighting first: the policy is reading pixels, and a scene that differs from the recordings will produce confident motion toward the wrong place.

1. 1
   Clear the bench

   Nothing fragile inside the working envelope, and a hand near the power for the first chunk, which commits to about three seconds of motion before it reconsiders.

2. 2
   Twenty attempts, varied placement

   Spread the object over the region you demonstrated and record every outcome as it happens rather than from memory.

3. 3
   Repeat with a second checkpoint

   One from around 70 percent of the run, one from the end, same twenty positions in the same order.

4. 4
   Then try nActionSteps at 25

   Same checkpoint, no retraining. If the success rate rises, your scene is less static than you assumed.

5. 5
   Record for the cluster, not in general

   Twenty targeted episodes covering the situation that failed beat two hundred more of what already works.

Troubleshooting on the SO-101

Is ACT the right model for this arm

The SO-101 does not change the answer, the task does. Pick ACT when the task is fixed and repetitive, when the scene resets the same way, and when you care that the motion looks like motion. Pick a pretrained model when object placement is genuinely open, when you want to say what you want in words, or when the policy has to handle objects that appear in no episode. The numbers behind that choice sit at /policies/act, the direct comparison at /compare/act-vs-smolvla, and if the dataset does not exist yet, /learn/record-your-first-dataset comes first.