How to train ACT on LeKiwi

What the base changes

LeKiwi is the SO-100 arm, Feetech STS3215 servos and all, mounted on a three-wheel holonomic base. The arm contributes its five joints plus the gripper. The base contributes its own commands on top of that, so the action vector the policy learns is wider than the six values a desk-mounted SO-100 produces. Check the feature schema in the dataset metadata if you want the exact width for your build, because it depends on how your base is wired and recorded.

ACT itself does not object to the extra dimensions. It learns whatever the dataset declares, and a wider action vector is a slightly larger output layer and nothing more. The cost is not architectural, it is on you: every added dimension is one more thing your demonstrations have to be consistent about, and a from-scratch model has no prior telling it that a wobble in your driving was unintentional.

Chunked actions over a moving base

ACT predicts a block of future actions in one forward pass and executes part of it before looking again. On a static arm the benefit is clear: fewer prediction points across an episode means fewer opportunities for small errors to accumulate into a state no demonstration ever visited, and actions from a single plan agree with each other, which is why the motion looks continuous rather than corrected. On a mobile base the same mechanism carries an extra risk, and it is worth understanding before you deploy rather than after.

Two reasons. First, the base integrates. A joint command that is slightly off leaves the joint slightly off; a base velocity that is slightly off leaves the whole robot progressively further from where the plan assumed it would be, and the error grows for as long as the chunk runs. Second, the observation expires faster. When the base drives, the camera view changes, so the frame that produced the chunk describes a scene the robot has already left. On a desk that frame stays valid for the whole chunk. On LeKiwi it does not.

The practical consequence is a setting, not a redesign. nActionSteps controls how much of each prediction gets executed before the policy replans, and it defaults to 100, which at 30 fps is a little over three seconds. Three seconds of open-loop driving is a long way. For any phase of the task where the base moves, start at 20 to 30 and accept the slightly rougher joins between chunks as the price of a policy that looks at the world several times a second.

Leave chunkSize at 100 while you do this. The model should still learn the long-horizon structure of the task; you are only choosing how much of each prediction to trust. Since nActionSteps is a runtime setting, all of these values can be tested against one trained checkpoint in a single session.

Prerequisites

• A LeKiwi with the arm calibrated and the base drive tested independently of any policy
• Both supplies correct and labeled, 7.4 V for the arm and 12 V for the base
• The desktop client paired to your account, which carries the driver and the recording path
• Cameras mounted rigidly to the robot, since anything that shifts while driving changes the observation in a way the policy cannot distinguish from motion
• At least 50 reviewed episodes of one task, base motion included, in LeRobot v2.1
• A clear floor. Evaluation involves a robot that can drive into things

ay-robots devices

# SERIAL PORTS
#   /dev/ttyACM0   LeKiwi arm    (6 servos responding)
#   /dev/ttyACM1   LeKiwi base   (3 wheel servos responding)
# CAMERAS
#   0  Onboard scene camera   1280x720
#   1  Wrist camera           1280x720
#
# How the base enumerates depends on how yours is wired. What matters is that
# every arm servo and every wheel answers before you record an episode.

ay-robots calibrate

Recording a dataset a from-scratch model can use

The general ACT rule is consistency over volume, and on LeKiwi it applies to your driving as much as to your grasping. Two hundred episodes in which you approached the table along a slightly different arc each time is a worse dataset than sixty in which you always came in the same way. The model is fitting your demonstrations and nothing else, so an inconsistent approach path is not variety, it is noise you paid to record.

1. Drive the same way every episode. Same approach arc, same speed, same stopping distance from the table.
2. Vary the starting position of the robot, not the path it takes from there. Start pose variation is what teaches it to handle a new starting point; path variation just averages.
3. Decide once whether the base moves during manipulation or parks first, and never mix the two within one dataset. Those are two different tasks to a model with no language input.
4. Discard episodes where you corrected the base mid-approach. A correction is a demonstration of driving badly and then fixing it, which is not the behavior you want copied.
5. Keep the floor and the lighting as constant as the scene, since the onboard camera sees both while driving.
6. 50 episodes is the floor. Mobile tasks generally want more than static ones because the space of starting positions is larger.

Starting the run

1. 1
   Upload and check the action width

   After processing, open the dataset preview and confirm the action dimensions include the base and not only the six arm actuators. A dataset that silently recorded arm-only data trains a policy that will never drive.

2. 2
   Open Training and select the dataset

   The form reads the feature schema from the dataset, so the wider action vector needs no configuration on your part. It just has to be in the data.

3. 3
   Choose ACT

   The lerobot parameter set appears, plus chunk size and nActionSteps, which no other model on the platform exposes.

4. 4
   Set nActionSteps before you start, not after

   For a task with base motion, 25 is a better first value than the default 100. Leave chunk size at 100 so the model still learns the full-length structure of the motion.

5. 5
   Keep the rest of the defaults and add a seed

   Batch 8, learning rate 1e-5, 100 000 steps, gradient accumulation 1. On a mobile platform you will be comparing runs more than usual, so the seed is not optional in practice.

6. 6
   Start the run and check that loss moves

   The pool rents a 24 GB card by VRAM. Watch the first few hundred steps: a curve that never moves usually means the dataset is not what you think it is, which on a mobile build is more likely than on a desk arm.

Defaults for a LeKiwi run

Cost and duration

LeKiwi does not change the tier. ACT trains on a 24 GB card that the pool rents from the spot market by VRAM, so a full 100 000 step run on 50 episodes takes 2 to 5 hours at 0.30 to 0.60 USD per hour, which is about 1 to 3 USD. The wider action vector adds a negligible amount of compute compared with the camera streams, so a mobile dataset trains at essentially the same price as a desk one.

That price is what makes the mobile workflow tolerable. Mobile tasks need more evaluation than static ones, and evaluation on hardware is where the real time goes. Being able to retrain overnight for a few dollars, rather than budgeting an A100 at 1.20 to 2.00 USD per hour, is the difference between iterating and rationing.

Deployment and evaluation on the floor

Inference runs locally at about 20 ms per action step, well inside a 30 fps loop, so the policy is never the reason the base reacts late. Evaluation, though, is a physically larger exercise than it is with a bolted-down arm: the robot has a reachable area measured in meters and a policy that has committed to a plan.

1. 1
   Reproduce the recorded environment

   Same floor, same lighting, same table position. The onboard camera sees all three while driving, and the policy is matching pixels.

2. 2
   First attempt from the exact recorded start

   If it cannot do the run it was trained on, nothing further you measure will mean anything.

3. 3
   Twenty attempts from varied starting positions

   Spread them over the region you demonstrated, note each outcome, and separate base failures from arm failures in your notes. They have different fixes.

4. 4
   Re-run the same twenty at nActionSteps 25

   Same checkpoint, no retraining. On mobile tasks this single change is often worth more than any hyperparameter you could have set before training.

5. 5
   Record for the failure you actually saw

   If it fails when approaching from the left, twenty new episodes approaching from the left. Targeted episodes beat generic ones at every dataset size.

Troubleshooting

When ACT fits a mobile manipulator, and when it does not

ACT is a good fit for a repeated LeKiwi route: the same approach to the same station, the same pick, run over and over, where 20 ms inference keeps the base responsive and chunking keeps the motion smooth. It is a poor fit when the robot has to navigate somewhere it has not been demonstrated, when you want to redirect it with a sentence, or when the object could be anywhere in a room. None of that is in the model, because nothing was in the model before your dataset. The policy facts are at /policies/act, the comparison with the language-conditioned option on the same GPU tier is at /compare/act-vs-smolvla, and the recording fundamentals that all of this rests on are at /learn/record-your-first-dataset.