Why demonstrations instead of a reward
Reinforcement learning needs a reward function and a robot that can fail thousands of times, reset itself and try again. On a desk arm with a servo bus and no automatic reset, neither part is available. Imitation learning replaces both with something you can actually produce: you teleoperate the arm through the task a few dozen times, and the recorded pairs of observation and action become the training signal.
The trade is precise. You get a working policy in an afternoon without writing a reward function or a simulator. You give up any mechanism that could discover behavior you never showed. The policy has no notion of success, it has a notion of what you did.
There are no counterexamples in an imitation dataset. A failed grasp you kept is a grasp the policy is asked to reproduce, and two different strategies for the same task teach it that the task is ambiguous. That usually comes back as an average of the two that works for neither.
What that implies about data quality
- Consistency first. One approach strategy per task, held for a whole recording session, beats a session where you changed your mind halfway through.
- Coverage second. Vary object placement across the area you want the policy to handle, because it will not extrapolate far past the placements you demonstrated.
- Fixed cameras. The policy reads pixels, so nudging a camera between sessions splits one dataset into two incompatible halves.
- Review and discard. Deleting a bad episode costs seconds. Training on it costs a GPU run plus an evaluation session to find out.
- One canonical task sentence per task, because the language-conditioned models read that string as an input rather than as a label.
How much data before a run is worth starting depends on whether the model has pretraining to lean on. SmolVLA fine-tunes a pretrained checkpoint and gets somewhere useful from about 30 consistent episodes. ACT starts from random weights, and GR00T N1.7 and Pi0.5 are large enough that thin data shows: all three sit at about 50 episodes as a working floor.
/so-100/imitation-learning covers the approach in full, including the task types it is bad at, and /learn/record-your-first-dataset is the hands-on version with an arm in front of you.