What one episode holds
- A per-frame action stream, the commands that went to the arm, six values on an SO-100 sized robot.
- A per-frame observation stream: joint state including the gripper position.
- One video per camera, aligned to those frames by timestamp rather than by frame number.
- One task string, written once and attached to the whole episode.
- Boundaries you chose: an episode begins where you reset the scene and ends where you decide the task is done.
Those boundaries are part of the data, not an administrative detail. If half your episodes end the moment the cube lands in the bowl and the other half end after five seconds of the arm hovering over it, you have taught the policy that hovering belongs to the task.
Why consistency beats count
The number people ask about first is how many episodes they need. The number that decides the outcome is how many of them agree with each other. Fifty episodes of one strategy give a policy something to imitate. Two hundred that mix two strategies give it a contradiction, and the usual result is an average that satisfies neither, executed with complete confidence.
| Model | Episodes before a run is worth starting |
|---|---|
| SmolVLA | about 30 |
| ACT | about 50 |
| GR00T N1.7 | about 50 |
| GR00T N1.5 | about 50 |
| Pi0.5 | about 50 |
These are floors, not targets, and they assume you varied object placement between episodes. A dataset where the cube starts in the same square centimeter every time reaches its floor and stops there, regardless of how many episodes you add.
Nothing in the format marks an attempt as unsuccessful. Play an episode back before you keep it, and drop the ones where you fumbled the grasp or corrected twice. Ctrl-C during recording discards the episode in progress, which is cheaper than sorting it out after the upload.
The mechanics of a recording session are in /learn/record-your-first-dataset, and /so-100/data-collection covers scene resets, camera placement and the mistakes that stay invisible until a policy fails.