What one episode actually contains
An episode is a single, complete attempt at a task, from a defined starting state to a success condition. For the SO-100 that means a synchronised stream of joint positions, gripper state and camera frames, plus one natural-language description of what you were trying to do.
| Stream | Typical rate | Why it matters |
|---|---|---|
| Joint positions (6 values) | 30 Hz | The action signal the policy learns to reproduce |
| Gripper state | 30 Hz | Often the difference between grasp and near-miss |
| Wrist camera | 30 fps | Close-range detail during the grasp |
| Scene camera | 30 fps | Object location and approach path |
| Task string | once per episode | Conditions language-driven policies |
A 20 Hz recording where actions and images line up trains better than a 60 Hz recording where the camera lags by 80 ms. The client timestamps every stream on arrival and drops frames it cannot align, rather than silently interpolating.
How many episodes do you need
There is no universal number, but the practical thresholds below hold up well for tabletop manipulation with a single SO-100 and two cameras. They assume you vary object placement between episodes, which most beginners do not do enough.
| Episodes | What you can realistically expect |
|---|---|
| 10 to 20 | Sanity check only. The pipeline works end to end, the policy does not. |
| 50 | A policy that succeeds from starting positions close to those you demonstrated. |
| 100 to 200 | Reasonable robustness across the workspace for one task. |
| 500+ | Tolerance for lighting changes, distractor objects and moderate variation in the object itself. |
Inconsistent demonstrations teach the policy that the task is ambiguous. If you change your approach strategy halfway through a recording session, you have effectively built two small datasets instead of one large one.
The five mistakes that quietly ruin a dataset
- Never resetting the scene the same way. If the cube starts somewhere different every time and you never demonstrate the same approach twice, the policy has nothing to generalise from.
- Recording failures without marking them. A failed grasp inside your training set teaches the policy to fail. Review and discard, or label them explicitly.
- Moving the camera between sessions. The policy learns pixel-to-action mappings. Nudging the tripod invalidates everything recorded before it.
- Changing lighting mid-session. Afternoon sun through a window is a different dataset from ceiling light at night.
- Teleoperating faster than you can be precise. Jerky demonstrations produce jerky policies, and the SO-100 servos will not thank you for it.
The recording loop in the client
- 1Describe the task
One clear sentence, phrased the same way every time for the same task. This string is what language-conditioned policies attend to. "pick up the red cube and place it in the bowl" is good, "test 4" is not.
- 2Reset the scene
Move the arm home and place the objects. The camera preview is there so you can confirm framing before you commit, not after.
- 3Record
After a three-second countdown, teleoperate the arm through the task. Deliberate, smooth movements. Stop as soon as the task is complete rather than idling in frame.
- 4Review
Play the episode back before you keep it. This is the step people skip and later regret. Discarding a bad episode costs seconds, training on it costs hours.
- 5Upload
The episode is written locally first and uploaded when you choose. Nothing leaves your machine until you say so.
Frequently asked questions
Do I need two cameras?▾
One works, two is noticeably better. A wrist camera gives the policy close-range detail during the grasp that a scene camera physically cannot see once the gripper occludes the object.
Can I merge recordings from two different SO-100 arms?▾
Yes, but calibrate both first and expect a small drop in performance. Servo zero points differ between physical units, so the same joint value means a slightly different pose on each arm.
How large is an episode on disk?▾
Roughly 5 to 15 MB for a 10 to 20 second episode with two cameras at 30 fps, depending on scene complexity. Budget a few gigabytes for a serious dataset.
Can I record without an internet connection?▾
Yes. Recording is entirely local. You need connectivity for the initial pairing and for uploading, not for the recording itself.