Pick a task you can actually finish

Your first task should be boring. Pick up one rigid object and place it in one location. No stacking, no tools, no multi-step sequences. You are validating a pipeline, not demonstrating ambition.

Good first task	Save for later
Pick up a cube, put it in a bowl	Stack three blocks in order
Push a block to a marked spot	Open a drawer and retrieve an item
Pick up a marker, place it in a cup	Fold a cloth

Set up the scene once and leave it

Fix your cameras so they cannot be nudged. Everything you record before you bump the tripod becomes worthless the moment you bump it, because the policy learns pixel positions.

Scene camera: the whole workspace, arm and object both visible throughout
Wrist camera if you have one: pointed where the gripper closes
Lighting: constant. Close the blinds rather than fight the sun
Background: uncluttered, and not changing between episodes

Record the first episode

1
Write the task string
One sentence, and use exactly the same sentence for every episode of this task. "pick up the red cube and place it in the bowl".
2
Reset
Arm to home, object placed. Vary the object position between episodes, but keep it inside the region you want the policy to handle.
3
Record
In the client: three-second countdown, then teleoperate. Move deliberately. Stop recording the moment the task is done. From the terminal it is the same recording, minus the countdown: you still drive the arm from the client or the web cockpit while it runs, Enter ends the episode and Ctrl-C throws it away. Pass the id of a dataset that already exists, because record appends to one rather than creating it. You create it in the web interface, either in the recording card on Manual Control or on the Datasets page, and save refuses an id with no folder behind it.
bash
```
ay-robots record --dataset red_cube_pick --task "pick up the red cube and place it in the bowl"
```
4
Review before you keep it
Play it back. Did the grasp succeed? Was the motion smooth? Was the object visible throughout? If any answer is no, discard it.

Discard failures, always

A failed grasp in your training set is not a neutral data point, it actively teaches the policy to fail. Reviewing costs seconds. Training on bad data costs hours and then lies to you about why the policy does not work.

Get to fifty

Repeat until you have fifty kept episodes. Vary the object position across the reachable area. Do not vary your strategy: approach the same way every time, or you are building two datasets that disagree with each other.

bash

# Record a batch without re-typing the task each time
ay-robots record --dataset red_cube_pick --task "pick up the red cube and place it in the bowl" --episodes 10

# See where you stand
ay-robots datasets list --local

Take breaks

Demonstration quality drops measurably when you are bored. Three sessions of twenty episodes produce better data than one session of sixty, even though the count is the same.

Upload

Recordings are written locally first. When you are satisfied with the set, upload it. Nothing leaves your machine before that point. Upload takes the dataset id, the same id that datasets list --local prints and the name of the folder it recorded into, not a path to that folder.

bash

ay-robots datasets upload red_cube_pick --name "SO-100 red cube pick v1"

Check the dataset version before you train GR00T

GR00T will not load a v3.0 dataset. There is no version setting for you to get wrong: the client writes v2.0, which is a v2.x layout like v2.1, and that is the layout GR00T reads. The version worth checking is one on a dataset that came from somewhere else, so read codebase_version in meta/info.json before you queue a run on data you did not record yourself.

Was this page helpful?

SO-100 getting started: from the box to the first movement

Train your first policy