What is actually in the directory
datasets/local/my_so100_pick/
metadata.json dataset name, episode count, size on disk
meta/
info.json fps, robot type and the feature schema
episodes.jsonl one line per episode, including its task string
tasks.jsonl the task strings, indexed
data/
chunk-000/
episode_000000.parquet per-frame joint state and action
videos/
chunk-000/
observation.images.wrist/
episode_000000/
frame_000000.jpg
frame_000001.jpg
observation.images.scene/
episode_000000/
frame_000000.jpgA fresh recording is not the shape a trainer reads. The frames are still individual JPEGs and meta/ has no stats.json. Both appear in the export step, which sits at the end of the recording wizard in the desktop client, behind the button that pushes the episode to Hugging Face: it re-encodes each camera directory into one H.264 MP4 per episode and computes the normalization statistics over the Parquet columns. Uploading is not that step. An upload archives the directory as it stands, which is why you can open a dataset that already reached the platform and still find folders of JPEGs in it.
datasets/exports/my_so100_pick/
meta/
info.json
episodes.jsonl
tasks.jsonl
stats.json per-feature mean and standard deviation, used to normalize
data/
chunk-000/
episode_000000.parquet
videos/
chunk-000/
observation.images.wrist/episode_000000.mp4
observation.images.scene/episode_000000.mp4The split is deliberate. Numbers a trainer touches on every step live in Parquet, which is cheap to scan. Frames live in video, which is the only sensible way to store them. Everything a loader needs to interpret the columns lives in meta/info.json. stats.json is the part people forget: the normalization statistics are computed from your data during the export, which is one reason a checkpoint belongs to the dataset it was trained on.
Two files repay a look before you spend money on a run. meta/info.json states the frame rate and the feature schema, so it is where you confirm that both cameras actually made it into the recording and that the action dimension is the width you expect. meta/episodes.jsonl holds one line per episode with its task string, which is the fastest way to notice that two tasks ended up in one dataset or that your phrasing drifted halfway through a session.
The v2 layout against v3.0
| v2.0 and v2.1 | v3.0 | |
|---|---|---|
| Per-episode files | One Parquet file per episode | Episodes batched into larger chunks |
| Scaling | Fine at the size one arm produces | Built for corpora far larger than that |
| Preview and archive on this platform | Yes | Yes |
| Accepted by the GR00T trainers | Yes | No, rejected outright |
| Accepted by ACT, SmolVLA and Pi0.5 | Yes | Yes, lerobot 0.5.x reads it natively |
A v3.0 dataset fails a GR00T run right after upload, and the error does not name the cause in plain language. Read codebase_version in meta/info.json before you queue the run: the value is written when the dataset is created and travels with it, and uploading does not change it. A recording made here reads v2.0, which is a v2 layout, so nothing about the version has to change before a GR00T run. Only a v3.0 tree has to be converted down to v2.1 first, and doing that before you queue is cheaper than doing it after you have paid for the card.
Recording here settles the question for you. The recorder stamps v2.0 and the export leaves that value alone, so what ends up on disk is already a layout the trainers read. The scaling advantages of v3.0 only start to pay off well past the dataset size one arm and one operator produce, and the cost of guessing wrong is asymmetric: a v2 tree trains on all five models here, while a v3.0 tree has to go back to v2.1 before GR00T can read it.
Datasets move with three commands, and none of them start training. Training runs on a rented cloud GPU, from the dashboard under Training or through the start_training tool on the MCP server. Upload takes the local dataset id, which is the id field from datasets list --local and matches the folder name under the local datasets directory. A filesystem path does not work there: the backend strips everything outside letters, digits, underscore and hyphen before it looks the dataset up, so ./my_so100_pick becomes a folder name that does not exist and the upload ends in a not found. Download runs the other way and takes the id the platform assigned, which is what datasets list prints without --local.
ay-robots datasets list --local
ay-robots datasets upload my_so100_pick --name "SO-100 pick v1"
# The other direction reads the cloud listing. Download takes the id the
# platform assigned, not the local folder name.
ay-robots datasets list
ay-robots datasets download <cloud-id>/so-100/lerobot goes through the format and the version split in detail, including which conversion exists in which direction: LeRobot ships the upward path from v2.1 to v3.0, and the way back for a GR00T run is a separate script in the training server.
Where this term does the explaining: symptoms whose cause sits in exactly what it describes.