The SO-100 and the LeRobot dataset format

Why the format matters more than it sounds

Every research group used to invent its own recording layout, which meant every dataset needed a bespoke loader. LeRobot, maintained by Hugging Face, ended most of that. Recording into it means you can hand your data to a policy trainer you did not write, and it will load.

What the layout looks like

A LeRobot dataset is a directory of Parquet files for the numeric streams, encoded video for the camera streams, and JSON metadata describing what the columns mean. A fresh recording is not quite that shape yet. The Parquet files and the meta/ files are there from the first saved episode, but each camera keeps a directory of numbered JPEGs per episode instead of a video, and there is no stats.json. The export step re-encodes each of those directories into one MP4 per episode and computes the per-feature statistics used to scale the data, which is what produces the layout below. There is exactly one place that starts it: the Upload step at the end of the recording wizard in the desktop client, the step that offers to push the episode to a Hugging Face repository. It is not a terminal command, and the dataset pages in the dashboard do not offer it.

my_so100_pick/
├── meta/
│   ├── info.json          # fps, robot type, feature schema
│   ├── episodes.jsonl     # one line per episode, with its task string
│   ├── tasks.jsonl        # the task strings, indexed
│   └── stats.json         # per-feature mean/std, used for normalisation
├── data/
│   └── chunk-000/
│       └── episode_000000.parquet   # joint states + actions per frame
└── videos/
    └── chunk-000/
        ├── observation.images.wrist/episode_000000.mp4
        └── observation.images.scene/episode_000000.mp4

Uploading does not convert anything, it packs the recording folder as it stands, which is why you can find JPEG directories inside a dataset that already reached the platform. Exporting first does not change that. The export leaves the recording untouched and writes its converted copy into a separate exports directory, and the upload keeps reading the recording. They also point at different places: the export runs behind the Upload step at the end of the recording wizard in the desktop client, so it is the route to a Hugging Face repository rather than to the copy on the platform, while ay-robots datasets upload is the route to the platform and leaves Hugging Face out of it.

Version 2.x versus 3.0

LeRobot v3.0 reorganised how episodes are chunked and how metadata is stored, mainly to handle datasets far larger than a single workstation records. It is the better format going forward. The catch is that the training ecosystem moved more slowly than the format did.

Recording here stamps v2.0, so codebase_version in meta/info.json reads v2.0 on a dataset you made yourself. That changes nothing on this page. v2.0 and v2.1 are two releases of the same layout, both load, and the split that decides whether a run starts is v2.x against v3.0.

The platform reads both versions for dataset preview and archiving, so you can inspect either. The constraint is on the training side, not on ingest.

Moving datasets between your machine and the platform

Recording writes into the local datasets directory, one folder per dataset. The CLI moves those folders to the platform and back, but the two directions do not take the same kind of id, and that is where people get stuck. Upload takes the local id: the id that datasets list --local prints, which is also the folder name. Everything outside letters, digits, underscore and hyphen is replaced before the backend looks that folder up, so a path such as ./my_so100_pick never resolves to anything. Download takes the id the platform gave the dataset when it arrived there, which you read from datasets list without --local. A local id will not match anything in the cloud listing, and a cloud id is not a folder name.

# What is on this machine
ay-robots datasets list --local

# Push one up, by its local id
ay-robots datasets upload my_so100_pick

# What is on the platform, with the ids the platform assigned
ay-robots datasets list

# Pull one back down, by its cloud id
ay-robots datasets download <cloud-dataset-id>

A pull does not restore the folder it came from. It names the new local folder after the cloud dataset name, with the same replacement of everything outside letters, digits, underscore and hyphen, and appends a numbered suffix when that name is taken. Pulling back something you pushed from this machine therefore lands next to the original as my_so100_pick-2 instead of overwriting it.

Frequently asked questions