Working with datasets

The LeRobot dataset format

AY-Robots stores all demonstration data in the LeRobot dataset format maintained by Hugging Face. The practical benefit is portability: a dataset recorded here loads directly into LeRobot-based training code, and datasets recorded elsewhere in the same format can be uploaded and used on the platform without conversion.

A dataset is a collection of episodes. One episode is one complete attempt at a task, from a defined starting state to the point where you stopped recording. On disk, an episode is split across three kinds of files: a Parquet file with the numeric streams, one video file per camera, and metadata that ties everything together.

The feature schema in info.json is what training code reads to understand your data: which observation keys exist, their shapes, and the recording rate. Two datasets are only compatible if their schemas agree, which is exactly what the merge validation checks.

Joint values are percentages, not degrees

All joint values in AY-Robots datasets are LeRobot-normalized: each value ranges from -100 to 100 and represents a percentage of that joint's calibrated range, not an angle in degrees. A reading of 0 is the center of the calibrated range, -100 and 100 are its limits.

Cloud datasets in the dashboard

Dashboard > Datasets lists your cloud datasets with episode counts, file sizes, and collection status. Datasets are created automatically when a teleoperation session records data, and you can also create one explicitly and fill it by uploading recordings from the desktop client.

• Create: new datasets appear from sessions automatically, or create an empty one and upload into it.
• Rename: change the display name at any time; the change does not touch the data itself.
• Delete: removes the dataset and all of its episodes and files. Deletion is permanent, so download an archive first if you might need the data again.

The episode browser

Opening a dataset takes you to the episode browser. Each episode has a video player for its camera streams and synchronized joint charts that plot the normalized joint values over the episode. Scrubbing the video moves the chart cursor with it, which makes it quick to spot problems: a jerky approach shows up as spikes in the charts, a failed grasp is visible in the gripper channel before you even watch the video.

Episode labels: Success, Recovery, Failure

Every episode can carry one of three labels. Success marks a clean demonstration of the task. Recovery marks an episode where something went wrong mid-attempt and you corrected it; these are valuable because they teach a policy how to get back on track. Failure marks episodes that did not reach the goal and should usually be kept out of training. Labeling as you review is much cheaper than debugging a policy that quietly learned from bad demonstrations.

Merging datasets

Merging combines several recordings into one larger dataset, for example a week of sessions on the same task or recordings from two calibrated arms. The merge takes multiple tar.gz archives, each containing a LeRobot dataset, and produces a single dataset with re-indexed episodes.

1. 1
   Prepare the archives

   Each input must be a complete LeRobot dataset packed as a tar.gz archive, including its meta directory. Partial archives without info.json cannot be validated and are rejected.

2. 2
   Upload multiple archives

   In Dashboard > Datasets, start a merge and select all tar.gz archives you want to combine in one step.

3. 3
   Validation

   The platform checks that fps, the feature schema, and robot_type match across all inputs. If any of the three differ, the merge is rejected with the mismatch named, because a combined dataset with inconsistent streams would silently corrupt training.

4. 4
   Review the result

   The merged dataset appears in your list with the combined episode count. Open the episode browser and spot-check a few episodes from each source before you train on it.

The three compatibility rules are strict on purpose. Frame rate mismatches change the meaning of every timestep, schema mismatches break loaders outright, and mixing robot types produces a dataset no single policy can act on. If you need to combine data from different arms of the same model, that is fine: robot_type refers to the model, not the physical unit.

The dataset marketplace

The marketplace lets you buy datasets recorded by others and sell your own. Purchased datasets arrive in your dashboard in the same LeRobot layout as your own recordings, so they work with the episode browser, merging, and training without any extra steps.

If you sell, curate before you list. Review the episodes, label them honestly, and make sure the task strings describe what actually happens in the recordings. A buyer evaluating your dataset sees the same episode browser you do, and datasets with clean labels and consistent demonstrations are worth more than raw dumps of everything you ever recorded.

Frequently asked questions