A robot arm beside a stylised representation of training data being collected and structured
DatasetsMarketplaceLeRobotSO-100Data Collection

From First Recording to a Listed LeRobot Dataset on the Marketplace

AY-Robots ResearchAugust 29, 202628 min read

The complete path from an arm on the desk to a dataset other people can download: pairing, calibration, what the recorder writes to disk, and the signals that separate a usable recording from an unusable one. Then the upload, the eleven automatic checks a listing has to pass before it goes live, and the fields that decide whether anyone ever finds it. Every refusal along the way is quoted as the software actually words it.

A dataset that sells is not a different object from a dataset that trains. It is the same recording with two extra properties: it survives an automatic check over the archive, and somebody can find it. Both are decided long before the listing form opens - at the moment the recorder writes its first info.json, and at the moment you type, or fail to type, the task string. What follows is the whole path from an arm on the desk to a live listing, in the order the software actually takes it.

The path in short
  • Pairing is one 8-character code with a 15-minute window; that code then stays on the machine as the robot's permanent credential.
  • Calibration here is a servo zero point. Recalibrating in the middle of a data collection splits the dataset into two incompatible halves.
  • Recording opens a buffer - the arm is driven by hand, by a leader arm, or from the cockpit. The command does not move anything.
  • The recorder writes LeRobot v2.0 with per-frame JPEGs, timestamps on the 1/fps grid, and a raw/ sidecar that carries the measured rate and a quality balance for the episode.
  • Only a dataset that sits in the account as an archive can be listed, and publishing freezes a private copy before any check runs.
  • Eleven automatic checks decide whether the listing goes live; the nine fields you type decide whether anyone ever finds it.
  • A wrong task string can be rewritten after the recording. A camera set that changed mid-collection, and a second calibration frame, cannot.

A listing binds to one archived dataset and nothing else

The listing wizard at /dashboard/client/marketplace/new starts with a dataset picker, and that is not decoration: a listing row carries a dataset id, and publishing refuses anything whose dataset is not in status archived. The picker offers three sources - Hugging Face, ay-robots and Upload. The middle tab lists what the account already holds, the third one puts a new archive there, and the Hugging Face tab is a view rather than an import: its own help text says that a repository on the Hub has to be uploaded once before it becomes selectable.

Three shapes of a dataset can exist in the object store, and only two of them can be frozen into a sale copy. A full upload leaves a single tarball. The delta upload that a paired robot uses deletes that tarball on purpose and keeps the unpacked tree as the source of truth - that case is packed on demand at publish time. A dataset that only exists on Hugging Face has no object to freeze at all, and publishing answers with a refusal that names the missing file.

The picker greys the dataset outWhat it meansWay out
listedAn active listing already exists for this dataset. A partial unique index in SQL allows exactly one listing per dataset in draft, checking, live or failed.Edit the existing listing, or delist it first.
purchasedThe row came from a marketplace purchase (source marketplace_purchase). Bought datasets must never be re-listed.None. Sell your own recordings.
not-archivedThe upload has not finished, so there is nothing to freeze.Wait for the upload, or upload the archive again.

Three refusals guard the creation step itself, and they are worth recognising by their wording rather than by their HTTP code. "Dataset has no archive yet - upload it before listing" means the row exists but its status is not archived. "Purchased datasets cannot be listed" is the check on the source field. "This dataset already has an active listing" arrives twice over: once as a friendly pre-check and once from the partial unique index in SQL, so two browser tabs racing each other still end up with one listing. Both write paths are rate limited per account - ten listings created per hour, six publish attempts per hour. The publish limit is the one people hit, by re-pressing the button while a check is still running; the wizard is already polling and will tell you on its own.

Pairing the arm is one code, and the code becomes the credential

  1. 1
    Ask for a code

    The robot wizard in the dashboard posts a robot name and robot type and gets an eight-character code back. The alphabet deliberately leaves out 0, O, I and 1 so it can be read aloud, and it expires after fifteen minutes.

    text
    POST /api/robot-pairing
    { "robotName": "desk-arm", "robotType": "so100" }
    -> { pairingId, code, expiresAt }   # expiresAt = now + 15 min
  2. 2
    Redeem it on the machine the arm is plugged into

    The robot host posts the code together with a hardware id. The answer carries the robot id and an auth token - and that token IS the code. The one-time code becomes the permanent credential of that robot, which is why it is worth treating like a password rather than like a PIN.

    text
    POST /api/robot-pairing/redeem
    { "code": "...", "hardwareId": "..." }
    -> { robotId, robotName, authToken }
  3. 3
    Or let the CLI do it

    One command on the machine with the arm. It writes config/pairing.json, the same file the desktop client reads, so both share one pairing. From then on the backend sends a heartbeat every 30 seconds and the robot shows up as online in the dashboard.

    bash
    ay-robots pair ABCD2345
    ay-robots status        # backend up? machine paired?
    ay-robots devices       # serial ports and cameras the backend can see

If the browser and the backend happen to run on the same computer, the wizard tries the local backend on port 8000 by itself and pairs without you copying anything. The visible code block is the fallback for a remote host - a Raspberry Pi next to the arm, most typically, and on a Pi it is the route that works, because the installer aborts on Linux ARM64: there is no ARM64 Linux build of the CLI, and none is on the way. ay-robots unpair removes the pairing again.

Calibration here is a zero point, not a camera model

Nothing in this step involves a checkerboard or a camera. ay-robots calibrate asks you to guide every joint into the neutral pose and press Enter. It then reads the current position of each of the six servos and stores it as an offset from the servo mid-point 2048. That is the whole of it, and it is what the term means everywhere else on this site too - see calibration. The other procedure that goes by the same word - putting camera and arm into one frame, so that a point in the image becomes a point the arm can drive to - is a different job entirely, and it is in hand-eye calibration without a checkerboard.

Never recalibrate in the middle of a data collection

The gripper channel is reported relative to the calibrated home, so a new zero point shifts the whole range. A dataset that mixes two calibration frames - one set of episodes at roughly +1 to +69 degrees, another at -152 to -86 - has an unusable normalisation statistic on the gripper channel, and no amount of training fixes that. If you must recalibrate, everything recorded before it is a separate dataset from everything recorded after.

Recording opens a buffer; your hands move the arm

There is one recording implementation and three ways to reach it. The CLI is explicit about this: it has no teleoperation loop of its own, and starts, stops and saves only, so that a second, divergent recording path cannot come into existence.

SurfaceWhat it sendsWho drives the arm
Desktop client / web cockpitrecording/start, then stop, then save or discardLeader arm mirror, keyboard or sliders in the same window
CLIay-robots record - the same four calls in a loop over --episodesYou, in the client or the cockpit, while the recording runs
Local MCP agentstart_recording, stop_recording (save true or false), get_recording_statusStill you. The tool description says it plainly: the agent only opens the buffer
A control interface for the arm, showing three live camera feeds, a dataset selector with start and record buttons, and a control panel with keyboard, mirror and slider tabs
The three drive modes from the table above are the three tabs on the right. The robot driver ships inside the application, which is the reason recording needs no Python environment on the machine with the arm.
bash
ay-robots record \
  --task "pick up the red cube and place it in the bowl" \
  --dataset my_so100_pick \
  --episodes 20 \
  --cameras wrist,top
Enter ends the current episode and saves it. Ctrl-C stops and discards the episode in progress rather than saving half of one.

The loop is driven by the Enter key, not by a timer. Per episode the command opens the buffer, prints one line to standard error and waits. Enter stops the episode and saves it; Ctrl-C stops it and throws it away, and the process leaves with exit code 130, so a shell script can tell an abort from a failure. Everything the command prints on success goes to standard output as JSON, which is what makes it scriptable at all - and with the global --json flag the human-readable commands report the same way.

text
episode 1/20: recording, press Enter to stop      # stderr, once per episode
aborted, episode discarded                        # Ctrl-C, exit code 130
error: <message>                                  # any failure, stderr, exit code 1

Two details of that command bite people. The default for --dataset is the literal name local, which resolves to a folder of that name inside the local dataset root - it is a dataset name, not a collection. And recording into a dataset that does not exist has been refused with a 409 since August rather than silently creating one, because a typo used to surface only at save time, when the episode was already gone. You create it in the web interface, either in the recording card on Manual Control or on the Datasets page.

The task string is the field most often thrown away

The cockpit's control tab opens every recording with the literal task text "teleoperation" and takes the real sentence from the description box in its save dialog afterwards. The CLI, the desktop client and the local MCP server's start_recording require the task up front instead. Leave that box empty and you end up with episodes whose only task text is "teleoperation" - which is worth nothing to a language-conditioned policy, and which was one of three faults in a demonstration set that cost 171 inference episodes in August. The older web recording dialogs have no description box at all and send only the dataset id on save, so there the text stays "teleoperation" and nothing in that dialog can change it later; record from the control tab or the CLI when the task string matters. Write one sentence, and write the same sentence for every episode of the same task.

A wrong task string is repairable after the fact, which is worth knowing before anyone throws a recording away over it. The recording backend has a labelling call per episode, and it takes a scope. With scope "dataset" - the common case, one sentence that was wrong for every episode - it rewrites meta/tasks.jsonl, every line of meta/episodes.jsonl and every raw/ sidecar, and touches no Parquet file at all, because the task_index column keeps pointing at the same index and only the text behind that index changes. Cost: a few kilobytes. With scope "episode" one episode gets its own task index, and that does mean rewriting its task_index column - roughly 20 MB read and written for one episode. An empty text is refused rather than silently falling back to "teleoperation", anything past 500 characters is cut, and a second episode edit running at the same time gets a 409 from the mutation lock instead of a half-renumbered dataset.

Cameras follow a similar rule. Without --cameras the backend records every camera it scanned, rather than the first one it found - a default that was changed after six datasets recorded on a five-camera rig turned out to be monocular. The gripper camera always gets the name wrist and the camera pinned in the device configuration gets primary, so an unplugged and replugged USB hub cannot quietly swap the viewpoints. If the configuration lists required cameras, a missing one refuses the start outright instead of producing an episode with one image track fewer. When a camera is missing at that point, the camera-not-detected page is the shorter route than guessing.

The AY-Robots recording tutorial page listing the steps from picking a task to uploading the dataset
The scene-level side of recording - task choice, resets, lighting - is covered step by step in the recording tutorial. This article picks up where the frames hit the disk.

What actually ends up on disk

text
<dataset>/
  metadata.json
  meta/
    info.json                 codebase_version, fps, features, totals
    episodes.jsonl            one line per episode, with its length
    tasks.jsonl               one line per distinct task string
  data/chunk-000/episode_000000.parquet
  videos/chunk-000/observation.images.<cam>/episode_000000/frame_000000.jpg
  raw/                        one sidecar per episode (see below)
The raw recording layout. Images are per-frame JPEGs, not MP4 - which is why info.json carries video_path: null.

The recorder stamps codebase_version: "v2.0" and robot_type: "so-100". That single line decides more than it looks like it does. The GR00T trainers read the v2 layout and reject v3.0, while ACT, SmolVLA and Pi0.5 go through lerobot and want v3.0 - so a fresh recording is immediately trainable for one family and needs a converted copy for the other. The marketplace, deliberately, accepts both: selling is not training, and a v3.0 dataset is not worthless, it is meant for a different model. The version is carried on the listing so a buyer can see it before paying. The background is written out on the LeRobot format page, and the failure mode has its own page under dataset rejected v3.

The timestamp column deserves a warning. It is written on the i / fps grid rather than from the wall clock, because LeRobot checks every neighbouring gap against 1/fps with a tolerance of 1e-4 seconds and aborts the load otherwise. The practical consequence: the timestamps in the Parquet file are not evidence about the real capture rate, and they will not show you a dropped frame. The measured rate lives elsewhere. The first save also fixes the dataset's fps for good; a later episode whose measured rate deviates by more than 15 percent is refused rather than stamped onto the wrong grid, so that one dataset cannot end up holding two time rasters.

Four refusals between pressing record and having an episode

The recording backend answers in German, so these are the strings you will be searching for when one of them appears. Three of them stop the start; the fourth arrives at save time, when the episode has already been driven.

What you seeWhat it meansWhat to do
409 Datensatz 'X' existiert nicht - erst anlegenRecording only ever appends; it never creates. The name is resolved before the cameras are opened, so this refusal costs nothing but the attempt.Create the dataset first, in the recording card on Manual Control or on the Datasets page. The CLI cannot do it: its datasets subcommands are list, upload and download, and there is no create.
409 Zu wenig Plattenplatz fuer eine Aufnahme: N GB frei, Grenze 3.0 GBThe free-space guard. Before it refuses, it sweeps known throwaway artefacts - _quarantaene/, .broken-* and .spool-salvage-* folders - and measures again; only a genuine shortage gets the 409. The same guard runs every 60 ticks during a recording, roughly every two seconds.Delete old datasets. The limit itself is 3 GB and sits in ROBOTAPP_RECORDING_MIN_FREE_GB.
409 Aufnahme passt nicht zum Bestand von 'X': Kamerasatz [...] passt nicht zum Bestand [...]The pre-flight against the existing episodes. If the target dataset already holds one, the new recording's camera set, each camera's resolution and the fps the dataset stands on are compared with meta/info.json first.Bring the settings back in line, or record into a new dataset. This is the check that stops a five-camera rig from producing five formats in thirteen episodes.
save returns success false with: Bildrate passt nicht zum Bestand: Episode lief mit N Hz, der Datensatz 'X' steht auf M fpsThe real frame rate is only known once the episode is over, so this one cannot be caught at the start. Beyond 15 percent deviation the save is refused rather than stamped onto the wrong grid.Save into a new dataset or discard the episode. The buffer and the spool survive the refusal, so the decision is yours and not the software's.

The middle two are worth reading as a pair. Both exist because the expensive failure is not a refused episode, it is an accepted one: a recording that dies halfway through when the card fills up, or a dataset that quietly ends up holding two camera sets and cannot be trained on as a whole. The disk guard during the recording behaves accordingly - it stops with the reason disk_full and leaves the buffer and the spool in place, so the episode you already drove can still be saved once you have made room.

The AY-Robots SO-100 troubleshooting index, with symptom groups for hardware, connection, dataset, training and deployment
Twelve symptom pages in five groups, ordered by what you see rather than by component. Which is what makes the message strings above the fastest way in: search the string, not the part you suspect.

Read your own dataset before the checker does

Every saved episode gets a sidecar in raw/. It is the only place where the recording says what it thinks of itself, and it is worth reading before you build a listing on top of forty episodes.

json
{
  "episode_index": 12,
  "schema": "ayrobots-raw-v2",
  "recorded_at": "2026-08-29T11:04:57.318402",
  "measured_fps": 16.7,
  "declared_fps": 20,
  "arm_id": "robot_1",
  "arm_missing_frames": 0,
  "task_description": "pick up the red cube and place it in the bowl",
  "cameras": {
    "primary": "videos/chunk-000/observation.images.primary/episode_000012",
    "wrist":   "videos/chunk-000/observation.images.wrist/episode_000012"
  },
  "motor_order": ["shoulder_pan", "shoulder_lift", "elbow_flex",
                  "wrist_flex", "wrist_roll", "gripper"],
  "units": "degrees",
  "qa": {
    "stale_camera_frames": 0,
    "spool_drops": 0,
    "camera_stall_ticks": 0,
    "camera_stall_run_max": 0,
    "divergenz_frames": 0,
    "divergenz_gelenke": [],
    "warnungen": []
  },
  "frames": [
    {
      "t": 0.0,
      "state":  [ ... six measured joint angles ... ],
      "action": [ ... six leader commands ... ],
      "cmd":    [ ... what was actually written to the servos ... ],
      "blocked": false,
      "intervention": false
    }
  ]
}
The per-episode sidecar. measured_fps next to declared_fps is the honest pair; the qa block is the balance the recorder kept while the episode ran.

The frames array at the bottom is the part that makes this file more than a report card. Its t column is real seconds since the start of the episode, not the i / fps grid, and each row carries three vectors rather than two: the measured pose, the leader's intent, and the command that was actually written to the servos. Those three agree in a clean episode and they stop agreeing when a joint is at its limit or a safety block fired. A sidecar runs to roughly 200 kilobytes per episode, which is why the episode list in the interface reads a review status out of meta/episodes.jsonl rather than opening all of them. The reason the file exists at all is that the LeRobot export throws information away - the wall clock is gone once the timestamps sit on the grid - and the raw copy is what a rebuild at a different target rate has to read. Such a rebuild reads raw/ and nothing else, and it renumbers the episodes while it does so, which has a consequence further down for anything you want to remember about one individual episode.

SignalWhere it comes fromWhat it tells you
measured_fps far from declared_fpsWall-clock span of the episode divided by frame countThe grid in the Parquet file is a fiction relative to what really happened. Episodes recorded at different real rates should not be compared, and mixing them across datasets produces differences that come from the raster, not from the robot.
camera_stall_run_max >= 15Consecutive ticks without a fresh camera frame while the arm was movingA run of still images that the save then filled in as byte copies. Images and state/action do not line up there. Measured runs of 35 to 111 frames have occurred.
spool_dropsImage ticks lost to storage back-pressureScattered single drops are mild; a contiguous run is the stall case above.
divergenz_framesFrames where a joint tracked the leader command more than 15 degrees off for at least a secondThe action column is a promise the follower did not keep. Common when a joint is at its limit.
arm_missing_framesRows written while the recording arm could not be readThose rows hold the last good pose. A bus dropout, not a movement.

None of that is what the marketplace checks. The marketplace verifies structure; these numbers are about content, and only you can see them. The same is true of the criteria that decide whether a recording is worth training on at all. Those sit one level above the file format, next to the scene-level rules on data collection:

  • One task string across the whole dataset, and a real sentence. Several different strings, or one of "teleoperation", "task", "demo", is a hard fault, not a nuance.
  • A visible grasp in every episode: the gripper opens, closes clearly short of its minimum because an object is between the jaws, and opens again to release. An episode that starts with the object already held is a placing demonstration, not a pick-and-place one.
  • One gripper calibration frame across the dataset - see the warning above.
  • A consistent start pose. More than about a tenth of the episodes starting more than 15 degrees away from the usual pose means the arm was not homed between episodes.
  • Episode duration around the reference band of 8 to 15 seconds for a pick-and-place task, and a longest episode no more than about twice the shortest. From a single frame a policy cannot infer what speed it is supposed to be moving at.
  • A share of frames without movement well under 30 percent. Above that the policy learns that standing still is a valid answer. Trimming the start and the end of each episode is the cheapest fix there is.
  • The same camera set in every single episode. A camera that appears in 40 of 50 episodes makes the dataset unusable for training, and one camera alone means the grasp contact is not observable.
  • Fifty to a hundred and fifty episodes per task, as the reference recipes put it. Fewer is not fatal, it is just thin.

Those criteria are not folklore. They are implemented, with numbers, in the acceptance check that this stack runs over a freshly recorded dataset before training - check_demo_dataset.py - and the table below is where the line actually sits. Two levels exist: a hint, which you can decide to live with, and a hard fault, which sets the exit code to 1 and means the dataset is not worth a GPU.

SignalCounts as a hint atCounts as a hard fault at
Task stringfewer than four wordsmore than one distinct string in the dataset, or one of the four texts that describe nothing: teleoperation, task, demo, empty
Gripper start stateup to a tenth of the episodes deviatingmore than a tenth starting in the other state
A grasp in the episodegrasps but never opens again - the placing half is missingany episode with no open-to-closed transition at all
Gripper calibration framesome episodes with a stroke under 20 degreestwo frames mixed in one dataset
Start poseany episode more than 15 degrees off the medianmore than a tenth of them more than 15 degrees off
Median episode durationover 20 secondsover 30 seconds
Longest against shortest episodefactor 2factor 3
Frames without movementover 15 percentover 30 percent
Episode countunder 50no hard limit - a thin dataset is thin, not broken
Camerasonly one camera, so the grasp contact is not observablea camera present in some episodes and missing from others

The report itself is a flat list - every line begins with ok, achtung or FEHLER - and ends on one of two verdicts: "TRAINIERBAR" with a count of hints, or "NICHT TRAINIERBAR" with the number of hard faults. There is an option to write a contact sheet of the first frames alongside it, which answers the one question the numbers cannot: does the object lie free at the start, and is the gripper open. Being precise about what this is: a script in the recording backend's own source tree, not a command of the packaged CLI and not a button in the web interface. If you are working from the desktop client alone, the table above is still the checklist - you just apply it by hand.

For the structural half there is a pre-flight the local backend already answers, and it is the same one the training panels in the web interface and the cockpit's training tab use to show a green or red mark before a training run; the desktop client does not call it. It reports missing or corrupt meta/info.json, an unsupported version, zero episodes, a missing action or observation.* feature, and a Parquet file count that does not match the episode count.

bash
ay-robots start                                     # backend on 127.0.0.1:8000
curl http://127.0.0.1:8000/training/datasets/my_so100_pick/validate
Returns {ok, errors[], warnings[], info{}} and never raises. The info block carries codebase_version, robot_type, episode and frame counts, fps and the feature keys.
json
{
  "ok": false,
  "errors": [
    "Der Datensatz enthaelt 0 Episoden - es gibt nichts zu trainieren."
  ],
  "warnings": [],
  "info": {
    "codebase_version": "v2.0",
    "robot_type": "so-100",
    "total_episodes": 0,
    "total_frames": 0,
    "fps": 20,
    "total_videos": 0,
    "feature_keys": ["action", "observation.images.primary", "observation.state"],
    "parquet_files": 0
  }
}

The errors list is the whole answer; ok is just whether it is empty. The texts are fixed and short enough to recognise: "Kein LeRobot-Format erkannt: meta/info.json fehlt" for something that is not a LeRobot tree at all, "Dieser Datensatz enthaelt noch keine Aufnahmen" for a dataset that was created but never recorded into, "Pflicht-Feature 'action' fehlt in meta/info.json" and "Kein 'observation.*'-Feature gefunden" for a converted tree that lost a column, and "Keine Episoden-Parquet-Dateien unter data/chunk-*/ gefunden, obwohl total_episodes=N" for the interrupted save. A Parquet count below the episode count is a warning here rather than an error - the marketplace is stricter about that than the trainer is. One difference runs the other way and is easy to trip over: this pre-flight treats v3.0 as a hard error, because the local trainer reads only v2.0 and v2.1. The marketplace does not. A dataset can be untrainable on your own machine and perfectly sellable.

One bad episode, and the gap it must not leave

The most tempting mistake in this whole path is deleting an episode file by hand. LeRobot v2.0 builds its episode boundary table from meta/episodes.jsonl in order and then indexes into it by position. Remove episode 3 out of 15 and episode 4 reads the boundaries of episode 3, every following episode is off by one, and the last one runs off the end of the tensor. Nothing crashes and nothing warns - the loader hands out wrong slices, which is the worst thing that can happen to a training set. Marking the episode as deleted instead has exactly the same effect, because no standard loader knows the key.

So the delete call renumbers. It removes the files and pulls the remaining episodes into a gapless sequence, and the honest cost is that every following episode's Parquet file has to be rewritten - the episode index and the global row counter live inside it - which is roughly 20 MB apiece. Images and raw sidecars are only renamed, which costs nothing, and the frequent case is free: deleting the last episode, the one you just drove and did not like, needs no renumbering at all. It is written batch-wise so that a Raspberry Pi with under two gigabytes never holds a whole episode with its embedded JPEGs in memory, and it refuses outright while a recording is running - "Aufnahme läuft - Episode erst danach loeschen" - because the save path derives its next episode number from precisely those files.

If a gap already exists, repair_dataset_index.py closes it, and that is the direct answer to the episode_count check further down: it renumbers the surviving episodes into a continuous run, pulls the video folders along, rewrites the episode_index column inside each Parquet file and fixes meta/episodes.jsonl and meta/info.json including the train split. It prints what it would do and changes nothing until you pass --apply, and it must not run during a recording, because the recorder is writing the same files.

Not every flawed episode has to go, though. An episode can also be marked, with one of three states: normal, recovery for a run where something went wrong and the operator recovered it - which is not waste but an extra lesson, and gets weighted differently by some recipes - and correction. The truth of that mark lives in the raw/ sidecar, deliberately, because it is the only file that survives a rebuild from raw data; it is mirrored into meta/episodes.jsonl as ay_status so the episode list can read it with one file access, and it is kept out of meta/info.json on purpose, because that file is rewritten from scratch by every single save.

Getting the dataset into the account

Three routes end in the same row. Which one you use depends on where the recording sits and how large it is.

A paired robot schedules a push by itself. Creating a dataset already schedules one, so the platform knows the dataset before its first episode exists, and every save reschedules it. The timer is debounced by 45 seconds and refuses to fire while a recording is running or about to run - packing a multi-gigabyte tree with gzip in the middle of an episode used to cost the episode.

The queue lives on disk and is written with fsync, so a restart or a power cut costs time rather than data. The upload itself runs in its own process: as a thread inside the backend it used to be killed by the health watchdog mid-transfer, which on one August day left 2 of 14 episodes in the cloud and 16 orphaned archives in the temp directory.

What the publish check actually verifies

Pressing publish does two things before any check runs. It freezes a copy of the archive under a versioned marketplace key, and it records the size and ETag of that copy. Everything afterwards - the checks, the previews and every buyer download - runs against the frozen copy, so the content cannot be swapped after a listing has passed. If the dataset exists only as an unpacked tree, the tree is packed into that frozen copy first, in the same background run as the check rather than inside the request.

The frozen copy has a fixed address - marketplace/{listingId}/v{n}/archive.tar.gz - and the preview frames land next to it under previews/pub/ and previews/locked/, named by episode and frame number. The version number goes up with every publish, so an edited and re-published listing never overwrites the copy an earlier buyer holds. The publish request itself does not wait for any of this: it answers 202 as soon as the freeze is planned and the status is on checking, and the work runs on after the response.

Two refusals happen before any check runs, and both name a path so you can tell which of the three storage shapes you are in. A dataset whose unpacked tree is over the four-gigabyte limit is turned down at planning time - "This dataset is too large to list: N GB unpacked in " - rather than after minutes of packing something the check would then throw away. A dataset with neither a tarball nor a tree gets a message that names the archive key it looked for, the tree prefix it looked for, and the way back: upload the .tar.gz in step one of the wizard under the same name. If the packing itself fails, the failure text says which tree it was packing and adds the line that matters - the dataset itself is untouched.

Then eleven checks run over the archive in a streaming pass; the tarball is never held in memory in one piece. Every check has to pass. A single failure puts the listing into status failed with the full report attached, the wizard shows which one and why, and you can fix the fields or the archive and publish again. A run that dies mid-flight - a deploy, a crash - is retried by a sweeper up to three attempts in total.

CheckConditionWhat usually trips it
decodablegzip and tar streams readableA truncated upload, or an archive that is not gzipped
uncompressed_sizeat most 4 GB uncompressedA long collection with several cameras. Split it into task-sized datasets
archive_sizethe frozen copy is byte-identical in size to what was recorded at freezeNothing you prepare - this is a server-side integrity guard
info_jsoncodebase_version parses to v2 or v3A hand-built or converted dataset with a missing or exotic version string
episode_counttotal_episodes equals the length of the episode list; in v2 one Parquet file per episode, in v3 at least one aggregated data fileAn interrupted save: metadata says 41 episodes, 40 Parquet files exist
fpsan integer between 1 and 240A dataset whose fps was written as a float
episode_lengthsevery episode has a length greater than zeroA zero-length episode left behind by a discarded recording
total_framestotal_frames equals the sum of the episode lengths, 1 percent toleranceMetadata edited by hand, or episodes deleted from data/ without fixing meta/
camerasat least one observation.images.* camera with real JPEG framesA video-only export. See the note below - this is the one that catches people
robot_typenon-emptyA converted dataset that lost the field
durationtotal_frames divided by fps between 1 second and 24 hoursFollows from a broken fps or frame count rather than being a fault of its own
Video-only datasets cannot be listed yet

The preview is rendered from single frames with an image library and no video decoder, so the cameras check looks for per-frame JPEGs, not for MP4 files. A dataset that stores its camera tracks only as video passes every metadata check and then fails here, with a message that says how many MP4 files it found instead. This hits the official LeRobot v3.0 layout in particular: its metadata is read without trouble, its aggregated video tracks are not. A v3.0 recording that stores images rather than video runs through completely. A raw recording from this stack always does, because it writes per-frame JPEGs.

When all eleven pass, the pipeline generates the previews in a second streaming pass: three randomly chosen episodes at eight frames each as the public preview, two more at three frames each blurred as the locked teaser. The blur is applied to the pixels, not in CSS, so it cannot be undone in a browser. Only the first camera key in alphabetical order feeds the previews - one consistent viewpoint. There is no upload field for a cover image anywhere in the wizard: the thumbnail on the browse card is the first public preview frame, and a dataset with no readable frames simply has no card image.

While it runs, the wizard polls the listing every three seconds and renders the report as it fills up: a progress bar counting passed checks against the total, and one line per check carrying the pipeline's own detail text rather than a paraphrase of it. That detail is where the useful part sits. episode_count reports "info.json N episodes, M episodes.jsonl lines, K parquet files", so the three numbers that disagree are in front of you; total_frames reports the declared frame count against the sum of the episode lengths and names the one percent tolerance; cameras lists the camera keys it found and how many frames it magic-checked. The headline error stored on the listing is the first failing check formatted as "name: detail", which is the string worth copying when you ask anybody about it.

A run that dies mid-flight - a deploy, a crash - leaves the listing sitting in checking, and a sweeper picks it up every five minutes. It waits fifteen minutes before touching a stuck run, and an hour if the sale copy is still being packed out of an unpacked tree, because packing four gigabytes takes longer than the normal window and a second run would report the half-written copy as broken. After three attempts in total it gives up, sets the listing to failed with "checks timed out" and notifies you. Publishing again is the correct response; it starts a fresh version and a fresh attempt counter.

The AY-Robots glossary entry for the LeRobot dataset format, showing the on-disk layout and the v2 versus v3 distinction
The version distinction that decides which trainer can read a dataset. The marketplace accepts both; a trainer does not.

The fields that decide whether anyone finds it

Half of a listing is measured rather than typed. Episode count, frame count, fps, duration, the camera list and the LeRobot version are all written onto the listing by the check pipeline, read out of the archive. What you fill in is the other half - and every one of those fields is wired to a filter or a search on the marketplace, which is the whole reason completeness pays.

FieldLimitWhere it acts
Title1 to 120 charactersFull-text search, and the task filter also matches against it
Taskup to 2000 charactersThe task filter - one sentence saying what the robot does. This is the field buyers actually search on
Descriptionup to 5000 charactersFull-text search, the detail page, and the meta description and structured data for search engines
Robot arm typeup to 100 charactersThe robot filter, matched by family so that so100, so-100 and so100_follower all land together. Prefilled from the dataset, so leave it alone unless it is wrong
Categoryone of nine fixed valuesThe category filter and the category landing pages
Tagsup to 10, normalised to lowercase and hyphensExact tag matching in the task filter. Free text is reformed rather than rejected: "Pick and Place" becomes pick-and-place
Environmentup to 2000 charactersIts own substring filter - table top, kitchen, workbench
Objectsup to 20 entries, 60 characters eachShown on the detail page. Not a filter today
Recording setupup to 2000 charactersShown on the detail page: cameras, frame rate, teleoperation or script

Environment, objects and recording setup used to sit behind a collapsed section and were therefore almost always empty. They now stand open, directly under the description and deliberately above the price field - what sits below the price is what nobody fills in. They are optional, and they are what a buyer uses to decide whether a dataset fits their own rig at all.

The public URL is not one of the fields you control. The slug is generated on the server from the title plus a six-character random suffix, and it is never taken from the browser. A later title edit leaves it alone, which is the right trade: the URL a buyer bookmarked keeps working, and the title in the search result is free to improve.

Two more things are recorded when the listing is created. An ownership attestation is mandatory and is stored with a timestamp, the user id and the IP - you are confirming that you hold the rights and that no third-party personal data is in the recording. And a licence is attached, defaulting to CC-BY-4.0, with CC-BY-NC-4.0, CC0-1.0, MIT and a custom option available.

What buyers actually filter on

The buyer's side works in brackets, and that is worth knowing while the form is still open, because a number just under a bracket edge is invisible. The minimum-episode filter offers 0, 50, 100, 200 and 500. Price splits into free, under 25, 25 to 100 and over 100 euros. Rating filters at 3, 4 and 4.5 stars. Duration is a slider in seconds, up to ten minutes. Four ready-made task chips send the literal substrings "pick and place", "sorting", "stacking" and "cleanup". Results come 24 to a page, sorted newest first unless the buyer changes it to price, rating or episode count.

And the two text searches do not look in the same places, which is the single most useful thing on this page. The search box matches title, description and summary - and not the task field. The task filter matches the task description, the title and an exact tag - and not the description. A dataset whose task sentence is only in the description is missing from every task search; one whose subject only appears in the task field is missing from the search box. Writing the same sentence into both is not redundancy here, it is the only way to be in both indexes. Tags meanwhile are reformed rather than rejected - up to ten, up to 30 characters each, lowercased, accents resolved, ß to ss, everything else to hyphens - and a tag in a script with no Latin equivalent normalises to nothing and is dropped in silence.

The recording path, already built

Pairing, joint calibration, multi-camera recording straight into the LeRobot format, and pushing a finished dataset to the cloud are shipped commands rather than steps you script yourself. What none of them can do is judge the recording for you: the automatic checks on a listing catch format and structure problems, not a camera that spent forty episodes pointing at the wrong half of the table.

Read the data-collection guide

Price, fee and payout

  • A listing is either free or priced between 1.00 and 10,000.00 EUR. The same bound is enforced in the database.
  • Dataset sales take a flat 10 percent platform fee, and the wizard shows the net figure under the price field before you publish. That is a different rate from teleoperation sessions, which carry 20 percent.
  • Free listings do not go through payment at all: an acquire call creates the purchase row, delivers the dataset and books nothing. Paid ones go through a payment intent whose amount always comes from the database, never from the browser.
  • Payouts are requested against the seller balance with an IBAN and an account holder, with a minimum of 10.00 EUR. The transfer itself is executed manually rather than automatically.
  • A listing is editable while it is draft, failed or live, and re-publishing an edited live listing keeps its original publication date rather than resetting it. Withdrawing it makes it archived if nothing was sold and delisted if something was; publishing accepts draft, failed and live, so treat a withdrawal as final rather than as a pause.

What a buyer receives, and why they cannot resell it

A purchase does not hand over a download link. It creates a dataset row in the buyer's own account, pointing at the same frozen archive - no second copy in the object store - carrying the listing's title as its name, its episode count, fps and camera list, and the source marketplace_purchase. From that moment on it behaves like any other dataset in that account: it shows up in the dataset list, in the dashboard, and in the cockpit's pull. If the buyer already owns a dataset with that name, the delivered one gets " (2)" appended, because names are the idempotency key on upload.

That source field is the whole reason the picker greys a bought dataset out. The archive it points at is somebody else's frozen sale copy, and re-listing it would offer for sale an object the seller does not control. A free listing runs through the same delivery, so a free dataset arrives in exactly the same shape as a paid one - and is just as unlistable afterwards.

Where an agent fits into this today, and where it does not

Two MCP servers exist and they do different things - the distinction is written into the code as an explicit warning against confusing them. The local one runs on the machine with the arm and can move servos. The hosted one runs on the website, knows the account, and has no hardware access at all; it says so in its own instructions. Setting the local one up, and what its nine tools can and cannot do, is written out in letting an agent drive a physically connected SO-100 over MCP.

StepLocal server (ay-robots mcp)Hosted server (/api/mcp)
Check the arm is thereget_status, list_devices, get_joint_positions-
Home the arm between episodesmove_home, move_joint - both refuse without confirm: true-
Record an episodestart_recording (task required), stop_recording (save true or false), get_recording_status-
See what was recordedlist_local_datasetslist_datasets, get_dataset - with an optional download link valid for 15 minutes
Decide which model can read it-check_dataset_compatibility, before any GPU is rented
Upload, list, price, publish--

The gap in the bottom row is real and worth naming rather than glossing over: neither server has a marketplace tool. There is no way for an agent to upload a dataset, create a listing or publish one today, and the hosted server's account tools are all read-only apart from starting and stopping training runs.

Where an agent would obviously earn its keep is the part of this article that is bookkeeping. Reading every raw/ sidecar and reporting which episodes carry a stall run. Checking that all episodes share one task string before an upload. Drafting task, description and tags from what the recording actually contains rather than from memory two weeks later. None of that exists as a feature - it is a consideration, written down here as one, and an agent can already do the first two with nothing but file access, since the sidecars are plain JSON. What it cannot do is press publish.

Operators and teleoperation

Teleoperation is how the arm moves while a recording runs, and it is also a role on the platform. The operator directory lists certified operators with a rating, a completed-session count, an hourly rate and the robot types they are certified for; a client browses them from the dashboard. A session records how long it ran, how many minutes were active, how many episodes were collected and how many megabytes of data came out of it, and it carries a platform fee of 20 percent - against the 10 percent on a dataset sale.

What is worth being precise about: a session's episode counter and a marketplace dataset are not the same object, and nothing in the platform automatically turns one into the other. A session produces episodes in a dataset on the robot host, and that dataset then travels the same path as any other - sync, archive, listing. If you are paying an operator to collect data you intend to sell, the task string, the camera set and the calibration frame have to be agreed before the session starts, because all three are decided at recording time - and of the three, only the task string can still be rewritten afterwards. The mechanics of a session are in the teleoperation documentation, the terms themselves in the glossary entries for teleoperation and leader and follower.

The AY-Robots operator recruiting page, with an SO-100 arm photographed on a desk and the note that this is the hardware operators control every day
The operator side of the same recording path. The hourly rate and the certified robot types are what a client browses; what decides whether the resulting dataset is sellable is agreed before the session, not on it.

Frequently asked questions

Can I list a dataset that only exists on Hugging Face?

Not directly. The wizard shows your Hugging Face repositories, but listing requires a finished archive in your ay-robots account, because publishing freezes a private copy of the archive and every check and every buyer download runs against that copy. A dataset that lives only on the Hub has no object to freeze, and publishing refuses with a message naming the expected file. Upload it once and it becomes selectable.

My listing failed on the cameras check but the dataset trains fine. Why?

Because the check is not about trainability, it is about the preview. Previews are built from per-frame JPEGs with an image library and no video decoder, so a dataset whose camera tracks are MP4 files fails there even though a trainer reads it happily. The message tells you how many MP4 files it found. Export with per-episode image frames, or list a recording that stores images.

Does a v3.0 dataset get rejected by the marketplace?

No. Selling is not training. The marketplace accepts v2.0, v2.1 and v3.0 and records the version on the listing so a buyer can see which model family can read it. The stricter rule belongs to the trainers: GR00T loads only the v2 layout, while ACT, SmolVLA and Pi0.5 want v3.0. The training-side check is a separate piece of code with its own policy, because it protects rented GPU time rather than the catalogue.

How many episodes should a dataset have before it is worth listing?

The reference recipes put 50 to 150 episodes per task, and fewer is thin rather than useless. There is no minimum in the checks - a single valid episode passes. What actually decides whether someone downloads it is the combination of episode count, a task description they can match to their own problem, and a camera set that fits their rig. The browse page sorts by episode count, so the number is visible without opening the listing.

I recalibrated halfway through. Is the dataset lost?

The recording is not lost, but it is two datasets. The gripper channel is reported relative to the calibrated home, so the two halves sit in different numeric ranges and a normalisation computed across both is meaningless. Split them and treat each half on its own. There is no repair for this after the fact, which is why the rule is to never recalibrate during a collection.

Can I change the price or the description after the listing is live?

Yes, while the listing is in draft, failed or live. The dataset binding and the ownership attestation are the two things fixed at creation, and so is the slug: it is generated on the server from the title plus a six-character suffix, and a later title edit does not change it, so a live URL stays a live URL. Re-publishing after an edit keeps the original publication date. Withdrawing is the one step to think about first - it leaves the listing archived or delisted, and publishing only accepts draft, failed and live.

My listing has been stuck in "checking" for half an hour. What now?

Wait a little longer before doing anything, then publish again. The check runs in the background after the publish request, and a deploy or a crash can leave it hanging. A sweeper runs every five minutes and re-runs anything that has been stuck for more than fifteen minutes - or more than an hour, if the sale copy is still being packed from an unpacked file tree, which is the normal state for a dataset that was synced from a paired robot. After three attempts it stops and marks the listing failed with "checks timed out", and you get a notification. Publishing again starts a fresh version with a fresh attempt counter.

Every episode says "teleoperation". Do I have to record the dataset again?

No. The task text can be rewritten afterwards, and for the usual case - one wrong sentence across the whole dataset - it costs a few kilobytes: the labelling call with scope "dataset" rewrites meta/tasks.jsonl, every line of meta/episodes.jsonl and every raw/ sidecar, while the task_index column in the Parquet files stays exactly as it is and simply points at a different text. Only relabelling a single episode differently from the rest needs its task_index column rewritten, around 20 MB for that one episode. What cannot be repaired afterwards is a camera set that changed mid-collection or a second gripper calibration frame - those are in the data itself, not in the metadata.

The next step depends on what you have. With an arm on the desk and no recordings yet, the recording tutorial covers the scene-level decisions this article assumes. With recordings already on disk, read one raw/ sidecar and check the task string across all episodes before you upload anything - both are cheaper now than after a listing has failed.

Ready for high-quality robotics data?

AY-Robots connects your robots to skilled operators worldwide.

Get Started