Take over when the policy goes wrong, then train it on your correction.
A policy trained by behavior cloning only knows the states your demonstrations visited. DAgger closes that gap with data from the states the policy itself reaches. AY-Robots runs the whole loop on an SO-100: drive a checkpoint, take over mid-run, keep the corrected episodes, compose the mix, and continue training from the checkpoint you just drove.
- HG-DAgger, human-gated
- SO-100 / SO-101
- ACT, SmolVLA, Pi0, GR00T N1.5 / N1.7
- Intervention rate per round
Why a trained policy does something that was never demonstrated
Behavior cloning treats control as ordinary supervised learning: observation in, joint target out, fitted on the frames a human recorded. That is only sound while the robot stays on the states the human visited, and it does not. A gripper that closes forty milliseconds early nudges the cube two millimeters off its demonstrated pose. The next observation is one the training set does not contain, so the action there is an extrapolation, and the state after that sits further out still. The training distribution and the distribution the learned policy actually induces are two different things, and the second one drifts away from the first as the episode runs.
Ross, Gordon and Bagnell described exactly this reduction failure in 2011 and quantified the damage: a classifier that errs with probability e under the expert distribution can make on the order of T squared times e mistakes over a horizon of T steps under the distribution it induces itself, because one mistake produces observations the expert never generated and the errors compound. Their fix is the algorithm this page is about. Run the current policy, collect expert labels for the states it visits, aggregate them with everything collected so far, retrain, repeat. More demonstrations of the same kind do not help, because they resample the same distribution. Labels on the states the policy reaches do. The variant implemented here is human-gated (HG-DAgger, Kelly et al.): the policy keeps control until a person decides it is going wrong and takes over, so corrections are produced only where they are needed, and the arm is never asked to drive into a state the operator would not allow.
- Behavior cloning is only valid on the distribution of states it was trained on.
- The failure is self-amplifying: a small deviation produces an unfamiliar state, which produces a larger deviation.
- The remedy is not more demonstrations, it is labels on the states the policy itself reaches.
- Human-gated means the operator decides when to intervene, not an autonomy score.
Why the error compounds
Drag the horizon. Under behavior cloning the expected extra cost grows with roughly the square of the number of steps, because each mistake produces states the demonstrations never covered; an aggregation loop keeps the growth close to linear (Ross et al., 2011).
Illustrative curves from the bounds in Ross et al. (2011), not measurements from your robot. The point is the shape, not the numbers.
One DAgger round, six steps
This is the loop as the platform actually runs it, not a schematic. Every step below maps to a control in the cockpit.
Walk through the loop
The same six steps, one at a time, with what happens on the arm and in the dataset at each of them.
Run the policy and record it
Start an inference run against a checkpoint you trained. The run is recorded while it happens, with the task text of the run itself, so the frames are usable as training data afterwards instead of being a video you can only watch.
Take over and correct
The moment the arm does the wrong thing, press Take over. The runner pauses and the arm is yours. With a leader arm it first drives to the follower pose and hands control across; with keyboard or slider input, chosen at the start of the run, the takeover is manual immediately. Correct the motion, then hand control back to the policy. Frames recorded during the takeover are flagged as interventions automatically.
Triage the run
Decide per run what it was: file it as a correction, keep it as an evaluation run, or discard it. The freeze frames around a handover stay in the raw directory and never enter the dataset.
Sync the correction dataset
Corrections collect in a per-policy correction dataset and go to your bucket through the automatic cloud sync. Nothing is merged into anything at this point; the corrections are simply a dataset of their own.
Compose the mix yourself
Use Compose dataset to build the training set for the next round: the original dataset plus the corrections, with episodes chosen explicitly per source. The result is an ordinary dataset that syncs and trains like any other. Nothing is blended in behind your back, and no simulation data is added automatically.
Continue training from the checkpoint
Train the composed dataset with Continue from checkpoint instead of starting from the base model, so the round begins from the policy you just drove. Be precise about what this is: it initializes the weights from that checkpoint, it is not an optimizer resume.
What the platform takes off your hands
Every item here is a step of the loop you would otherwise build and maintain yourself.
Takeover without a leader arm
Pick keyboard or slider input at the start of the run and you can correct with a laptop alone. Keyboard nudges are clamped server-side to two degrees per joint and four on the gripper; slider targets are absolute and the server moves at most six degrees toward them per call. The clamps are enforced by the server, not by the UI, so a stuck key cannot throw the arm.
Teleoperation docsInterventions marked per frame
Every frame recorded while you hold the arm carries an intervention flag, and the action column carries the full commanded pose. You do not maintain a flag column by hand or align it with frame indices afterwards.
LeRobot dataset formatTriage: correction, evaluation, or bin
Each run gets one decision on the save card. Correction runs feed the next training round, evaluation runs stay out of training so they remain a clean measurement, and bad runs go away instead of quietly poisoning the mix.
Sessions and episodesCompose the mix explicitly
The training set for round n is assembled by you: original dataset plus corrections, episodes selected per source. No automatic blending, no hidden simulation data, and the composed result behaves like any other dataset.
DatasetsContinue from a checkpoint
Point a training run at the checkpoint you just drove instead of the base model, so each round starts where the last one ended. It initializes weights only; the optimizer state is not restored, which is why round one is worth treating as a feasibility test.
TrainingGPUs rented by the round
ACT and SmolVLA on a 4090, Pi0 and GR00T N1.5 or N1.7 on an A100 with 80 GB. You start a round, the pod comes up, checkpoints land in your bucket, and you pay for the hours the round took.
GPU pricingPlan your rounds
Intervention rate is the progress metric of the loop: corrected frames divided by frames of the run. Set where you start and how much each round buys you, and see how many rounds it takes to get where you want.
| Round | Intervention rate | Corrected frames |
|---|---|---|
| 1 | 30.0 % | 900 |
| 2 | 22.5 % | 675 |
| 3 | 16.9 % | 506 |
| 4 | 12.7 % | 380 |
| 5 | 9.5 % | 285 |
| 6 | 7.1 % | 214 |
| Σ | 2 960 | |
A model, not a forecast. Real rounds are lumpy, and a round that does not move the rate has bought you nothing; that is exactly the signal worth watching.
Building the loop yourself versus running it here
None of this is impossible by hand. It is a question of how many evenings go into plumbing instead of into rounds, and there is one row where doing it yourself is plainly the better answer.
| Step of the loop | Set up by hand | On AY-Robots |
|---|---|---|
| Taking over mid-run | A leader arm, or your own code on the servo bus. Keyboard and slider takeover, including safe limits, is something you write and then debug on real hardware. | Leader arm, keyboard, or sliders, chosen when the run starts. Angle clamps live in the server. |
| Marking interventions | You add the flag column, keep it aligned to the frame index, and re-check it every time the recording format changes. | Every frame recorded during a takeover is flagged automatically, with the commanded pose in the action column. |
| Sorting the runs | Directory conventions and shell scripts. The freeze frames around a handover are yours to find and filter out. | One decision per run on the save card. Handover frames stay in the raw directory. |
| Building the mixed dataset | Merge scripts per format version. Getting episode indices, metadata, and video references consistent is the tedious part. | Compose from original plus corrections with per-source episode selection; the result is a normal dataset. |
| Starting the next round from the last policy | You wire the checkpoint into the trainer yourself, and you can also restore the optimizer state if you want a true resume. | One field for the base checkpoint. Weights only: it initializes from the checkpoint, it is not an optimizer resume. |
| Control over the training loop | Total. Your loss, your schedule, your ablations, your instrumentation, no platform in the way. If the research question is the training loop itself, do it by hand. | A fixed path with a set list of policies and the hyperparameters the form exposes, not arbitrary code. |
The papers this is built on
Read these before you argue with the loop. Each link goes to the abstract page, not to a paywall.
- A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
Stephane Ross, Geoffrey J. Gordon, J. Andrew Bagnell · 2011 · AISTATS 2011 (PMLR 15)
The original DAgger paper: it states the compounding-error problem, gives the T-squared bound for the plain supervised approach, and proposes aggregating data from the states the learner itself visits.
- HG-DAgger: Interactive Imitation Learning with Human Experts
Michael Kelly, Chelsea Sidrane, Katherine Driggs-Campbell, Mykel J. Kochenderfer · 2019 · arXiv:1810.02890, ICRA 2019
The human-gated variant: the expert decides when to take control instead of being queried on states the policy chose, which is the mode this platform implements.
- EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning
Kunal Menda, Katherine Driggs-Campbell, Mykel J. Kochenderfer · 2019 · arXiv:1807.08364, IROS 2019
The other way to gate interventions: ensemble disagreement as a confidence signal for when the learner may act alone. Useful contrast to letting a person judge it.
- ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning
Ryan Hoque, Ashwin Balakrishna, Ellen Novoseller, Albert Wilcox, Daniel S. Brown, Ken Goldberg · 2021 · CoRL 2021
Treats human attention as the scarce resource and asks for help only on novel or risky states, which is the right frame when one person supervises the arm all afternoon.
- DART: Noise Injection for Robust Imitation Learning
Michael Laskey, Jonathan Lee, Roy Fox, Anca Dragan, Ken Goldberg · 2017 · CoRL 2017
The honest alternative: instead of correcting the policy online, perturb the demonstrations so the expert shows recovery. Worth knowing before you commit to interventions.
- Interactive Imitation Learning in Robotics: A Survey
Carlos Celemin, Rodrigo Perez-Dattari, Eugenio Chisari, Giovanni Franzese, Leandro de Souza Rosa, Ravi Prakash, Zlatan Ajanovic, Marta Ferraz, Abhinav Valada, Jens Kober · 2022 · arXiv:2211.00600
The map of the field: which forms of human feedback exist, which interfaces carry them, and where DAgger-style intervention sits among them.
- Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
Tony Z. Zhao, Vikash Kumar, Sergey Levine, Chelsea Finn · 2023 · arXiv:2304.13705
The ACT paper. Action chunking is the reason a cheap arm can be driven by an imitation policy at all, and ACT is the fastest policy to iterate a DAgger round with.
- π0: A Vision-Language-Action Flow Model for General Robot Control
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn et al. · 2024 · arXiv:2410.24164
A flow-matching VLA built on a pretrained vision-language model, and one of the checkpoints you can fine-tune here; the paper is explicit that new skills come from fine-tuning, not from the base model alone.
Questions people actually ask
Do I need a leader arm for DAgger?
No. Choose keyboard or slider input when you start the run and the takeover is manual from the first frame, driven from the browser. A leader arm is more pleasant for fine motion, and it is the mode where the arm aligns itself before handing over, but the loop runs without one.
How many DAgger rounds do I need?
There is no honest fixed number. Watch the intervention rate: if it does not fall from one round to the next, that round bought you nothing, and the problem is usually the task setup, the cameras, or the original dataset rather than the number of rounds.
Is this real DAgger or something looser?
It is HG-DAgger, the human-gated variant. Classic DAgger queries the expert on states the policy selected, including states no sane operator would let a real arm reach. Here a person decides when to intervene, and only those segments become labels.
Do I train only on the corrections?
No, and you should not. Training on corrections alone gives you a policy that only knows how to recover. The composed dataset is the original data plus the corrections, with the episodes from each source chosen explicitly.
Does continuing from a checkpoint resume the training run?
It initializes the weights from that checkpoint. The optimizer state is not restored, so it is not a resume in the strict sense. In practice the weights are what carries the learned behavior across a round, but it is worth knowing which of the two you are getting.
How is the intervention rate calculated?
Frames flagged as intervention divided by the total frames of the run. It is shown in the takeover status, and it is the one number worth writing down per round next to the checkpoint you drove and the mix you trained.
Which policies can I run this loop with?
ACT, SmolVLA, Pi0, and GR00T N1.5 or N1.7. The loop itself is policy-agnostic because it only produces datasets and checkpoints; the practical difference is how long a round takes and which GPU it needs.
Can I do this without owning a robot?
You can record and train without one by buying datasets on the marketplace or commissioning operators, and you can drive a real SO-100 in the browser on the live page. A DAgger round is different: it needs an arm you can take over mid-run, so for the loop itself you want hardware on your own desk.
A real SO-100, live in the browser. No signup, no hardware needed.
Run your first round this week
Install the client, drive a checkpoint you already have, and take over the first time the arm reaches past the cube. That single run is a DAgger round in miniature: everything after it is composing the mix and paying for the GPU hours.