Running a DAgger Loop on an SO-100 with a VLA Policy

Your policy runs. It reaches for the cube, closes the gripper a centimetre too early, and keeps going as if it had it. Nothing errors, and no amount of staring at the training loss explains it. The fix is not another 20 000 gradient steps on the same demonstrations. It is to put your hand back on the arm exactly where it goes wrong, record what you did instead, and train the next checkpoint on the old data plus that correction. That is a DAgger round, and this is how one runs on an SO-100 with a vision-language-action policy.

The theory is elsewhere: why dataset aggregation works at all and what human gating changes about it. This is the operating manual, and it assumes a trained checkpoint, a working camera set, and an arm that moves. The six steps below are the loop as implemented on the DAgger page of this platform, but the sequence is the same from your own scripts.

Why the second round is not just more data

Behaviour cloning trains on the states a human visited. At test time the policy visits the states it causes, and small action errors compound into states no demonstration covered. Ross, Gordon and Bagnell formalised that failure for AISTATS 2011 and answered it with an iterative algorithm that trains a stationary deterministic policy and, under their reduction, has to perform well under the state distribution it induces: run the current policy, have the expert label the states it actually reached, add those to the dataset, retrain, repeat. Kelly et al. made the query practical with HG-DAgger, where the human decides when to take control instead of labelling states without holding the controls; they report improved performance over both DAgger and behaviour cloning on a simulated and a real autonomous driving task. Human gating is what makes the loop tolerable on a desk arm - you only move your hands when something is going wrong.

Two consequences matter more in practice than the theory does. Corrections are not ordinary demonstrations: they concentrate at the bottleneck regions Mandlekar et al. describe, where a small deviation drops the policy into states the demonstrations never covered. And a dataset made only of those hard parts is a badly shaped dataset - Belkhale, Cui and Sadigh argue from the data side that state diversity is not always beneficial, and that action divergence and transition diversity together decide dataset quality. The mixed dataset is not a compromise, it is the point.

Freeze these before round one

A DAgger round compares a policy against itself across time. Anything you change between rounds that is not the dataset makes that comparison meaningless.

• Camera positions and mounts, wrist camera included. Loosen one clamp and you changed the observation distribution, not the policy.
• Exposure and white balance, if your capture stack lets you pin them. Auto-exposure drifting between rounds is a slow, invisible domain shift.
• Arm calibration and servo zero positions. If you must recalibrate, treat everything recorded before it as a separate dataset.
• The task text. Every VLA here conditions on it; rewording it mid-loop is a different task.
• Lighting, table surface, object set. A new object is a new experiment, not the next round.
• The recording frame rate. Comparing intervention rates across two sampling rasters produces differences that come from the raster.

The round, end to end

1. 1
   Run inference and record it

   Start the run against the checkpoint you want to improve, then start the recording into the inference root. Recorded that way it inherits the run's own task text, which is what the policy was trained on, rather than the default teleoperation label. Without the recording you can watch the failure but not train on it.

   bash

   # two calls, not one: the run, then its recording
   POST /inference/start      # model_id, and hf_repo_id = the checkpoint to drive
   POST /recording/start      # root=inference
   # root=inference also makes the recording inherit the run's task text

2. 2
   Take over when it goes wrong

   Press Take over and choose the input mode: leader arm, keyboard or sliders. The runner pauses, you correct, you hand back. Frames recorded while you were driving are flagged as interventions automatically.

   bash

   POST /inference/takeover/start   # input = leader | keyboard | sliders
   POST /inference/takeover/nudge   # keyboard, relative delta per call
   POST /inference/takeover/set     # sliders, absolute target
   POST /inference/takeover/stop    # back to the policy

3. 3
   Triage the episodes

   Decide per episode: file as correction, keep as evaluation, or discard. A run the policy completed without help is evaluation data.

4. 4
   Sync the correction dataset

   Corrections collect in a local dataset per policy and go to cloud storage through the automatic sync. Nothing is mixed in that you did not put there.

5. 5
   Compose the mixed dataset

   Combine the original dataset with the corrections, choosing episodes explicitly per source. The result is an ordinary dataset from that point on.

   bash

   POST /training/datasets/compose
     sources  = [ original_dataset, korrekturen_<policy> ]
     episodes = explicit selection per source

6. 6
   Continue training from the checkpoint

   Train the mix from the previous checkpoint rather than the base model. Note which checkpoint and which mix; without that pair the round is not reproducible.

   bash

   # field on the training job
   base_checkpoint = s3://ay-robots/checkpoints/<run>/<checkpoint>
   # the platform passes it to the training pod as BASE_CKPT_S3

Step 2 in detail: the two ways to take over

With a leader arm

In leader-follower mode the takeover is a handover between two arms that are not in the same pose. Pressing Take over pauses the runner and drives the leader onto the follower's current pose, so nothing jumps when torque transfers. If that alignment drive times out, you align the leader by hand and release only once the two are within five degrees. From then on you teleoperate normally and the action column records what you commanded.

Be honest about this path: the alignment drive and the torque handover are the least tested part of the loop on real hardware. Test the handover on a slow, harmless pose before you rely on it in a run you care about. A leader arm produces the smoothest corrections of the three modes, and also has the most that can go wrong mechanically.

Without a leader arm: keyboard and sliders

Most people reading this own one arm. That is enough. Choose keyboard or slider input at the moment you press Take over, and the takeover is immediate and manual - there is no second arm to align, so there is no alignment step. The follower holds its pose and waits for input.

The clamps are enforced server-side, not in the interface, because a mistyped delta on a bus-servo arm is a collision. Keyboard corrections come out stepwise and slightly coarse; slider corrections are smoother, because the server walks toward the target while the interface keeps streaming. Either way the action column receives the full commanded pose vector and the intervention marking is identical to the leader path, so keyboard corrections land in the same dataset without a format difference.

Q / A   joint 1      R / F   joint 4
W / S   joint 2      T / G   joint 5
E / D   joint 3      Z / X   gripper

When to press the button

Early rather than late. A correction that starts after the gripper has closed on nothing teaches recovery from a failure the policy should not have entered, and recovery data is worth far less than avoidance data. Interrupt at the first moment you are confident the trajectory is wrong, correct through the difficult part, hand back as soon as the state is one the policy handled before. ThriftyDAgger automates that decision by gating interventions on novelty and estimated risk under a fixed human budget, but on a single arm with a human already watching, the human gate is cheaper and better calibrated than anything you will tune.

Step 3 in detail: triage decides the quality

After the run you have a recording with some frames marked as interventions. Three destinations exist, and the wrong one quietly poisons the next round.

• File as correction when the intervention was a genuine fix: the policy was heading somewhere wrong and your input showed the right thing from a state the policy itself produced.
• Keep as evaluation for clean autonomous runs and for runs you took over out of caution. Evaluation episodes are how you measure the next checkpoint, and they must never be trained on.
• Discard runs ruined by something unrelated - a dropped camera frame, a stalled servo, an object you knocked over. A messy correction is worse than no correction.

Step 5 in detail: composing the mix

Composition takes the original dataset plus the correction dataset and produces a new, ordinary LeRobot dataset that trains like any other. The important property is that the episode selection is explicit per source - nothing is blended in automatically. That sounds minor until the first time a policy behaves strangely and you have to reconstruct what it was trained on.

The open question is the ratio, and nobody has a number that transfers. What the literature does agree on is that corrections should count for more than their frame share. Mandlekar et al. retrain iteratively on the data their intervention system collects, so the policy learns to traverse the bottlenecks, and report that agents trained that way outperform agents trained on an equivalent number of samples from non-interventional demonstrators. Sirius goes further and re-weights training samples by approximated human trust, reporting an 8 percent gain in simulation and 27 percent on real hardware in policy success rate against the methods it compares with, at twice the convergence speed. None of the training entry points here expose a sample-weighting knob, so the crude substitute is to keep every correction episode while subsampling the original demonstrations - and to write down what you did.

Step 6 in detail: what continuing from a checkpoint really means

Training the mix from the base model works but throws away the previous round and costs a full run. Continuing from the previous checkpoint is faster and usually better. It is also more limited than the phrase suggests.

Latency compounds inside a DAgger loop in a way it does not during a demo: at roughly 485 ms per action step you take over because the arm hesitated, not because it was wrong, and hesitation corrections are not useful training data. If you are iterating on data rather than chasing a final success rate, iterate on a fast model. Shukor et al. describe SmolVLA as designed to train on a single GPU and deploy on consumer GPUs or CPUs, with an asynchronous inference stack that decouples action prediction from execution to allow higher control rates - the property that keeps a takeover loop responsive.

The dataset formats are not interchangeable either. GR00T takes LeRobot v2.0 or v2.1, and the Isaac-GR00T repository describes its input as a flavour of the LeRobot v2 format with an added modality description file; the newer trainers here expect v3.0. A mix composed in the wrong version fails at load time rather than producing a bad policy - the better failure mode, still a wasted queue slot. The dataset documentation lists which format each trainer takes.

Four ways to waste a round

1. Training on the corrections alone

The most common failure and the most tempting shortcut. A correction-only dataset is almost entirely the difficult middle of the task, with approach and retreat missing; the policy gets better at the hard part and forgets how to arrive there. Aggregation is not an implementation detail of the method, it is the mechanism: the old data is what holds the rest of the behaviour in place while the corrections move one part of it.

2. Moving a camera between rounds

A camera that shifts two centimetres between rounds produces a policy worse than the one you started with, and a diagnosis that costs a day. Every VLA here conditions on images; joint state alone does not disambiguate where the object is. Photograph the setup before the first round and check that photograph before each later one.

3. Letting handover artefacts into training

Covered above, and on the list because it is invisible. The symptom is a policy that stalls for a fraction of a second exactly where the previous round's operator took over. It looks like hesitation; it is imitation.

4. Calling a warm start a resume

If you believe optimizer state carried over, the initial loss spike reads as a bug and you go hunting for corrupted data. If you know the optimizer started fresh, the spike is expected and you look at what comes after it. Same numbers, opposite conclusions.

Measuring the round

The metric for a human-gated loop is the intervention rate: frames recorded while you were in control, divided by total frames of the run. It is in the takeover status, and it is the only number that answers the question the round asked. Training loss falls whether or not the policy improved; success rate is binary and noisy at the sample sizes a desk arm produces. Intervention rate is continuous, measured on the states the policy itself caused, and it drops as the policy needs you less.

Compare it only across runs recorded under identical conditions. The full argument, and how to build an evaluation set that survives more than two rounds, is in the article on measuring a DAgger loop. Round one is realistically a feasibility test: you are checking that the takeover works on your hardware, that corrections land with their flags, and that the continued run loaded the checkpoint you named. Rounds two and three are where the rate should start moving. If it has not moved by round four, the problem is upstream of DAgger.

If you do not have a checkpoint yet

The loop has no entry point without one. Record a first dataset, train a first policy, run it - recording, training and running the policy cover that path. The recording client is on the download page, GPU tiers and hourly rates on the pricing page, and what a usable episode looks like in the SO-100 data collection guide. Get the demonstrations right before the corrections: DAgger is a repair mechanism, and it works far better on something that was nearly right already.

None of this is a solved problem and none of it is one click. Interactive imitation learning is an active research area precisely because its questions - when to intervene, how to weight what the human did, how much old data to keep - have no settled answers; the survey by Celemin et al. maps what is still open. What the loop does have is measurable convergence when it is run carefully, on hardware that costs a few hundred euros. Freeze the setup, intervene early, triage honestly, mix deliberately, and record the intervention rate every time.