DAgger: How to Fix an SO-100 Policy That Drifts

Your policy works for two seconds. The arm leaves home, tracks toward the cube, and the wrist ends up a centimetre off. From there everything it sees is new, the gripper closes on air, the run is over. Nothing in the training data showed the arm what to do from a centimetre off, so the policy has nothing to say.

That is not a bug in your training run. It is the defining failure of behaviour cloning, it has a published bound, and the fix predates every VLA: DAgger, short for Dataset Aggregation, from Ross, Gordon and Bagnell at AISTATS 2011. Below: the bound, the human-gated variant, and the loop on an SO-100.

Why behaviour cloning drifts, with the actual bound

Behaviour cloning is supervised learning with one twist that breaks supervised learning. You fit observation to action on pairs from a human driving the arm, then deploy. The deployed policy picks its own next state, so training happens on the expert's state distribution and testing on the policy's own. Those coincide only if it never errs.

Ross and Bagnell put a number on that gap in 2010, restated as Theorem 2.1 of the DAgger paper. With per-step error epsilon under the expert's state distribution and cost bounded in the unit interval, over an episode of length T the total cost can exceed the expert's by T squared times epsilon. The paper calls the bound tight.

Published bounds at epsilon = 0.01 and u = 1, not a measurement of your arm, and the two epsilons are not the same quantity: the behaviour cloning one is measured under the expert's state distribution, the DAgger one under the distributions the learner itself induced. Cost is bounded in the unit interval, so the behaviour cloning column saturates at T; only its growth rate still says anything. Doubling the horizon quadruples the behaviour cloning worst case and doubles the DAgger one, which is why long-horizon tasks dominate pages like policy only works in one setup and policy freezes mid-motion.

DAgger in one page

1. 1
   Start from demonstrations

   Collect a normal dataset by teleoperation and train the first policy with plain behaviour cloning. This is the iteration where beta equals 1: the expert drives.

2. 2
   Roll out the mixed policy

   At iteration i the system executes pi_i = beta_i * expert + (1 - beta_i) * learner, beta_i being the probability of taking the expert's action.

   python

   # DAgger, Ross, Gordon and Bagnell, AISTATS 2011, Algorithm 3.1
   D = []
   pi_hat = init_policy()
   
   for i in range(1, N + 1):
       beta = 1.0 if i == 1 else 0.0          # the parameter-free variant
       pi_i = lambda s: expert(s) if random() < beta else pi_hat(s)
   
       states = rollout(pi_i, horizon=T)      # states the LEARNER visits
       D_i = [(s, expert(s)) for s in states] # labels the EXPERT gives
   
       D = D + D_i                            # aggregate, never replace
       pi_hat = train(D)
   
   return best_on_validation(pi_hat_history)

3. 3
   Label the visited states

   Record the states the rollout passed through and the action the expert would have taken in each. That asymmetry is the whole idea.

4. 4
   Aggregate, do not replace

   Union the new set into the running dataset; every earlier round stays in. That is where the name comes from.

5. 5
   Retrain from the aggregate

   Train the next policy on all of D, then return to step 2. Keep the checkpoint that scores best on validation.

Any beta schedule works as long as its running average goes to zero, and the paper reports that the simplest choice, beta = 1 on the first iteration and 0 after, often performs best in practice. On Super Tux Kart the controller stopped falling off the track after 15 iterations, while the supervised baseline did not improve as more expert laps were added.

HG-DAgger: the human holds the gate

HG-DAgger, from Kelly, Sidrane, Driggs-Campbell and Kochenderfer, keeps the aggregation and drops the mixing. HG stands for human-gated: the novice drives, the human watches, and when things go wrong the human takes over. Only the timesteps where the human has control are recorded.

Two things follow. Labels stay clean, because the human produces a continuous trajectory instead of annotating a state they do not control. And the data is recovery data: it starts where the policy got into trouble and ends where the human considers it salvaged. The same run yields a risk threshold learned from where the human actually intervened; rollouts started inside the resulting permissible set had a mean collision rate 12 times lower than those started outside.

Running the loop on an SO-100 with LeRobot

LeRobot ships this as a rollout strategy. In 0.6.1 on PyPI, released 2026-08-03, lerobot-rollout takes --strategy.type with five registered values, base, sentry, highlight, episodic and dagger, and --inference.type with two, sync and rtc. Pin the version: 0.5.1 has no lerobot-rollout entry point, the command first appears in 0.6.0 of 2026-07-06.

pip install "lerobot==0.6.1"

# confirm the registered strategy, robot and teleoperator names on your own install
lerobot-rollout --help

1. 1
   Record the base dataset

   An ordinary teleoperated recording. Defaults are 60 s per episode, 60 s reset, 50 episodes. The recording walkthrough covers the camera and calibration setup that must stay identical across rounds.

   bash

   lerobot-record \
     --robot.type=so100_follower \
     --robot.port=/dev/ttyACM0 \
     --robot.id=my_follower \
     --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, wrist: {type: opencv, index_or_path: 2, width: 640, height: 480, fps: 30} }" \
     --teleop.type=so100_leader \
     --teleop.port=/dev/ttyACM1 \
     --teleop.id=my_leader \
     --dataset.repo_id=me/cube-v0 \
     --dataset.num_episodes=50 \
     --dataset.single_task="Pick up the cube and put it in the box"

2. 2
   Train the base policy

   ACT is the sensible start, for reasons below. The training walkthrough and the ACT on SO-100 guide have the hyperparameters.

   bash

   lerobot-train \
     --dataset.repo_id=me/cube-v0_20260823_101500 \
     --policy.type=act \
     --output_dir=outputs/train/act_cube_v0 \
     --job_name=act_cube_v0 \
     --policy.device=cuda \
     --wandb.enable=true

3. 3
   Run the policy with a human on the gate

   The policy drives; you pause when failure looks imminent; the leader powers up onto the follower's pose; you take over with torque released, teleoperate back to a familiar state, finish the sub-task, hand control back. Nothing is recorded during the pause. Space pauses, tab starts and stops a correction, escape ends the session. Use the same leader you recorded round zero with.

   bash

   lerobot-rollout --strategy.type=dagger \
     --robot.type=so100_follower \
     --robot.port=/dev/ttyACM0 \
     --robot.id=my_follower \
     --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}, wrist: {type: opencv, index_or_path: 2, width: 640, height: 480, fps: 30} }" \
     --teleop.type=so100_leader \
     --teleop.port=/dev/ttyACM1 \
     --teleop.id=my_leader \
     --policy.path=outputs/train/act_cube_v0/checkpoints/last/pretrained_model \
     --dataset.repo_id=me/rollout_cube_hil_r1 \
     --dataset.single_task="Pick up the cube and put it in the box" \
     --dataset.fps=30 \
     --strategy.record_autonomous=true \
     --duration=1800
   
   # for Pi0, Pi0.5 or SmolVLA add real-time chunking so the arm does not stutter:
   #   --inference.type=rtc

4. 4
   Merge round zero with round one

   A first-class merge operation exists, so no stitching parquet by hand. It refuses unless fps, robot_type and the feature set match, and that is where a dagger dataset trips: it carries an extra boolean intervention column your demonstrations do not have.

   bash

   # 1. drop the column the demo dataset does not have
   lerobot-edit-dataset \
     --operation.type=remove_feature \
     --operation.feature_names='[intervention]' \
     --repo_id=me/rollout_cube_hil_r1_20260823_143000 \
     --new_repo_id=me/rollout_cube_hil_r1_flat
   
   # 2. aggregate round 0 and round 1 into one dataset
   lerobot-edit-dataset \
     --operation.type=merge \
     --operation.repo_ids='[me/cube-v0_20260823_101500, me/rollout_cube_hil_r1_flat]' \
     --new_repo_id=me/cube-v1-aggregate

5. 5
   Fine-tune on the aggregate

   Point the trainer at the merged dataset, not the corrections alone. The LeRobot page says to fine-tune on combined data, but its own example command points at the HIL dataset by itself.

   bash

   lerobot-train \
     --dataset.repo_id=me/cube-v1-aggregate \
     --policy.type=act \
     --policy.pretrained_path=outputs/train/act_cube_v0/checkpoints/last/pretrained_model \
     --output_dir=outputs/train/act_cube_v1 \
     --job_name=act_cube_v1 \
     --policy.device=cuda

6. 6
   Read the intervention rate, not the loss

   LeRobot's HIL-SERL guide calls a run successful when the human intervenes a lot at the start and less as it goes on. If interventions per episode stay flat across two rounds, loss falls but the policy does nothing is the better next stop.

How much correction data, and how to weight it

The most useful number here is not a bound, it is a budget. Mandlekar et al. started from 30 demonstrations, then ran three rounds, collecting in each until intervention samples reached roughly 33 percent of the initial dataset's sample count. That keeps human-labelled samples equal across methods regardless of base policy quality.

Success rates in percent over 3 seeds, simulated Sawyer arm in robosuite, 2-layer LSTM policies; threading is one operator, coffee machine averages three. Not SO-100 numbers, so take the ordering, not the absolutes. Three rounds of corrections beat an equal budget of fresh demonstrations on both tasks. The data collection guide covers the case where the answer is still more demonstrations.

Two ways to run the loop

Where this does not help, and the latency wall

One class of failure sits outside DAgger's reach. If the policy fails identically in every episode from the first frame, that is a broken input or a broken training run, not compounding error: check loss falls but the policy does nothing and policy only works in one setup. A gripper that never closes is mechanical first, and a joint stopping short is joint stops early.

The other limit is the dataset format. GR00T N1.7 and N1.5 take LeRobot v2.0 or v2.1, and a v3.0 dataset crashes the GR00T loader, so every correction round against GR00T has to come out in the accepted version. ACT, SmolVLA and Pi0.5 take v3.0; the dataset rejected as v3 page has the conversion.

Which policy to run the loop against

ACT is the obvious start: 20 ms per action step means your takeover lands roughly when you decide it, and four rounds cost 4 to 12 USD. It has no base model, so round zero must be real demonstrations. SmolVLA is the compromise at 30 minimum episodes, compared under ACT against SmolVLA. GR00T N1.7 and Pi0.5 are cloud-only. See the policy overview.

Action chunking means the model commits to a block of future actions at once, so a takeover cannot land mid-chunk without something absorbing the discontinuity; that is what --inference.type=rtc is for. For the counterweight on data scale see the BC-Z write-up, for the hardware end the SO-100 complete guide, and for the loop on your own arm imitation learning on the SO-100.