HG-DAgger and the Gated Variants: Who Decides When to Take Over a Robot Policy

A cloned policy fails where its training data ran out. The fix is to keep collecting data under the policy's own state distribution instead of the demonstrator's, which is what DAgger does and what the introduction to dataset aggregation covers from first principles. It leaves open the awkward part of the original algorithm: while the learner drives, the expert is supposed to say what they would have done, for every state, without being in control.

In a simulator with a scripted expert that is free. On a table with a real arm on it it is neither free nor safe. The HG-DAgger authors state the objection precisely: such sampling schemes "require the expert to provide action labels without being fully in control of the system", which "can decrease safety and, when using humans as experts, is likely to degrade the quality of the collected labels due to perceived actuator lag" (Kelly et al., 2019). Two failures hide in that sentence: the policy keeps driving into states nobody wants it in, and the labels bought with that risk are worse than the ones a human produces while holding the controls. Every variant below answers the same question — put a gate on control and let someone decide when it opens. They differ in who that someone is.

Human-gated and robot-gated: the split that matters

Interactive imitation learning has its own survey; Celemin and colleagues catalogue it along feedback type, interface, learning model, user experience, application and benchmark. The distinction that decides your engineering is simpler than any of those axes, and recent work names it directly: a method is human-gated when the supervisor decides when to intervene, and robot-gated when the agent decides when to ask for help (Cai et al., 2025). Everything else is machinery serving one of those two choices. The trade is visible from the start: a human gate costs continuous attention, and a robot gate promises to give that attention back — but only if the signal it gates on is trustworthy, and building that signal is its own research problem.

SafeDAgger: a classifier that predicts its own deviation

Zhang and Cho's SafeDAgger (2016) is the earliest of these gates. Querying the reference policy is the expensive part, so a second, small network — the safety policy — predicts whether querying is worthwhile at all. It "returns a binary label indicating whether the primary policy is likely to deviate from a reference policy without querying it". The label is defined against a squared L2 deviation between the two policies' actions with a threshold tau, and the classifier is fit with binary cross-entropy. At run time it decides per state whether the learner keeps driving or the reference takes over. The abstract reports fewer reference queries in a car racing simulator plus a convergence speed-up the authors attribute to an automated curriculum effect, without giving a figure for either.

The catch is in the definition, not the results. Training the classifier requires the reference policy's action on every state used to fit it, which assumes the reference is another program. If your reference is a person with a leader arm, that label set is not cheap and the method loses the property it was designed for.

EnsembleDAgger: doubt and discrepancy, both

Menda, Driggs-Campbell and Kochenderfer (2019) treat the gate as a probabilistic question. EnsembleDAgger "approximates a Gaussian Process using an ensemble of neural networks" and reads the ensemble variance as a confidence proxy: high variance means a region unlike the training data, which — assuming the expert avoids failure states — correlates with proximity to failure. The rule is a conjunction. The learner may act only when the L2 distance between its mean action and the expert's action stays below one threshold (discrepancy) and the variance of its predicted action stays below a second (doubt). If either fails, the expert takes control. The goal is to maximise the learner's share of actions while constraining the probability of failure; the paper reports improved safety and learning against other DAgger variants on an inverted pendulum and MuJoCo HalfCheetah.

LazyDAgger: stop the switch from chattering

Hoque and colleagues (2021) attacked a cost the earlier papers did not measure: the switch itself. Each intervention "interrupts other work the human is doing, incurs latency with each context switch between supervisor and autonomous control, and requires time to perform". A gate that opens and closes ten times a minute is worse than one opening once for ten seconds, at identical autonomous share. LazyDAgger extends SafeDAgger with asymmetric thresholds — harder to enter supervisor control than to stay in it — so the system does not oscillate at the boundary. In simulation it "can reduce context switches by an average of 60% over SafeDAgger on 3 continuous control tasks while maintaining state-of-the-art policy performance"; in fabric manipulation with an ABB YuMi it "reduces context switches by 60% while achieving a 60% higher success rate than SafeDAgger at execution time".

ThriftyDAgger: novelty or risk, under a budget

ThriftyDAgger (Hoque et al., CoRL 2021) starts from the supervisor's budget rather than the policy. It "uses a learned switching policy to solicit interventions only at states that are sufficiently (1) novel, where the robot policy has no reference behavior to imitate, or (2) risky, where the robot has low confidence in task completion", with a new metric for estimating that risk. The budget is an input, not an outcome: you state how much human time you will spend. Applied at execution time the method "achieves a 100% success rate on both the simulation and physical tasks", and a user study with ten participants controlling a three-robot fleet alongside a concentration task reports that it "increases human and robot performance by 58% and 80% respectively compared to the next best algorithm while reducing supervisor burden". The fleet setting is the natural home for this work; Fleet-DAgger later formalised interactive fleet learning with multiple robots and supervisors, proposing Return on Human Effort as the quantity to optimise.

HG-DAgger: the human holds the gate

Kelly et al. (2019) go the other way. There is no switching policy. The learner is rolled out "until the expert observes that the novice has entered an unsafe region of the state space", at which point the expert takes control and guides the system back. The aggregation rule is the strict part: "Expert action labels are only collected and added to the dataset during these recovery trajectories, during which the human expert has uninterrupted control of the system." Nothing is labelled while the human is a spectator.

It does not stop at the switch. HG-DAgger also "learns a safety threshold for a model-uncertainty-based risk metric that can be used to predict the performance of the fully trained novice in different regions of the state space": it logs how uncertain the learner was at the moments the human intervened and averages the last quarter of those records, where the learner most resembles its final form. That is not a gate the robot operates but a diagnostic telling you where the finished policy is expected to hold. The evaluation covers a simulated and a real-world driving task and reports improved performance over both DAgger and behavior cloning.

One related line changes what counts as a label. Spencer and colleagues' Expert Intervention Learning holds that "any amount of expert feedback, whether by intervention or non-intervention, provides information about the quality of the current state, the optimality of the action, or both", formalised as a constraint on the learner's value function and solved with no-regret online learning. Under that reading, the stretches where the human watched and did nothing are weak positive evidence rather than empty. EIL learns collision avoidance from about one minute of expert control.

The variants side by side

What each gate buys, and what it charges

Read down the "needs" column and a pattern falls out: every robot-gated signal there is a proxy that has to be manufactured, and each proxy has its own bill.

• Discrepancy signals (SafeDAgger, LazyDAgger, half of EnsembleDAgger's rule) need a reference policy. Either you have a scripted expert, in which case the interesting problem is already solved, or a human keeps producing actions in the background so a distance can be computed.
• Doubt signals need calibrated epistemic uncertainty. An ensemble of small control networks approximates it; an ensemble of a three-billion-parameter vision-language-action model is a memory and latency problem first.
• Novelty and risk signals need a learned estimator of task completion, fitted on enough successful and failed rollouts. On a task you are still getting to work, that data does not exist in round one.
• The human gate needs attention and nothing else — the only signal available on day one, on any policy architecture, with no extra model to train.
• No robot gate removes the human from the room. They reduce how often the human must act, not whether they must be present.

There is a quieter difference in what the gate produces. A robot gate firing mid-trajectory hands a person a moving arm at an arbitrary pose. A human gate lets the operator pick the moment — after the reach, before the grasp, not halfway through a swing. The recovery segments from the two are not equally clean, and those segments are the entire product of the round.

Why the human-gated form wins on real hardware

This is an engineering argument, not a benchmark result — no paper we found runs all five gates on the same manipulator with the same policy class. It rests on four points.

First, the supervisor is there anyway. Nobody leaves an SO-100 arm running unattended next to objects it can knock over. Once someone is watching, the marginal cost of giving them a takeover button is near zero; building a calibrated risk estimator is a project.

Second, the uncertainty you can actually measure on a modern policy is often the wrong quantity. A diffusion or flow-matching head produces variance across sampled action chunks, and that variance reflects the multimodality the head was trained to represent, not epistemic ignorance about the state. EnsembleDAgger's doubt term uses an ensemble precisely to capture the latter. The two look like the same number and are not; the action chunking and flow matching entries cover how those heads emit actions in the first place.

Third, the failures that matter in manipulation are often semantic rather than statistically unusual. A policy closing the gripper two centimetres early, or reaching confidently for the wrong one of two identical cubes, is not in a high-variance state — it is confident and wrong. No variance threshold catches that; a person watching catches it immediately.

Fourth, label quality — the original HG-DAgger point, and the one most often skipped. Labels collected while the human has uninterrupted control beat labels narrated over a moving system. If the goal is corrective data worth aggregating, the burden of proof lies with the automated gate.

The gate is half the method; data handling is the other half

A gate decides which frames a human drove, not what to do with them, and that second decision is where rounds are most often wasted. The first rule is the one the D in DAgger stands for: aggregate, do not replace. Fine-tuning a checkpoint purely on a few hundred correction frames gives you a model that has seen almost nothing but recoveries and has forgotten the nominal trajectory.

The second rule is that intervention frames are not ordinary frames. Mandlekar et al. built a remote-teleoperation system for 6-DoF manipulation letting operators monitor policies and take over on failure, then trained iteratively with an algorithm that "encourages the policy to learn how to traverse bottlenecks through the interventions"; agents trained on that data outperformed agents trained on an equal number of ordinary demonstration samples. Sirius pushes the idea into deployment, "re-weighing training samples with approximated human trust and optimizing the policies with weighted behavioral cloning". Both need a per-frame marker of what was human-driven, which is why the intervention flag matters more than it looks.

The third rule is that automatic mixing is a trap. A composed dataset whose sources you cannot enumerate is one you cannot debug when the next round gets worse. On this platform the compose step is explicit for that reason — original dataset plus corrections, episode selection per source, and the result is an ordinary LeRobot dataset that syncs and trains like any other; the dataset documentation covers the format side.

Setting a gate a person can actually hold

"The human decides" is not a specification. Two operators with different thresholds produce incomparable rounds, and a drifting threshold produces a trend you cannot read. Write the triggers down before the first run.

1. Name the conditions that open the gate — approach off by more than a fixed margin, gripper closing on nothing, a joint drifting towards a limit, a stall of more than a second or two — and keep the list unchanged for the round.
2. Name what does not open it. Overshoot the policy recovers from on its own is training signal; taking over there deletes a recovery it already knows.
3. Take over early enough for a clean handover, hand back as soon as the state is recoverable.
4. Log why you took over, per episode. Three rounds later it is the only record of whether the same failure returns.
5. Keep cameras, task text and operator constant across rounds. Change one and the numbers stop being comparable.

Mechanically the handover has two variants. With a leader arm, the leader has to reach the follower's current pose before torque transfers, or the arm jumps; this alignment move is the least-tested part of the chain on real hardware. Without a leader arm the takeover is manual from the first keypress: the follower is held and the operator nudges it joint by joint from the keyboard or drags sliders, with server-side clamps keeping each input small — a couple of degrees per keypress, a handful per slider update, because a large step into a held pose is how you strip a servo. The step-by-step version with the key bindings is in the walkthrough of a DAgger round on an SO-100; background on both teleoperation modes is in the teleoperation docs and the leader-follower entry.

Reading whether the gate did anything

The metric of a human-gated round is the intervention rate: intervention frames divided by frames of the run. It has one job — if it does not fall across rounds, the round bought nothing, and the next one should change something other than the amount of data.

It is a biased metric and you should know how. It measures the operator's threshold as much as the policy's competence, which is why the trigger list stays fixed; an operator who relaxes over a session produces a falling curve with nothing behind it. It also ignores severity — ten frames to stop a collision and ten to nudge a grasp look identical. Pair it with a fixed evaluation set no correction data was ever drawn from. The article on measuring a DAgger loop works through the evaluation design, including how to keep evaluation episodes out of the training mix.

Open questions worth keeping in view

The benchmarks are weak. Spencer and colleagues examined those used to test imitation learning approaches and found them "realizable and simple and thus insufficient for capturing the harder regimes of error compounding seen in real-world decision making problems" — and, against the surrounding literature, found plain behavior cloning did well on them. That is reason to be sceptical of any ranking of DAgger variants resting on those tasks, including some numbers in the table above.

Robot gates on large policies are not solved either. The AIM work replaces uncertainty with a proxy Q-function mimicking the human intervention rule and reports a 40% improvement in take-over cost and learning efficiency over ThriftyDAgger — in continuous and discrete control, not on a vision-language-action model driving a manipulator. Whether any of these gates transfers to a fine-tuned VLA is open. The survey makes a related point: the field's terminology and structure are not unified across the literature, which makes methods hard to compare. On your own arm, your logs are the evidence you have.

For a single arm on a desk the choice is made: hold the gate yourself, write down what opens it, and spend the effort on the data handling behind it rather than on automating the switch. The platform side is described on the DAgger loop page, training options per policy family under training and pricing. For background, start with imitation learning and imitation learning on the SO-100; for a first run, run your first policy is the shorter path.