DAgger Loop ඉක්මනුවෙන් සිතිය: Intervention Rate, Evaluation Protocol සහ එයට අතරින් ඇති උගුලු

A DAgger round feels productive while you are in it. You drive the policy, take over when it fumbles the grasp, save the corrections, retrain, and the next run looks - you would swear - a little smoother. Two rounds later you cannot say whether anything changed, because "a little smoother" is not a quantity.

The loop needs one number per round, and in a human-gated loop that number is nearly free: the fraction of the run during which you had control instead of the policy. This article defines it so two people compute the same value, logs it, reads the resulting curve, and names the four ways it misleads you when it stands alone. For the theory this piece assumes, see the DAgger explainer and the human-gated variant; the hands-on counterpart is running a DAgger loop on an SO-100.

Why a round needs a number at all

DAgger exists because of a distribution problem, and distribution problems are invisible to the eye. Ross and Bagnell showed that imitation learning on a fixed demonstration set leads to compounding errors and a regret bound growing quadratically in the time horizon: once the learner leaves the states the demonstrator visited, nothing in the data tells it how to come back. DAgger fixes this by iterating - run the policy, label the states it visits, aggregate, retrain - which Ross, Gordon and Bagnell frame as a reduction to no-regret online learning.

That framing has a consequence people skip. The guarantee concerns the sequence of policies over iterations, not any single run. It says nothing about whether your round three helped. The only way to know is to measure each iteration. Kelly and colleagues introduced HG-DAgger because classic DAgger asks the expert for action labels while the novice is still in control, which the paper argues can decrease safety and, with human experts, is likely to degrade label quality through perceived actuator lag. HG-DAgger also learns a safety threshold for a model-uncertainty-based risk metric and reports improved performance over both DAgger and behavioural cloning on a driving task. It publishes no intervention counts; those you produce yourself.

Defining the rate so two people get the same number

The definition is one line: intervention frames divided by frames of the run. Everything difficult hides in what counts as a frame and what counts as a run.

Frames, not episodes

Counting episodes with at least one intervention is useless: ten episodes with one nudge each and ten in which you drove eighty percent both give "10/10". The frame count separates them, and it is the granularity the recording already has - in the LeRobot dataset format every timestep is a row, so the intervention flag is one boolean column next to state and action. Here it is written per frame the moment takeover starts and cleared when control returns.

The denominator includes the corrections

Two denominators are defensible - total frames of the run, or frames the policy drove autonomously - and they diverge badly at high intervention levels. Take over for 400 of 1000 frames and the first gives 40 percent, the second 67 percent. Comparing one round measured the first way against the next measured the second is how a real improvement disappears. Take total frames and never revisit the choice mid-series.

Decide once what happens to handover frames

There is always a seam. When control passes over, some frames belong to neither side: the arm is held, the leader is aligning, the recording is frozen. In this pipeline those handover frames stay in the raw stream and never enter the curated episode - they are neither policy behaviour nor a correction worth training on. Count the seam the same way every round: at 30 Hz, six takeovers with a two-second seam each is 360 frames, enough to move a percentage point.

Logging it so the number survives the session

A metric in the status panel of a running process is a readout: it disappears on restart and cannot be plotted. One append-only line per run covers the whole series - including which checkpoint you drove, since that changes every round and mixing them up invalidates what follows.

{"round": 2, "run_id": "inf-2026-08-24-1108", "checkpoint": "ckpt-8500",
 "frames": 5412, "intervention_frames": 611, "rate": 0.113,
 "takeovers": 7, "input": "keyboard", "denominator": "total_run_frames",
 "handover_frames": "excluded", "episodes_kept_as_correction": 5,
 "train_mix": {"base_demos": 120, "corrections": 41},
 "eval_protocol": "protocol-A", "eval_trials": 20, "eval_successes": 11}

Two fields carry the weight. train_mix is what you fed the next training job; without it nothing can be attributed. eval_protocol names the fixed test you ran afterwards, and is the field most often left empty. In the ay-robots cockpit the takeover status reports the intervention frame count for the running session, so logging is transcription rather than instrumentation; the dataset documentation covers where the per-frame columns land.

Reading the curve: three rounds, one verdict

Three rounds is the minimum for a reading, because two points are always a line. The table below is a bookkeeping template with placeholder numbers, not measurements from any run. You are looking for a monotone decrease matched by an increase in success on a fixed test.

Rounds one and two are doing work. Round three is not: 11.3 against 11.1 percent is inside the run-to-run variation of anything measured on a physical arm, and a fourth round of the same corrections spends an afternoon for nothing. The flat segment says change something structural.

• The remaining failures are not correctable by teleoperation - object out of reach, gripper geometry, a joint at its limit. No correction data fixes a kinematic wall.
• The corrections are too few against the base dataset to move the gradient, and nothing weights them.
• The corrections contradict each other, so the policy averages two strategies and lands between them.
• The failure is upstream: a camera moved, the lighting changed, the wrist view no longer matches training.
• The task is at the ceiling of this policy class on this hardware, and the next step is more base data.

The last two are not failures of the loop. In Sirius-Fleet a declining need for the human is the designed outcome: as robot autonomy improves, its anomaly predictors adapt their prediction criteria, leading to fewer requests for human intervention and gradually reducing human workload over time. There the criterion adapts on purpose. In a manual loop yours adapts too, silently - so a rate that flattens because the task is at its ceiling looks exactly like one that flattened because the operator stopped noticing. Which is the next problem.

Trap 1: a falling rate is not a rising success rate

The intervention rate measures exactly one thing: how much of the run you decided to own. That decision is yours, made in real time, and it moves. In round zero you take over at the first bad approach angle; by round three you have watched the policy recover from a dozen of them and you let it try. Your threshold moved; the policy may not have. The rate falls either way.

Human trust is at least a modelled quantity in the literature rather than an assumed constant. Sirius re-weights training samples with approximated human trust and optimises the policy with weighted behavioural cloning, reporting an 8 percent boost in simulation and 27 percent on real hardware over the state of the art in policy success rate, at twice the convergence speed. That treats trust as a weight on the data. It does not correct the threshold in your head between round zero and round three.

You cannot remove the drift, so pair the rate with something your threshold cannot touch: a fixed set of trials in which you do not intervene at all, scored against a criterion written before the round. A rate that falls while the paired success rate stays put is the signature of habituation - worth catching, because from inside the loop it feels like progress.

Trap 2: without a fixed protocol, you measure the room

Real-robot evaluation is expensive and hard to repeat. The SIMPLER authors motivate simulated evaluation with the observation that real-world evaluation of such policies is not scalable and faces reproducibility challenges likely to worsen as policies broaden their task spectrum. RoboArena attacks it from the other side, with more than 600 pairwise real-robot evaluation episodes across seven generalist policies, run by evaluators at seven academic institutions on the DROID platform. Its evaluators pick their own tasks and environments but must judge pairs of policies double-blind; the paper reports that this crowd-sourced ranking tracks generalist-policy performance more accurately than conventional, centralised evaluation.

You will not run 600 paired episodes on one SO-100 in a workshop. What you can do is remove the variance you control - a list boring enough that it usually gets skipped.

Then the arithmetic, unforgiving at the trial counts a workshop can afford. Under the normal approximation, 60 percent success over 20 trials carries a standard error near 11 percentage points - the square root of 0.6 times 0.4 over 20 - and the difference between two such rounds carries about 15. A jump from 60 to 70 percent is therefore consistent with nothing having happened. Fifty trials bring the per-round figure to roughly 7 points, and cost an afternoon.

This asymmetry is the argument for the intervention rate: computed over thousands of frames rather than twenty binary outcomes, it moves earlier and more smoothly. The caveat is that frames within an episode are heavily correlated - one bad grasp yields a hundred consecutive intervention frames - so the effective sample size is closer to the number of takeovers. Treat the rate as the early indicator and the success rate as slow ground truth.

Trap 3: training only on corrections bends the policy

The corrections are the interesting data, so the instinct is to train on them. Do not. They come by construction from the narrow slice of state space where the policy was already failing and say almost nothing about the majority of the run that worked. Fine-tune on that slice alone and you get a policy good at recovering from a botched approach that has forgotten how to make a clean one.

Mandlekar and colleagues built their remote intervention system around this. Their framing: manipulation tasks contain bottleneck regions requiring a sequence of precise actions - inserting a pod into a coffee machine is their example - where small deviations lead into states the demonstrations never covered. Their algorithm trains iteratively on the new data so the policy learns to traverse those bottlenecks, and they report that agents trained on intervention data beat agents trained on an equivalent number of samples from non-interventional demonstrators.

The mixing ratio is a hyperparameter and deserves to be written down. Composing the next dataset is where it is decided: original demonstrations plus the correction set, episode selection explicit per source rather than a rule that quietly pulls in whatever is available. Here that is one compose step in the cockpit, and the result is an ordinary dataset - see the training documentation. What no tool does for you is recording which ratio produced which curve.

Trap 4: the human is not a consistent expert

DAgger's theory assumes an expert. You are not one in the technical sense: your corrections are not samples from a fixed conditional distribution over actions. The ACT authors name this when listing the obstacles to fine manipulation - errors compound over time, and human demonstrations can be non-stationary. Their system still reaches 80 to 90 percent success on six real tasks from ten minutes of demonstrations, so the problem is tractable, not absent.

The robomimic study makes the point from the data side. Across six offline algorithms on five simulated and three real-world multi-stage tasks, its lessons include sensitivity to design choices, dependence on demonstration quality, and - the one that hurts most here - variability arising from stopping criteria, because training and evaluation objectives differ. The checkpoint with the lowest loss is not reliably the one with the highest success rate.

There is a hardware version of this: the input mode shapes the correction. A leader-follower setup produces continuous, human-paced trajectories. Keyboard nudges are clamped hard by the server - two degrees per call on the arm joints, four on the gripper - so the same intent arrives as a staircase of small steps. Sliders send absolute targets and the server moves at most six degrees toward them per call. Three modes, three action distributions; mixing all three into one correction set and then wondering why the approach turned twitchy is self-inflicted. The teleoperation documentation covers what each mode sends.

Weighting the interventions: IWR and what came after

If corrections are the valuable minority, the principled fix is weight rather than exclusivity. Intervention Weighted Regression is the reference implementation: the data is partitioned into intervention and non-intervention samples, and the two partitions are sampled in equal proportion during training. A correction set that is five percent of the frames then contributes half the gradient, without the nominal behaviour vanishing from the distribution.

Equal proportion is a starting point, not a law. Sirius generalises it by replacing the binary partition with a continuous weight from approximated human trust; Sirius-Fleet moves the decision upstream, using a visual world model and anomaly predictors to decide when a human is asked at all. The common thread: the intervention flag is a training signal, not just a bookkeeping column.

Four metrics worth logging next to the rate

• Takeovers per run. Twenty short corrections and one long one give similar rates and describe different policies - one jittery, one with a single blind spot.
• Mean intervention length. Rising length with a falling count means the remaining failures are the hard ones, which is what late progress looks like.
• Time to first intervention. A policy that gets further before needing help is improving even when the total rate is flat.
• Where interventions cluster. Bin the flag by normalised episode progress; a stable peak across rounds points at one bottleneck.

A round protocol you can actually run

A measured round has six steps and produces one row in the log. The first four are the loop; the last two make it a measurement.

1. 1
   Freeze the evaluation protocol before round zero

   Write down start pose, object placement, cameras, trial count, timeout and success criterion. Photograph the table. If the document has to change, the series restarts.

2. 2
   Measure the baseline

   Run the protocol with no interventions and record the successes; then run one instrumented session with takeover enabled and record the rate. Those two numbers are round zero.

3. 3
   Collect corrections in one consistent style

   One input mode per round, one operator if you can manage it. Take over on a criterion you can state out loud - 'gripper more than two centimetres off at approach' - and hold it for the round.

4. 4
   Triage every episode the same day

   Correction, evaluation or discard. Ambiguous episodes get discarded, not saved on the theory that more data helps - a correction in which you yourself fumbled is worse than no episode.

5. 5
   Compose the mixture explicitly

   Original demonstrations plus corrections, episode selection per source. Record base count, correction count and any weighting - this is the field you will want in three weeks.

6. 6
   Continue from the checkpoint, then re-measure

   Train the composed dataset from the previous checkpoint rather than the base model, run the frozen protocol plus one instrumented session, append the row, and compare against the previous two rounds.

Two limits change how you read a weak round. Continuing from a checkpoint initialises the weights from it - not an optimiser resume, so the schedule starts fresh and a short run from a converged checkpoint can move very little. And the leader-align motion at the start of a takeover has the least mileage on real hardware of anything in the chain; if corrections all begin with an odd transient, look there before blaming the mixture.

What the numbers cannot tell you

None of this is solved, and a tidy curve should not persuade you otherwise. The intervention rate measures a joint system - policy, hardware, operator, room - and attributes nothing on its own. It cannot judge whether the corrections were good, and it will fall happily on a task that got easier because the object drifted two centimetres closer over three weeks.

What it does is turn a vague impression into a column you can argue with. The alternative is not a better metric; it is three weeks of collecting corrections the curve would have told you, after round three, to stop collecting. The survey literature defines interactive imitation learning as human feedback given intermittently during robot execution, allowing an online improvement of the behaviour; the family is wide, and no arrangement of it works without a number per round. See also the SO-100 imitation learning guide for the base dataset you mix against, running a policy for the inference side, and the DAgger loop page for the implemented pipeline.

Keep the log in the repository, not in a notebook: rounds are days apart, hardware gets rebuilt, and the person reading the curve in October is you with no memory of August. The documentation FAQ covers the operational details left out here.