Action Chunking Explained: Chunk Size, Executed Steps, Jitter

Action chunking in one paragraph

A policy trained by imitation learning normally maps one observation to one action. At 30 frames per second over a twelve second episode that is 360 decisions in a row, each mistake feeding back into the next observation. Action chunking changes the output rather than the network: from one observation the model emits a block of consecutive actions covering the next second or three, and the controller plays part of it back before asking again.

The term comes from Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware by Zhao, Kumar, Levine and Finn, submitted to arXiv on 23 April 2023. It introduced Action Chunking with Transformers, which "learns a generative model over action sequences" and reports "80-90% success" on tasks such as slotting a battery "with only 10 minutes worth of demonstrations". At around 80 million parameters, ACT is the smallest of the five policies you can train here.

# lerobot/policies/act/configuration_act.py, main branch, read 2026-08-23
n_obs_steps: int = 1
chunk_size: int = 100
n_action_steps: int = 100

# Inference.
# Note: the value used in ACT when temporal ensembling is enabled is 0.01.
temporal_ensemble_coeff: float | None = None

The failure it fixes: compounding error

Behaviour cloning assumes training and test data come from the same distribution. A centimetre off the demonstrated trajectory, the arm sees a frame no human ever produced, so the next prediction is worse and the one after worse again. The ACT paper: "Small errors in the predicted action can incur large differences in the state, exacerbating the 'compounding error' problem of imitation learning".

Chunking attacks the number of decisions rather than the quality of each one. Committing to k steps at a time reduces the effective horizon of the task "by k-fold". A second benefit gets less attention: human teleoperation data is full of short pauses, which the paper calls temporally correlated confounders. A single-step model sees near-identical frames with different labels and averages them into a stall. A chunked model reproduces the pause as a pause, because it sits inside one chunk of the episode.

chunk_size and n_action_steps are not the same knob

This is where people lose a day. chunk_size belongs to the trained network. It fixes the output shape and what the loss is computed against: ACTConfig exposes action_delta_indices as list(range(chunk_size)), so the dataloader slices that many future rows from the LeRobot dataset per sample. Change it and you retrain. n_action_steps belongs to the runtime loop only, and changes on an existing checkpoint without touching a weight.

The lerobot docstring spells the discard out: "the model predicts 100 steps worth of actions, runs 50 in the environment, and throws the other 50 out." Mechanically it is a queue: ACTPolicy.select_action holds a deque with maxlen=n_action_steps, refills it from predict_action_chunk() when it runs dry, and pops one action per control tick.

# lerobot main. Both are plain draccus overrides on the policy config.
lerobot-train \
  --dataset.repo_id=${HF_USER}/so101_dataset_test \
  --policy.type=act \
  --policy.chunk_size=100 \
  --policy.n_action_steps=25 \
  --policy.device=cuda \
  --output_dir=outputs/train/act_chunk100_exec25 \
  --job_name=act_chunk100_exec25 \
  --steps=20000

What upstream actually ships

Every modern manipulation policy chunks. They disagree on how long the chunk is and how much of it they trust. Repository values below were read from the main branch on 23 August 2026, because these defaults move.

Diffusion Policy is the clearest published statement of the trade-off: the authors swept action horizons and "found the action horizon of 8 steps to be optimal for most tasks that we tested" out of a 16 step prediction. Executing half is far more conservative than lerobot's ACT default of executing all 100. NVIDIA went further: N1.7 expanded action_horizon from 16 to 40, and the README example executes 8 of those 40 per call.

Jitter lives at the seam between chunks

The symptom is recognisable: the arm moves smoothly for about a second, twitches, moves smoothly again, twitches again, at a regular interval. Count that interval. If it matches n_action_steps divided by your frame rate, you have found it. The ACT paper names the mechanism, that "a new environment observation is incorporated abruptly every k steps and can result in jerky robot motion". Inside a chunk a decoder produced the trajectory as one sequence; across the boundary nothing enforces continuity.

Temporal ensembling removes the seam by never creating one. Query the policy every timestep and you hold several overlapping predictions for the same future timestep, each from a different observation, which you average. The paper writes the weights as w_i = exp(-m * i), "where w_0 is the weight for the oldest action" and "a smaller m means faster incorporation" of new observations. The reference implementation hard-codes k = 0.01.

# lerobot/policies/act/modeling_act.py -> ACTTemporalEnsembler
self.ensemble_weights = torch.exp(-temporal_ensemble_coeff * torch.arange(chunk_size))
self.ensemble_weights_cumsum = torch.cumsum(self.ensemble_weights, dim=0)

# from the class docstring:
#   - Setting it to 0 uniformly weighs all actions.
#   - Setting it positive gives more weight to older actions.
#   - Setting it negative gives more weight to newer actions.

# and in ACTPolicy.select_action:
if self.config.temporal_ensemble_coeff is not None:
    actions = self.predict_action_chunk(batch)
    action = self.temporal_ensembler.update(actions)
    return action

What temporal ensembling costs

A 30 Hz loop gives 33 ms per step, and lerobot's RolloutConfig defaults to fps = 30.0. Of the five policies here, the comparison page lists ACT at 20 ms per action step, SmolVLA at 245 ms, GR00T N1.7 at 152 ms, GR00T N1.5 at 165 ms and Pi0.5 at 485 ms. Only ACT fits inside 33 ms. For the other four, executing several steps per query is what lets the loop close, and that is why ACT is still the default choice on an SO-100.

Beyond ensembling: what lerobot ships today

Ensembling was the 2023 answer and is still the right first move for ACT. For slow vision-language-action models it is unavailable. Real-Time Execution of Action Chunking Flow Policies by Black, Galliker and Levine, at NeurIPS 2025, targets the boundary artefact directly. It names the "pauses or out-of-distribution jerky movements at chunk boundaries" and generates the next chunk while executing the current one, "freezing" the actions guaranteed to execute and "inpainting" the rest, "out of the box with no re-training".

On the main branch in August 2026 that is a flag, not a prototype. lerobot-rollout has a pluggable inference backend: --inference.type=sync is the default, "one policy call per control tick", and --inference.type=rtc is "Real-Time Chunking for slow VLA models". Its tunables are execution_horizon, where the docs give "typical values: 8-12 steps", and max_guidance_weight, where "for 10 steps flow matching (SmolVLA, Pi0, Pi0.5), a value of 10.0 is an optimal value". Even here the reactivity knob is the execution horizon.

# from the lerobot-rollout docstring, main branch, read 2026-08-23
# Base mode - RTC inference for slow VLAs (Pi0, Pi0.5, SmolVLA)
lerobot-rollout \
    --strategy.type=base \
    --policy.path=lerobot/pi0_base \
    --inference.type=rtc \
    --inference.rtc.execution_horizon=10 \
    --inference.rtc.max_guidance_weight=10.0 \
    --robot.type=so100_follower \
    --robot.port=/dev/ttyACM0 \
    --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
    --task="pick up cube" --duration=60

A tuning procedure that converges

Do not sweep both numbers at once. They are not independent and you will read the results wrong. This order changes one thing at a time, cheap steps before the expensive one.

1. 1
   Read the dataset frame rate first

   Chunk length is a duration, not a count, and the conversion factor is your recording rate. Read it off the dataset instead of assuming 30.

   python

   from lerobot.datasets.lerobot_dataset import LeRobotDataset
   
   ds = LeRobotDataset("${HF_USER}/so101_dataset_test")
   print(ds.fps, ds.num_episodes, ds.num_frames)
   # chunk_size 100 at 30 fps = 3.33 s of committed motion
   # chunk_size 100 at 50 fps = 2.00 s, the ALOHA setting

2. 2
   Set chunk_size to about one motion primitive

   Reach, grasp, lift, place. Time one in your recordings and multiply by the frame rate. Tabletop pick and place at 30 fps lands between 50 and 100, so the default of 100 is a reasonable first guess.

   bash

   lerobot-train \
     --dataset.repo_id=${HF_USER}/so101_dataset_test \
     --policy.type=act \
     --policy.chunk_size=100 \
     --policy.n_action_steps=100 \
     --policy.device=cuda \
     --steps=20000

3. 3
   Roll out at the default and time the twitches

   A regular twitch every 3.3 seconds at 30 fps is the chunk boundary. An irregular tremor is a different problem, usually data or servos.

   bash

   lerobot-rollout \
     --strategy.type=base \
     --policy.path=${HF_USER}/act_policy \
     --robot.type=so101_follower \
     --robot.port=/dev/ttyACM0 \
     --robot.cameras="{ front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
     --task="put the red brick in a bowl" \
     --duration=60

4. 4
   Halve n_action_steps until the arm reacts

   100, then 50, then 25, no retraining. You want the point where the arm corrects for an object you nudged mid-episode. Below roughly 10 you pay forward passes for no extra reactivity.

   bash

   # same checkpoint, new runtime behaviour
   lerobot-rollout \
     --strategy.type=base \
     --policy.path=${HF_USER}/act_policy \
     --robot.type=so101_follower \
     --robot.port=/dev/ttyACM0 \
     --duration=60

5. 5
   Only then reach for ensembling or RTC

   Enable ensembling if the loop still has budget at n_action_steps=1, starting at 0.01. If motion turns sluggish rather than smooth, stale predictions dominate the average: use a smaller coefficient. For a slow VLA, use the RTC backend instead.

   bash

   lerobot-rollout \
     --strategy.type=base \
     --policy.path=${HF_USER}/pi0_policy \
     --inference.type=rtc \
     --inference.rtc.execution_horizon=10 \
     --robot.type=so100_follower \
     --robot.port=/dev/ttyACM0 \
     --duration=60

Doing it yourself versus doing it here

conda create -y -n lerobot python=3.12
conda activate lerobot
conda install ffmpeg -c conda-forge

git clone https://github.com/huggingface/lerobot.git
cd lerobot
pip install -e ".[core_scripts,training,feetech]"

# 1. record
lerobot-record \
  --robot.type=so101_follower --robot.port=/dev/ttyACM0 \
  --robot.id=my_follower_arm \
  --teleop.type=so101_leader --teleop.port=/dev/ttyACM1 \
  --teleop.id=my_leader_arm \
  --dataset.repo_id=${HF_USER}/so101_dataset_test \
  --dataset.num_episodes=50 \
  --dataset.single_task="put the red brick in a bowl"

# 2. train with explicit chunk settings
lerobot-train \
  --dataset.repo_id=${HF_USER}/so101_dataset_test \
  --policy.type=act --policy.chunk_size=100 --policy.n_action_steps=25 \
  --policy.device=cuda --steps=20000 \
  --output_dir=outputs/train/act_so101

Where chunking does not help

Chunking is a horizon trick. It adds no information, repairs no dataset and compensates for no camera that moved between recording and rollout. Being clear saves a week on the wrong number.

• A policy that never learned the task executes its confusion in smoother blocks. Smooth and wrong is still wrong.
• It does nothing about observation-side distribution shift. If the lighting changed, see policy only works in one setup.
• It cannot recover from an error inside the committed block: at n_action_steps=100 and 30 fps, an object that moves just after the query stays invisible for three seconds.
• Longer chunks reduce reactivity by construction. Diffusion Policy puts it plainly: an action horizon greater than 1 helps, "but too long a horizon reduces performance due to slow reaction time".
• It does not make a small dataset behave like a large one. Below ACT's 50 episode minimum no chunk setting rescues the run, and total compute is unchanged, only how often you spend it.

Symptom to setting

Two of those rows are not tuning problems: policy freezes mid motion and arm twitches then sags have their own pages, indexed with the rest at the fix pages. No hardware yet? The live arm streams a physical SO-100 without signup.

Choosing a policy with chunking in mind

ACT has no base model: it exists only after you train it, so the settings are yours from the first run and the training steps are cheap enough to sweep. The VLAs arrive with a chunk length chosen on someone else's robot and a diffusion or flow matching action head. Fine-tune one and you inherit that length, so your reactivity budget comes entirely from the executed-steps flag.

For the numbers side by side, ACT versus SmolVLA and ACT versus GR00T N1.7 lay out the tiers, and the arena entry for ACT links each benchmark to its paper. Background: the VLA overview, the pi-zero flow matching article, the SO-100 complete guide and train your first policy.