Episode Length and What It Does to Training

What an episode actually is, in frames

An episode is one uninterrupted attempt at the task: you press record, you drive the arm through the pick, you stop. In a LeRobot dataset it is not a file any more. Since the v3.0 format, many episodes live inside the same Parquet and MP4 shards, and the episode boundaries are resolved through meta/episodes/ records that carry per-episode lengths and byte offsets. What you get back when you index the dataset is a frame, not an episode. The dataset length is the total frame count, not the episode count.

That distinction is the whole article. Every training sample is one frame plus a window of future actions hanging off it. How long your episodes are decides how many of those windows fit inside an episode, and how many of them run off the end. For imitation learning that is the difference between a dataset that teaches a horizon and one that teaches a fragment.

# Episode statistics of any LeRobot dataset on the Hub, without downloading it
curl -sL https://huggingface.co/datasets/lerobot/svla_so101_pickplace/resolve/main/meta/info.json \
  | python3 -c 'import json,sys; d=json.load(sys.stdin); \
      print(d["codebase_version"], d["robot_type"], d["fps"], "fps"); \
      print(d["total_episodes"], "episodes,", d["total_frames"], "frames"); \
      print(round(d["total_frames"]/d["total_episodes"],1), "frames/ep =", \
            round(d["total_frames"]/d["total_episodes"]/d["fps"],1), "s")'

# v3.0 so100_follower 30 fps
# 50 episodes, 11939 frames
# 238.8 frames/ep = 8.0 s

Three reference datasets, read from their meta/info.json on the same day, give a useful sense of the real range. All three carry exactly 50 episodes, which is also the ACT minimum on this platform and the lerobot recording default.

The simulated ALOHA set is exactly 500 frames per episode because simulation episodes are cut at a fixed length. The two real SO-100 sets are averages, and they sit at 8 and 13 seconds. Nobody recorded for a minute.

The recording defaults are a ceiling, not a target

Here is what teleoperation recording gives you if you change nothing. These are the field defaults in src/lerobot/configs/dataset.py on the lerobot main branch, read on 24 August 2026 at version 0.6.2 in development. The last release on PyPI at that point was 0.6.1, published 3 August 2026.

Left alone, the defaults also set your wall clock. Fifty episodes at 60 seconds of recording plus 60 seconds of reset is 100 minutes at the arm, and a large fraction of the frames you paid for with that time will be the arm holding a finished pose. Set the two timers deliberately instead.

lerobot-record \
  --robot.type=so100_follower \
  --robot.port=/dev/tty.usbmodem58760431541 \
  --robot.id=my_follower \
  --robot.cameras="{front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
  --teleop.type=so100_leader \
  --teleop.port=/dev/tty.usbmodem58760431551 \
  --teleop.id=my_leader \
  --dataset.repo_id=${HF_USER}/pick-cube-short \
  --dataset.single_task="Pick up the red cube and drop it in the bin" \
  --dataset.num_episodes=50 \
  --dataset.fps=30 \
  --dataset.episode_time_s=15 \
  --dataset.reset_time_s=10 \
  --dataset.streaming_encoding=true \
  --dataset.encoder_threads=2 \
  --display_data=true

Action chunking is the other half of the equation

A chunked policy predicts a block of future actions from one observation. Action chunking came out of the ACT paper, which framed it as a way to cut the effective horizon of a task. The paper's phrasing is direct: predicting the next k timesteps instead of one reduces the effective horizon of the task by k-fold, which is what keeps compounding error under control. Their ablation is the number everyone quotes: success rises from 1 percent at k=1 to 44 percent at k=100, then tapers slightly at k=200 and k=400 as the policy approaches fully open-loop control.

Every policy on this platform chunks, but they chunk at very different scales. These are the config defaults in lerobot main, plus the action horizon in NVIDIA's own SO-100 example config for GR00T.

Note the observation column. All four use n_obs_steps=1: one camera frame, one joint state, no history. The GR00T SO-100 config makes this explicit, with delta_indices=[0] for both video and state and range(0, 16) for the action. There is no context window in the language-model sense. The only temporal context these policies have is the chunk they are about to emit.

The learnable horizon is shorter than the episode

It is tempting to read episode length as how much of the task the policy learns. It is not. What a chunked policy learns is a mapping from one observation to the next k actions, and nothing longer. The episode contributes two things on top of that: it decides how many such mappings exist, and it decides how consistent they are with each other. A 60 second episode does not teach a 60 second behaviour. It teaches eighteen overlapping 3.33 second behaviours that happen to be stitched together by the fact that they came from one continuous attempt.

This is why effective horizon is the right phrase and context is the wrong one. There is no context window being filled. The policy re-reads the world every k steps and starts again from whatever it sees. Continuity across an episode is something you get by accident, because consecutive observations look similar and the demonstrations were consistent, not because the architecture carries state. When people say a policy lost track of the task, what usually happened is that two points in the episode looked alike from the camera and the policy picked the wrong continuation. Longer episodes make that collision more likely, not less.

The ratio that predicts trouble: k over T

Call the chunk size k and the episode length in frames T. Two derived numbers follow, and both of them are more useful than either input on its own. T/k is how many independent decision points the policy gets per episode. k/T is how much of the task it commits to from a single frame.

fps, episode_s, chunk = 30, 8, 100
T = int(episode_s * fps)
print(f"frames per episode   T        = {T}")
print(f"chunks per episode   T/k      = {T/chunk:.1f}")
print(f"ratio                k/T      = {chunk/T:.2f}")
print(f"frames with padding  (k-1)/T  = {(chunk-1)/T:.1%}")
print(f"action steps masked  (k-1)/2T = {(chunk-1)/(2*T):.1%}")

# frames per episode   T        = 240
# chunks per episode   T/k      = 2.4
# ratio                k/T      = 0.42
# frames with padding  (k-1)/T  = 41.2%
# action steps masked  (k-1)/2T = 20.6%

Worked out across the episode lengths people actually record on an SO-100, at 30 fps:

The platform exposes chunkSize and nActionSteps directly in the ACT on SO-100 training form, both defaulting to 100, so this is a field you change rather than a code edit. The other four policies do not expose it, which is worth knowing before you pick a model on the policies page.

What happens in the last k-1 frames of every episode

Ask a dataset for the frame 30 steps from the end with a chunk size of 100 and you are asking for 70 actions that do not exist. LeRobot does not error and it does not skip the sample. It clamps the query indices to the episode boundary and records a mask.

# src/lerobot/datasets/dataset_reader.py, DatasetReader._get_query_indices
ep_start = ep["dataset_from_index"]
ep_end = ep["dataset_to_index"]
query_indices = {
    key: [max(ep_start, min(ep_end - 1, abs_idx + delta)) for delta in delta_idx]
    for key, delta_idx in self.delta_indices.items()
}
padding = {
    f"{key}_is_pad": torch.BoolTensor(
        [(abs_idx + delta < ep_start) | (abs_idx + delta >= ep_end) for delta in delta_idx]
    )
    for key, delta_idx in self.delta_indices.items()
}

The mask is then honoured by the loss, so those repeats do not teach the policy to freeze at the end of a chunk. ACT divides by the count of valid steps rather than the full tensor size, which means a heavily padded batch is not down-weighted either. It simply carries less signal.

# src/lerobot/policies/act/modeling_act.py, ACTPolicy.forward
abs_err = F.l1_loss(batch[ACTION], actions_hat, reduction="none")
valid_mask = ~batch["action_is_pad"].unsqueeze(-1)
num_valid = valid_mask.sum() * abs_err.shape[-1]
l1_loss = (abs_err * valid_mask).sum() / num_valid.clamp_min(1)

So the honest version of the story is not that short episodes poison the loss. It is that they spend a large share of your recording effort on targets that get thrown away. Exactly k-1 frames per episode produce a partly padded window, and the total share of masked action steps is (k-1)/2T.

Short and clean against long and messy

The angle people usually want settled: is it better to record thirty crisp eight-second picks or ten sixty-second sequences where you fumble twice and recover? Both produce a working dataset. They fail in different ways, and the failure modes are not symmetric. Short episodes fail quietly, by handing the optimiser less signal than you think you gave it. Long episodes fail loudly, by teaching contradictory actions for visually similar states.

There is also a bookkeeping difference that people underestimate. Training length is counted in gradient steps over frames, not over episodes. Fifty episodes of 8 seconds at 30 fps is about 12,000 frames; fifty of 60 seconds is 90,000. At the ACT defaults this platform sends, batch size 8 over 100,000 steps, the short dataset is seen roughly 67 times and the long one roughly 9 times. Neither number is automatically wrong, but they are very different regimes, and if you shorten your episodes without adjusting anything you have quietly changed how many epochs your training steps add up to.

What the published datasets actually chose

It helps to see how much the accepted answer varies. These figures come from the papers and project pages, fetched and checked on 24 August 2026. The seconds-per-episode column is arithmetic on the totals the sources state, not a number the sources publish directly.

Two things stand out. The first is that the spread is narrow given how differently these datasets were built: a lab with two arms and a fixed protocol, a distributed collection effort across three continents, and a pile of hobbyist uploads all landed between about 8 and 17 seconds. The second is that control rate and episode length move together. BridgeData V2 runs at 5 Hz and averages 38 steps; ALOHA runs at 50 Hz and averages 400 to 700. In seconds those are close. In frames they differ by more than an order of magnitude, and it is the frame count that your chunk size has to live inside.

Nothing published sits at a minute. The DROID collection and BridgeData V2 went about it very differently and still landed in the same band. If you want the wider view on what makes a demonstration usable at all, the data collection guide covers the parts that are not about length.

Picking an episode length for your task

1. 1
   Time the task by hand first

   Do the pick with the leader arm five times and note the wall clock. That median, plus about half again for the attempts that go badly, is your episode_time_s ceiling. Do not start from 60.

   bash

   # 8 s median, 15 s ceiling
   --dataset.episode_time_s=15 --dataset.reset_time_s=10

2. 2
   Convert to frames and check k/T

   Multiply by fps. Then divide your policy's chunk size by that number. Aim for k/T at or below roughly 0.2, which is where the ACT ablation peaked, and for at least five decision points per episode.

   python

   T = 8 * 30            # 240 frames
   for k in (100, 50, 40, 30, 16):
       print(k, round(k/T, 3), round(T/k, 1))
   
   # 100 0.417 2.4
   # 50  0.208 4.8
   # 40  0.167 6.0
   # 30  0.125 8.0
   # 16  0.067 15.0

3. 3
   Record a pilot of five episodes and read the metadata back

   Do not record fifty before you check. Five is enough to see whether your real average matches your plan.

   bash

   lerobot-record \
     --robot.type=so100_follower --robot.port=/dev/tty.usbmodem58760431541 --robot.id=my_follower \
     --teleop.type=so100_leader --teleop.port=/dev/tty.usbmodem58760431551 --teleop.id=my_leader \
     --dataset.repo_id=${HF_USER}/pilot --dataset.single_task="Pick up the red cube" \
     --dataset.num_episodes=5 --dataset.episode_time_s=15 --dataset.reset_time_s=10
   
   lerobot-edit-dataset --repo_id ${HF_USER}/pilot --operation.type info

4. 4
   Throw away the episodes that ran long

   An episode that hit the ceiling almost always means the attempt went wrong. Delete those rather than keeping them for volume. This writes a new dataset and leaves the original alone.

   bash

   lerobot-edit-dataset \
     --repo_id ${HF_USER}/pick-cube-short \
     --new_repo_id ${HF_USER}/pick-cube-clean \
     --operation.type delete_episodes \
     --operation.episode_indices "[3, 11, 27]"

5. 5
   Set chunk_size when you train, not after

   For ACT the chunk size is a policy config field, so it is fixed at training time. lerobot's own defaults are batch_size 8, steps 100000 and seed 1000, which match the ACT defaults this platform sends.

   bash

   lerobot-train \
     --dataset.repo_id=${HF_USER}/pick-cube-clean \
     --policy.type=act \
     --policy.chunk_size=40 \
     --policy.n_action_steps=20 \
     --batch_size=8 \
     --steps=100000 \
     --seed=1000 \
     --output_dir=outputs/train/act_pick_cube

Two ways to get this right

git clone https://github.com/huggingface/lerobot.git
cd lerobot
pip install -e ".[core_scripts]"

lerobot-find-port          # identify the two USB serial ports
lerobot-calibrate --robot.type=so100_follower --robot.port=/dev/tty.usbmodem... --robot.id=my_follower

# then lerobot-record with your own episode_time_s, then lerobot-train

Where episode length stops being the problem

Two limits are worth stating plainly, because both get blamed on the data when they are not the data.

The first is inference latency. At 30 fps you need a new action every 33.3 ms. This platform lists ACT at 20 ms per action step, GR00T N1.7 at 152 ms, GR00T N1.5 at 165 ms, SmolVLA at 245 ms and Pi0.5 at 485 ms. Only ACT fits inside a 30 fps budget. Everything else has to run its chunk open loop while the next one computes, which is a large part of why chunking exists at all. No episode length fixes that, and putting the inference across a public internet hop makes it worse. Remote inference works for slow pick-and-place and not for fast reactive motion.

The second is that episode length cannot buy you memory. Every policy here uses n_obs_steps=1. If your task genuinely requires knowing what happened ten seconds ago, a longer episode does not give the policy that information, it just gives it more ambiguous frames. That is an architecture question, and the VLA overview and the model arena are better places to start on it than a recording flag. For the hardware side of a first dataset, the SO-100 setup guide covers calibration and camera placement, which matter more than episode length does.

A short checklist before you record fifty episodes

1. Time the task by hand. Set episode_time_s to about 1.5x the median, not to 60.
2. Set reset_time_s to what the reset actually takes. It is unrecorded time and it doubles your session length.
3. Compute k/T for the policy you intend to train. Aim at or below 0.2. See the dataset docs for the format side.
4. Record five, read the metadata back, then commit to fifty.
5. Tap right arrow the moment the task is done. Dead frames at the end teach the arm to stop.
6. Delete the episodes that hit the ceiling before training. The training docs cover what happens after that.