Insertion and Assembly Tasks on a Low-Cost Robot Arm

Insertion is not pick-and-place with a smaller target

A pick-and-place policy is allowed to be sloppy. Close the gripper 4 mm off centre on a cube and the cube slides between the fingers, the error washes out, and the episode still counts. Insertion has the opposite property. The moment a peg touches the rim of a hole, misalignment stops being a position error and becomes a force, and an SO-100 answers a force by winding up servo current until the part scrapes, the arm sags, or the servo overheats.

That difference is why contact-rich tasks are the place where a cheerful demo turns into a week of debugging. This page covers three things: the real tolerance budget on a hobby-price arm, why the control mode underneath the arm works against you, and which of the five trainable policies actually closes the loop fast enough to react to contact. Platform numbers come from the policy catalogue. Everything else comes from papers and repositories opened on 24 August 2026, with versions named where they matter.

The tolerance budget nobody writes down

Before you argue about model architectures, do the stack-up. Insertion succeeds when the total error between the held part and the hole is smaller than the clearance. Every term below adds into that total, and most of them are properties of the hardware rather than of the policy you train on top of it.

Why position control fights you

The STS3215 is a smart servo, not a joint with a torque interface. In teleoperation and during rollout, LeRobot configures every motor the same way. This is the configure step from the SO follower in lerobot main, checked on 24 August 2026:

# src/lerobot/robots/so_follower/so_follower.py  (lerobot main, 24 Aug 2026)
def configure(self) -> None:
    with self.bus.torque_disabled():
        self.bus.configure_motors()
        for motor in self.bus.motors:
            self.bus.write("Operating_Mode", motor, OperatingMode.POSITION.value)
            self.bus.write("P_Coefficient", motor, self.config.position_p_coefficient)  # default 16
            self.bus.write("I_Coefficient", motor, self.config.position_i_coefficient)  # default 0
            self.bus.write("D_Coefficient", motor, self.config.position_d_coefficient)  # default 32

            if motor == "gripper":
                self.bus.write("Max_Torque_Limit", motor, 500)   # 50% of max torque to avoid burnout
                self.bus.write("Protection_Current", motor, 250) # 50% of max current to avoid burnout
                self.bus.write("Overload_Torque", motor, 25)     # 25% torque when overloaded

# and the entire proprioceptive observation:
obs_dict = self.bus.sync_read("Present_Position", num_retry=self.config.num_read_retries)

Two things in that snippet decide how insertion behaves. First, Operating_Mode is POSITION for every joint, so the servo's only job is to drive its output shaft to the last Goal_Position it was given. When the peg jams, the position error grows, the proportional term grows with it, and the servo pulls more current into a part that is not going to move. There is no compliance term to give way. Second, the register table in LeRobot does define Present_Load at address 60 and Present_Current at 69, but get_observation never reads them.

The consequence is blunt. A LeRobot dataset recorded from an SO-100 contains joint angles and camera frames. It contains no measurement of contact. A policy trained on it cannot learn a force strategy, because it has never observed a force. Whatever it learns about contact, it learns indirectly from pixels: the wrist camera view of a peg that stopped moving.

What the contact-rich literature actually does instead

It is worth looking at what the systems with high insertion success rates have in common, because the answer is consistent and it is not "a bigger model". They add a sensing or control channel that the SO-100 does not have.

The HIL-SERL detail is the one to sit with. Their low-level controller is an impedance controller running at 1000 Hz that accepts 10 Hz setpoints from the policy, with the position error clamped so the spring-damper cannot generate an unbounded force on contact. The neural network is not doing the contact handling. It is nudging the equilibrium point of a spring that handles contact for it, a thousand times per second. Nothing in the SO-100 stack plays that role, which is why behaviour cloning on this hardware has to solve a harder problem than the same algorithm solves on a Franka.

The other useful reference point is the pure geometry. IndustReal's peg set is round and rectangular pegs with maximum dimensions of 8, 12 and 16 mm at 0.5 to 0.6 mm clearance, and the NIST board it draws from includes round pegs at 4, 8, 12 and 16 mm plus USB, RJ45, waterproof and DSUB connectors. The board designs, CAD and STL files are free downloads from NIST, so you can print a real benchmark instead of inventing one, and the public dataset directory is a reasonable place to look for what other people recorded against similar targets.

Which of the five trainable policies handles insertion best

The honest answer is that latency decides this, not architecture. LeRobot records at 30 fps by default, which means each recorded action step is 33 ms of real time. A policy that needs longer than 33 ms to produce an action cannot react within one step of the data it was trained on, no matter how good it is. Combine that with the action chunking defaults and you get the table below.

Read the fifth column as "how long the arm moves blind". A peg approach and mate on a small arm takes on the order of one to three seconds. With ACT's shipped defaults the entire insertion happens inside one chunk, from a single observation, with no chance to correct. That is not a subtle effect. It is the difference between a policy that nudges its way in and one that runs a memorised trajectory and hopes.

ACT wins on latency, not on cleverness

At 20 ms per action step, ACT is the only one of the five that fits inside a 33 ms control period, so it is the only one you can genuinely run closed loop at the rate the data was recorded. Two flags get you there. Lowering n_action_steps shortens the blind window directly. Turning on temporal ensembling smooths the seams between chunks, but LeRobot ships temporal_ensemble_coeff as None and its config validation forces n_action_steps to 1 when you enable it, because the model has to be queried every step to form the ensemble.

# closer to closed loop: predict a long chunk, commit to a short slice of it
lerobot-train \
  --policy.type=act \
  --dataset.repo_id=$HF_USER/so100_insert_usb \
  --policy.chunk_size=100 \
  --policy.n_action_steps=10 \
  --steps=100000 \
  --batch_size=8 \
  --output_dir=outputs/train/act_insert

# or full per-step re-planning with temporal ensembling (ACT paper used 0.01)
#   --policy.temporal_ensemble_coeff=0.01 --policy.n_action_steps=1

The cost is real: querying the model ten times more often means ten times the compute at rollout, and on a 20 ms model that is still affordable. The ACT paper's own ablation shows why the chunk exists in the first place. Averaged over their simulated settings, success went from 1 percent at chunk size 1 to 44 percent at chunk size 100, then tapered slightly above that. Short chunks fight compounding error; long chunks fight contact. Insertion sits exactly on that trade-off, which is why the default is wrong for it in one direction and 1 is wrong in the other.

GR00T N1.7 exposes the knob by name

GR00T N1.7 is the one model here that names the receding horizon in its CLI. The Isaac-GR00T repository renamed the old --action-horizon flag to --execution-horizon precisely to separate "how many actions the model predicts" from "how many it executes before re-planning". The base nvidia/GR00T-N1.7-3B checkpoint uses action_horizon 40, the documented default execution horizon is 16, and the repository's own SO100 example runs with --execution-horizon 16. N1.7 also moved to a relative end-effector action space and a flow-matching DiT head that the repository documents as dropping from 32 to 16 diffusion layers between N1.6 and N1.7. The platform lists it at 152 ms per action step against 165 ms for N1.5, so the newer model is also the quicker one.

# Isaac-GR00T, examples/SO100/README.md (main, 24 Aug 2026)
uv run python gr00t/eval/open_loop_eval.py \
  --dataset-path examples/SO100/finish_sandwich_lerobot/izuluaga/finish_sandwich/ \
  --embodiment-tag NEW_EMBODIMENT \
  --model-path /tmp/so100_finetune/checkpoint-10000 \
  --traj-ids 0 \
  --execution-horizon 16 \
  --steps 400

Pi0.5 is the slowest, and has the most interesting answer for it

Pi0.5 represents actions with flow matching rather than discrete tokens, and the pi0 paper is explicit about why: flow matching lets the model handle high-frequency action chunks up to 50 Hz and dexterous tasks that the authors say pose a major problem for autoregressive VLAs. Pi0.5 keeps that structure, pre-training with discrete FAST tokens and then attaching the flow-matching action expert during post-training for finer action granularity. The theory is right for contact. The practice on this platform is that each action step costs 485 ms, which is roughly fifteen recorded frames at 30 fps.

For completeness: SmolVLA sits in the middle at 245 ms with chunk_size 50 and n_action_steps 50, and it is the cheapest of the four pretrained models to iterate on because it fits a 24 GB card. It is a good choice for testing whether your demonstrations contain a learnable strategy at all, before you spend an A100 run on the same data. The ACT against GR00T N1.7 comparison is the one worth reading if you are deciding between the extremes of that latency range.

Recording insertion demonstrations a policy can learn from

Contact-rich data has failure modes that pick-and-place data does not. The list below is the order that has actually saved time, and none of it needs hardware you do not already have. If you have never recorded an episode before, start with record your first dataset and come back.

1. 1
   Fixture the target, then randomise the part

   Bolt or tape the receptacle down so its pose is constant, and randomise the starting pose of the part being inserted. This is what ALOHA did (targets randomised along a 15 cm reference line) and what the off-the-shelf benchmark did with nests allowing about +/-3 mm and +/-5 degrees. Randomising both ends at once on a five-joint arm with no force sensing is how you end up on policy only works in one setup.

2. 2
   Add a wrist camera and point it at the mating face

   The gripper is the only place the policy can see contact from. A wrist view is what lets it distinguish 'touching the rim' from 'in the hole', since no other channel carries that information. Two views is the usual minimum: one overview for approach, one wrist for the mate.

   bash

   lerobot-record \
     --robot.type=so100_follower \
     --robot.port=/dev/tty.usbmodem58760431541 \
     --robot.id=black \
     --robot.cameras='{
       front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30},
       wrist: {type: opencv, index_or_path: 2, width: 640, height: 480, fps: 30}
     }' \
     --teleop.type=so100_leader \
     --teleop.port=/dev/tty.usbmodem58760431551 \
     --teleop.id=blue \
     --dataset.repo_id=$HF_USER/so100_insert_usb \
     --dataset.num_episodes=60 \
     --dataset.single_task="Insert the USB plug into the socket" \
     --display_data=true

3. 3
   Slow the mating phase down, not the approach

   LeRobot's defaults are fps 30, episode_time_s 60 and reset_time_s 60. Move quickly to a pre-insert pose, then spend most of the frames on the last two centimetres. Frames spent on the approach teach the model nothing it cannot get from ten other episodes; frames spent on the mate are the only ones that carry the correction behaviour.

4. 4
   Record recoveries on purpose

   Deliberately miss, back off 5 mm, re-approach at a slightly different angle, then succeed. Without recovery examples in the data, a behaviour-cloned policy has no representation of 'this is not going in' and will keep pressing. This is the cheapest available substitute for the online corrections that HIL-SERL uses, and it is also what keeps you off policy freezes mid-motion.

5. 5
   Replay before you train, and look for the stall

   Replay an episode on the arm and compare the commanded action against the recorded state. A sustained gap means the follower was jammed while the leader kept moving. Delete those episodes or re-record them. This is the single highest-value check in contact-rich data collection.

   bash

   lerobot-replay \
     --robot.type=so100_follower \
     --robot.port=/dev/tty.usbmodem58760431541 \
     --robot.id=black \
     --dataset.repo_id=$HF_USER/so100_insert_usb \
     --dataset.episode=7

Two ways to get from demonstrations to a policy that inserts

# lerobot main was version 0.6.2 on 24 Aug 2026
git clone https://github.com/huggingface/lerobot.git
cd lerobot
pip install -e '.[core_scripts,training]'   # dataset + hardware + viz + training extras

lerobot-find-port                 # identify the follower and leader serial ports
lerobot-setup-motors --help       # write servo ids one motor at a time
lerobot-calibrate --help          # homing offsets and joint ranges

# record, then train, then roll out
lerobot-record  ...               # see the two-camera example above
lerobot-train   --policy.type=act --policy.n_action_steps=10 ...
lerobot-eval    --help

Where this platform does not help

Being useful about contact-rich work means being clear about the parts of it this stack does not solve. None of the following is fixed by choosing a different model or recording more episodes.

• There is no force feedback anywhere in the loop. The servos expose Present_Load and Present_Current, the robot class does not read them, and none of the five policies has an input for them if it did.
• There is no compliance to tune. No impedance controller, no admittance controller, no remote centre of compliance at the wrist. Compliance on this arm is whatever the printed parts and the gearbox happen to give you.
• Remote inference is not viable for the mating phase. The control loop is already 20 to 485 ms per action step depending on the model, and adding public-internet round trips turns a working policy into a hesitant one. It is fine for slow pick-and-place. It is wrong for fast reactive contact.
• Three of the five models are cloud-only here. GR00T N1.7, GR00T N1.5 and Pi0.5 all need an A100 or H100 80 GB tier; SmolVLA and ACT also run locally on a 24 GB card, which for contact work is an argument in their favour beyond cost.
• The dataset format is a real constraint. GR00T wants LeRobot v2.0 or v2.1 and crashes on v3.0; Pi0.5, SmolVLA and ACT want v3.0. Converting down is a step, not a checkbox, and it is the reason for dataset rejected as v3.
• Nothing here gives you a spiral search. The classical fallback when a peg will not seat is a small force-guided spiral, and the benchmark data shows why it works so well on round pegs. Reproducing that behaviour from demonstrations alone is possible but you have to demonstrate it, many times, deliberately.

What success rate is a reasonable target

Calibrate expectations against published numbers rather than against a demo video. An industrial cell with force control, sub-0.1 mm camera calibration and roughly 0.14 mm control accuracy reached 96 to 100 percent on connector insertion. A sim-to-real system on a Franka with no force sensing reached 83 to 99 percent over 600 trials, after specific algorithmic work that lifted naive insert policies from about 12 percent. Behaviour cloning on a bimanual 50 Hz teleoperation rig got 96 percent on slotting a battery into a spring-loaded compartment and 20 percent on threading a velcro tie.

On a 110 to 150 EUR arm with a 0.088 degree encoder step and no force channel, a chamfered target with a millimetre or more of clearance is a task that fine-tuning can plausibly get into the high tens of percent. A sub-half-millimetre press fit is not a data problem, it is a hardware problem, and the correct response is to change the part, add a chamfer, or add compliance at the end effector with a piece of foam or a sprung finger. Design the task down to the arm before you scale the dataset up. If you want the wider background on how these models are built, the VLA overview and the pi-zero flow matching article cover the architecture side, and collecting high-quality VLA training data covers the recording side in more depth.