Teleoperation

  • teleop
  • tele-operation
  • remote robot control
  • robot arm teleoperation
Definition

Teleoperation is a human driving the robot in real time instead of a policy driving it. On this platform it is how every demonstration gets recorded: from the browser cockpit, from the desktop client, or by moving a second physical arm that the robot mirrors joint for joint.

Last updated 2026-08-09

The three ways to drive an arm here

MethodWhat you needWhere it fits
Browser cockpitA browser, nothing installedTrying the loop before you own hardware, or driving a paired arm from another room
Desktop clientThe client from /download and an arm on USBEveryday recording. The driver ships with it, there is no Python environment to set up
Leader armA second arm of the same designThe cleanest demonstrations, because the follower stays torqued and your hand never carries the arm

Driving and recording are separate jobs. The record command opens and closes episodes and writes them to disk, it does not move anything: you teleoperate throughout. Enter ends an episode and keeps it, Ctrl-C discards the one in progress rather than saving half an attempt.

bash
# Ten episodes, two cameras, one task sentence you reuse every time
ay-robots record --task "pick up the red cube and place it in the bowl" \
  --episodes 10 --cameras wrist,top
The recorder runs while you drive. It never drives for you.
The arm at /live is real hardware

The browser cockpit at /live drives a physical SO-100, with no signup and nothing to install. It is the cheapest way to learn how a five joint arm answers your inputs before deciding how many of them you want on your desk.

Why your driving shows up in the policy

A policy trained by imitation reproduces the motion style it was shown, not an idealized version of it. Nervous corrections in your demonstrations come back as nervous corrections. Pace matters in both directions: driving faster than you can be accurate produces a dataset of near misses, and an episode driven at a pace the model cannot decide at will not replay the same way, since Pi0.5 at about 485 ms per action step reconsiders the scene roughly twice a second.

Keep the operator constant too. Two people with different habits recording into one dataset is the same problem as one person switching strategy halfway through, and it is harder to spot afterward because both halves look competent.

The mechanics of a session are in /learn/record-your-first-dataset, and /so-100/data-collection covers what separates a usable episode from a wasted one.