Live teleoperation

The browser cockpit

Open an environment and switch to the Control tab. If the robot is online, meaning its heartbeat is arriving, the cockpit shows the camera feeds and the control interface. Operators use it during sessions; clients use the same cockpit to test their own arms after pairing or between bookings.

If you want to feel the controls without an account, there is a public demo at /live. It drives the follower arm of a real demo rig, so what you see there is the production control path, not a simulation.

Keyboard control: impulses and cooldown

Control is impulse-based. A keypress sends one bounded impulse to a joint, and a short cooldown follows before the next impulse is applied. Holding a key repeats impulses at the cooldown rate. The arm therefore moves in small, predictable increments instead of following your key press continuously.

This design is deliberate. Over a network connection, a continuous control signal turns latency spikes into overshoot: the arm keeps moving on stale input. Bounded impulses cap how far any single command can take the arm, which makes control predictable even when the connection wobbles, and on a decent connection the latency is low enough for natural control. The active key map is displayed in the cockpit, so you never have to memorize bindings.

Practical technique: think in taps, not holds. Tap toward the target, watch the result, tap again. Operators who fight the cooldown by hammering keys produce exactly the jerky trajectories that make datasets worse; operators who work with it produce smooth, reviewable episodes.

Cameras and stream quality

The cockpit shows all cameras of the robot in a grid; click a tile to switch to a single-camera view and back. The camera names come from the client setup, so a well-configured robot gives you tiles like wrist and scene rather than anonymous numbers. Which view to use is a matter of task phase: the scene view for approach and orientation, the wrist view for fine work around the grasp.

A quality slider trades image quality against latency. Lower quality means less data per frame and a snappier feed; higher quality means more detail but more delay. For active control, bias toward responsiveness; for inspecting a scene while the arm is idle, turn the quality up. The right setting depends on your connection, which is why it is a slider and not a fixed value.

Leader and follower mirroring

For local rigs with two arms, the platform supports mirroring: you move a leader arm by hand and the follower reproduces the motion. This is the classic setup for recording imitation learning data, because human hand motion through a leader arm is far smoother than any key-based input.

1. 1
   Bring both arms into the same start pose

   Physically guide the leader and the follower into matching positions. The closer the match, the faster the next step completes.

2. 2
   Enable mirroring in the cockpit

   The platform starts comparing joint positions between the two arms, but the follower does not move yet.

3. 3
   Wait for the release

   Mirroring engages only when every joint pair is close together. This gate prevents the follower from snapping violently to the leader pose the moment mirroring turns on.

4. 4
   Drive the leader

   Move the leader by hand; the follower tracks it. Work deliberately, as you would when teleoperating remotely: the smoothness of your hand motion is exactly what ends up in the dataset.

Direct mode and the proxy

The cockpit has two transport paths. In direct mode the browser streams straight from the robot, which is the lowest-latency path and kicks in automatically when the robot is reachable from your network, for example on the same LAN. When it is not reachable, the connection runs through the platform proxy instead. The fallback is automatic and needs no configuration; you simply get the best path available from where you are sitting.

Emergency stop

The cockpit has an emergency stop that immediately halts the command flow to the robot. Reach for it whenever the arm is doing something you did not intend, before you analyze why. After a stop, resume control deliberately once the scene is safe.

Frequently asked questions