Remote Robot Operations for Small Companies: What It Takes

A remote robot deployment at a small company usually looks like this. One SO-100 sits on a bench in a workshop, a storeroom or a lab. Nobody in that room touches it during a shift. An operator two hundred kilometres away drives it from a browser. A rented GPU may or may not be in the loop. That is three separate network links with three completely different requirements, and the most common mistake is treating them as one.

The video feed going out to the operator can absorb a hundred milliseconds and the operator will barely notice, because people adapt. The command stream going back to the servos can absorb it too, for the same reason. The link between a camera and a policy cannot, because the policy does not adapt. It emits an action for a scene that has already moved on. Confuse the three and you get something that demos well and falls apart on a real task.

The three links, and why they are not one link

Only the third row is hard. A human on the first two rows is a closed-loop controller with an enormous tolerance for delay, which is why teleoperation over long distances has worked for decades. A trained policy on the third row has no such tolerance, and the tolerance it does have comes almost entirely from action chunking, not from the network being fast.

What published remote deployments actually measured

RoboTurk (Mandlekar et al., CoRL 2018) is still the most useful reference for anyone planning this, because it published the network experiment instead of asserting the conclusion. Remote workers drove a simulated 7-DoF Sawyer arm with an iPhone as a motion controller and a browser as a viewport, over two WebRTC channels: one from phone to teleoperation server, one from browser to teleoperation server. The pilot dataset was 137.5 hours of manipulation data and over 2,200 successful task demonstrations, collected in 22 hours of total system usage because many operators ran in parallel.

They then throttled the link with the Cellsim network emulator and re-ran the tasks. The result is the part worth remembering.

They also ran a real-world stress test: operators in California driving a deployment hosted in a Chinese datacentre roughly 6,500 miles away, against the same operators driving a deployment in Oregon roughly 500 miles away. The tasks still got done. Mean completion times differed by 24 seconds on the assembly task and 28 seconds on the picking task. That is the honest price of a long human link: slower, not broken.

The follow-up paper (IROS 2019) put the same platform in front of three physical Sawyer arms and collected 111 hours across 54 operators in one week, all of them remote, none of them in front of the robots. Read that paper for the operational details rather than the dataset. A mutual-exclusion lock allowed exactly one operator per arm at a time, because physical robots do not scale like simulator instances. Operator input was low-pass filtered to reject high-frequency commands so that stochastic hardware and network delays would not turn into oscillation. Every participant got a five-minute tutorial and a direct line to someone watching the robots, so unsafe control could be stopped by a person. The front-facing RGB camera served double duty as the operator's video feed and as a 30 Hz recording stream, with robot sensor readings logged at roughly 100 Hz.

Two more data points are worth having in your head. Telexistence announced on 10 August 2022 that its TX SCARA restocking robot would go into 300 FamilyMart stores across Japan starting that month; NVIDIA's write-up of the same announcement states the AI restocks beverages automatically more than 98 percent of the time, and that in the rare cases where it misjudges a placement, remote operators on standby take manual control through a VR system. And the DROID dataset shows the distributed version of this: 76,000 trajectories and 350 hours across 564 scenes and 84 tasks, collected by 50 people in North America, Asia and Europe over twelve months. Remote and distributed operation is not experimental. It is how most of the modern data was collected.

The network you actually need at the site

Start from what actually crosses the wire. LeRobot's own examples run cameras at 640x480 or 1920x1080 at 30 fps. Operator video is the cheap half: RoboTurk ran full teleoperation sessions inside a 2.4 Mbps cap, and throttling that to 500 kbps did not produce a statistically significant change in completion time. The number that surprises people is on the model side: LeRobot's asynchronous inference path uses gRPC, whose default maximum message size is 4 MB, and the project notes that observations with multiple high-resolution cameras routinely exceed it. Video for humans is cheap because it is a compressed stream. Observations for a model are expensive because they are individual frames.

The harder problem is reachability, not throughput. A workshop router does outbound NAT, has no static address, and may sit behind carrier-grade NAT with no public address at all. The standard answer is a WireGuard mesh, and Tailscale's own write-up of NAT traversal estimates that a direct connection is achievable more than 90 percent of the time using STUN, ICE-style path racing and port mapping. The other cases fall back to a relay, and a relay is a detour you did not budget for.

1. 1
   Put the site host on a mesh

   Give the machine next to the arm a stable address that does not depend on the site's router. Name it something you will recognise in a status listing six months from now.

   bash

   curl -fsSL https://tailscale.com/install.sh | sh
   sudo tailscale up --hostname=shopfloor-arm

2. 2
   Confirm you got a direct path, not a relay

   The status output names the connection type per peer: direct with the peer's address, or relay with a DERP city code such as nyc, fra, tok or syd. A city code on the far side of the planet means your packets are taking a detour.

   bash

   tailscale status
   tailscale ping shopfloor-arm

3. 3
   Open the right things on the site firewall

   Outbound TCP 443 reaches the coordination server and the relays. Direct WireGuard tunnels use UDP with a source port that defaults to 41641, and STUN uses UDP 3478. If the site blocks outbound UDP, you will work, but every packet is relayed.

   bash

   # outbound, from the site host
   # TCP 443   coordination server and DERP relays
   # UDP 41641 direct WireGuard tunnel (default source port)
   # UDP 3478  STUN

4. 4
   Measure the round trip before you design around it

   Do this at the hour the arm will actually be used, not at 2am. Record the median and the tail. The tail is what makes an arm stutter.

   bash

   ping -c 200 shopfloor-arm | tail -2

Driving the arm: the manual path

Nothing here is platform-specific. LeRobot ships the whole chain as command-line entry points, and the split between a host process at the robot and a client process elsewhere already exists in the codebase for the LeKiwi mobile base. That split is the pattern to copy for a static arm as well.

1. 1
   Install LeRobot on the machine next to the arm

   The Feetech extra pulls the servo SDK. The lekiwi extra adds it plus pyzmq, which is what the host and client processes talk over. Install the same thing on the operator's machine.

   bash

   git clone https://github.com/huggingface/lerobot.git
   cd lerobot
   pip install -e ".[feetech]"    # SO-100 / SO-101 servo bus
   pip install -e ".[lekiwi]"     # adds pyzmq for the host/client split

2. 2
   Find the serial port and calibrate

   Calibration is per physical arm and is stored against the id you give it. Use the same id when you teleoperate, record and evaluate, or the numbers will not line up.

   bash

   lerobot-find-port
   
   lerobot-calibrate \
       --robot.type=so100_follower \
       --robot.port=/dev/ttyACM0 \
       --robot.id=shopfloor_arm

3. 3
   Prove it works with everything in one room first

   Run leader and follower on the same machine, with cameras, before you introduce a single network hop. If it is not smooth here, the network will not be the reason it is not smooth later.

   bash

   lerobot-teleoperate \
       --robot.type=so100_follower \
       --robot.port=/dev/ttyACM0 \
       --robot.id=shopfloor_arm \
       --robot.cameras="{front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
       --teleop.type=so100_leader \
       --teleop.port=/dev/ttyACM1 \
       --teleop.id=bench_leader \
       --display_data=true

4. 4
   Split it across the network

   LeKiwi's host process is the reference implementation of the site side. It publishes two ZMQ endpoints: commands on port 5555 and the video stream on 5556. The client connects to tcp://<site-ip>:5555 and tcp://<site-ip>:5556, and on a mesh that site address is the mesh address, not a port forward.

   bash

   # on the site host, next to the arm
   python -m lerobot.robots.lekiwi.lekiwi_host --robot.id=my_awesome_kiwi
   
   # operator side, after setting remote_ip and port in the example
   python examples/lekiwi/teleoperate.py
   # [INFO] Connected to remote robot at tcp://172.17.133.91:5555
   #        and video stream at tcp://172.17.133.91:5556.

5. 5
   Record while you operate

   Every remote session is a dataset if you record it. The defaults are 60 seconds per episode, 60 seconds of reset time, and 50 episodes, which is exactly the minimum episode count most policies on this platform want.

   bash

   lerobot-record \
       --robot.type=so100_follower \
       --robot.port=/dev/ttyACM0 \
       --robot.id=shopfloor_arm \
       --robot.cameras="{front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
       --teleop.type=so100_leader \
       --teleop.port=/dev/ttyACM1 \
       --teleop.id=bench_leader \
       --dataset.repo_id=${HF_USER}/shopfloor-pick \
       --dataset.num_episodes=50 \
       --dataset.single_task="Pick the bracket and place it in the tray"

Where the model fits, and where it must not

There are only four placements, and choosing between them is arithmetic, not taste. The control period sets the budget: 50 Hz means 20 ms, 30 Hz means 33 ms. Everything the network adds comes out of that. On this platform the model itself already eats most of it, with ACT at 20 ms per action step, GR00T N1.7 at 152 ms, SmolVLA at 245 ms and Pi0.5 at 485 ms. Compare the five side by side on the policies page before you decide where any of them should run.

The cloud-GPU row is the one people get wrong, so it is worth being precise about what makes it survivable. Action chunking means the policy emits a batch of future actions at once, and asynchronous execution means the robot keeps consuming the current chunk while the server computes the next one. LeRobot's async inference exposes exactly two knobs that matter: actions_per_chunk, how many actions come back per request, and chunk_size_threshold, how empty the queue must get before the client sends a fresh observation. The defaults in the code are 30 fps, a threshold of 0.5, and a weighted average that blends 30 percent of the old chunk with 70 percent of the new one on the overlap.

# both ends need the async extra
pip install -e ".[async]"

# on the GPU machine
python -m lerobot.async_inference.policy_server \
     --host=0.0.0.0 \
     --port=8080

# on the site host, next to the arm
python -m lerobot.async_inference.robot_client \
    --server_address=gpu-box:8080 \
    --robot.type=so100_follower \
    --robot.port=/dev/ttyACM0 \
    --robot.id=shopfloor_arm \
    --robot.cameras="{front: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
    --task="Pick the bracket and place it in the tray" \
    --policy_type=smolvla \
    --pretrained_name_or_path=user/my_policy \
    --policy_device=cuda \
    --actions_per_chunk=50 \
    --chunk_size_threshold=0.5 \
    --aggregate_fn_name=weighted_average

How far you can push this is a published number rather than a guess. The real-time chunking paper (Black, Galliker and Levine, NeurIPS 2025) ran Pi0.5 with a 50-step horizon at a 20 ms control period and five denoising steps, giving a model latency of 76 ms for the baselines and 97 ms for their method, over remote inference on a LAN that added 10 to 20 ms. They then injected an extra 100 ms and 200 ms to simulate worse links. Their method held throughput flat across all of it and succeeded at tasks as precise as lighting a match with inference delays in excess of 300 ms. The temporal-ensembling baselines could not run at plus 100 or plus 200 ms at all: the oscillation was severe enough to trigger the robot's protective stop. The same paper notes that an optimised 7B OpenVLA achieves no better than 321 ms of latency on a server-grade A100, and that Pi0 reports 13 ms of network latency in perfect conditions on a wired link.

• If your task needs reactive correction inside 100 ms, the model goes at the site. There is no network trick that fixes this.
• If your task is a slow pick-and-place with a settled scene, a cloud GPU is fine, and action chunking is doing the work, not the network.
• If you see the arm stutter at chunk boundaries, that is an inference-delay symptom, not a policy quality symptom. LeRobot's lerobot-rollout exposes --inference.type=rtc for exactly this on slow VLA models.
• If the arm freezes mid-motion rather than stuttering, that is a different failure. See policy freezes mid-motion.

Doing it yourself, or doing it here

What it costs to stand one up

The hardware is the cheap part, which is the whole reason this is worth doing at a small company. The expensive parts are the ones nobody puts in the spreadsheet: the person who has to be near the arm, and the hours spent finding out that your link is relayed.

The by-product: you are collecting a dataset either way

The strongest argument for putting an arm somewhere and driving it remotely is not the driving. It is that a recorded remote session is a LeRobot dataset, and a LeRobot dataset is the input to every model on this platform. DROID exists because 50 people in three continents each recorded their own sessions on their own arms. Your version of that is smaller, but the mechanism is identical: the work you were doing anyway becomes the training data for the checkpoint that eventually does part of it for you.

The minimum episode counts are concrete. SmolVLA wants 30 episodes, and ACT, GR00T N1.5, GR00T N1.7 and Pi0.5 each want 50. At the LeRobot default of 60 seconds per episode plus 60 seconds of reset, 50 episodes is under two hours of operator time. If you are already paying someone to drive the arm for a shift, you are already paying for the dataset. Browse the public dataset directory to see what other people's recorded sessions look like before you commit to a recording setup of your own, and read the data quality guide before you record fifty episodes you cannot use.

Where remote operation does not help

This is the part most write-ups skip. Remote operation moves the operator, not the physics. Everything that requires a hand in the room still requires a hand in the room, and in a small company that hand belongs to somebody who has another job.

• Resets. The RoboTurk team named this as their top future-work item: alleviating the need for manual task resets, by collecting reversible tasks so remote operators could reset the workspace themselves. They had not solved it. Neither has anyone else.
• Anything on the floor. A dropped part ends the session until someone picks it up.
• USB and serial faults. A wedged adapter needs a physical unplug, or a scripted power cycle you built in advance. See arm not detected.
• Camera drift. A bumped camera silently invalidates every episode after it, and the operator watching the feed is the least likely person to notice, because the feed still looks fine to them. See policy only works in one setup.
• Safety. There is no remote equivalent of standing next to a machine and seeing that somebody has walked into its reach.

A short checklist before you put an arm in a room you are not in

1. Measure the round trip at working hours and write down the median and the 99th percentile.
2. Confirm the mesh peer is direct and not relayed, and check it again after the site changes routers.
3. Prove teleoperation with everything in one room before you add a single network hop.
4. Decide the model placement from the control period, not from where the GPU is convenient.
5. Bolt the cameras. Then bolt them again.
6. Label the 7.4 V supply so the person who reconnects it after a power cut cannot get it wrong.
7. Name the person on site who gets called, and write down what they are allowed to do.
8. Record every session from day one. The dataset is free at collection time and expensive to recreate.

If you want to feel the delay before you commit budget to any of this, the fastest honest test is to drive a real arm over the public internet yourself and see what your own connection does to it. Then read the RoboTurk write-up for the research version, and the SO-100 setup guide for the hardware you would be putting in that room. If you plan to staff this with more than yourself, the operator side and the security docs are the two pages to read next.