Your policy drives real robots- without a GPU of your own.
Upload GR00T, Pi0.5, SmolVLA, or ACT as a checkpoint, serve it in the cloud, and watch the arm move live in your browser. From checkpoint to moving arm in minutes.
- 100+
- policies ready to deploy
- 5
- architectures supported
- Minutes
- from checkpoint to endpoint
- Live
- watch in your browser
Cloud inference is for the model, not for the safety loop
A rented GPU serves the policy. Everything that must react within one frame stays on the machine beside the arm.
A card that serves GR00T N1.7 or Pi0.5 means an A100 with 80 GB of VRAM. Few teams own one. Cloud inference rents that card only while the arm is moving: the platform restores the environment for the model family, loads your checkpoint and hands the robot an endpoint URL. When you stop, the idle timeout destroys the pod.
The safety path stays local. The robotapp on the robot host owns the motors. Stop, pause, speed limit and both stages of the emergency stop are local HTTP routes (/inference/stop, /inference/pause, /estop/soft, /motors/limp) that work whether or not the GPU answers. The soft stage lowers the arm and releases torque; the hard stage cuts torque immediately.
Use cloud inference to test a fine-tune on the real arm, compare architectures on the same task, or show a policy to a customer without a GPU. Do not use it when the loop must close in under 30 ms end to end: ACT at about 20 ms per step belongs in the desktop client.
Latency per model, honestly measured
Inference time per action chunk on the serving GPU - the same values the dashboard shows you when choosing a model.
On top of that comes the network path between arm and GPU - the dashboard shows you both separately (p50/p99 live).
The inference panel in the cockpit
This is the panel your team starts a run from: model, trained policy, target (local or cloud), an instruction in plain language. The log shows provisioning, loading and the first predictions with p50 and p99 latency, and the emergency stop sits in the same window. The panel below is the real component with demo data.
Pick a model, then a trained policy (or the base model), and enter an instruction in plain language. The model computes joint targets and streams them to the robot until you stop the pilot.
Choose the model: GR00T, Pi0.5, SmolVLA, ACT or Cosmos.
Four ways teams use it
Same endpoint mechanics, different reasons to rent the card.
Test a fine-tune on the real arm, the same hour it finished
Training writes the checkpoint to s3://ay-robots/checkpoints/<dataset>-<model>-r<run>. The inference side resolves that path by the same rule, so the checkpoint is servable as soon as the last step is saved. You pick the policy in the cockpit, the platform checks that config.json and model.safetensors actually exist under that prefix, and only then rents a GPU.
A run that did not finish shows up as a plain error before a single GPU minute is billed.
- Task
- Verify a fresh GR00T N1.7 or Pi0.5 fine-tune on the SO-101 without a local A100.
- What goes in
- Checkpoint in the bucket, an instruction, a paired arm with cameras.
- What you get
- First predictions about seven minutes after the click; live p50/p99 in the dashboard.
Three steps to a moving arm
Upload a checkpoint
Your fine-tune from training - or a checkpoint from the marketplace. LeRobot format, no repackaging.
GPU starts for you
The platform rents the right cloud GPU, loads the model, and lets you know as soon as the endpoint is up - in minutes, not days.
The arm moves, you watch
The SO-100 pulls its actions from the serving API. In your browser you see cameras, joint angles, and latency live.
How an endpoint comes to be
Every step below is a real route in the platform. Nothing here is a sketch.
It starts with a checkpoint: one of your fine-tunes at s3://ay-robots/checkpoints/<dataset>-<model>-r<run>, or a base model from the allow list (nvidia/GR00T-N1.7-3B, nvidia/GR00T-N1.5-3B, lerobot/pi05_base). ACT and SmolVLA only exist as a trained policy. The resolver finds the loadable leaf, checks for config.json and model.safetensors and reports the size: about 12.6 GB for GR00T N1.7, 16.6 GB for Pi0.5.
Then the GPU. POST /api/inference/pod searches the GPU marketplace by VRAM, not by card name: 20 GB class (RTX 3090/4090) for ACT and SmolVLA, 80 GB class (A100/H100) for Pi0.5 and GR00T. Only datacenter hosts with a public port and 97 percent reliability qualify, EU first. The pod restores the prebuilt environment and starts inference_server.py on port 8765.
Loading is a single POST /load_policy with hf_repo_id and model_type. The server syncs the weights and answers GET /health with loaded=true. A pod serves exactly one policy; a second run of the same checkpoint reuses the warm pod. The platform passes the endpoint URL as sidecar_url to the robotapp, which sends observations and executes action chunks at 30 fps.
Billing starts when the pod is rented, not at the first prediction, and runs by the minute. The dashboard shows how many minutes your credit still covers. The pod keeps running until you stop it, so a forgotten endpoint costs money.
The requests, as the cockpit sends them
Two platform routes and two pod routes are enough for an endpoint. The cockpit uses exactly these; nothing is abstracted away.
Authentication: a logged-in session in the cockpit, or the robot's pairing code as Bearer (alternatively x-robot-token) from the robotapp. The pod URL is only ever known to the platform and to the robot.
# 1. Pod anfordern (Cockpit-Session oder Robot-Bearer)
POST /api/inference/pod
{ "modelId": "groot1.7",
"checkpoint": "s3://ay-robots/checkpoints/<dataset>-groot1.7-r<run>" }
-> { "status": "provisioning", "endpointUrl": null, "modelType": "groot1.7",
"resolvedRepo": "s3://ay-robots/checkpoints/.../checkpoints/last/pretrained_model",
"checkpointSizeBytes": 12600000000 }
# 2. Pollen, bis der Pod steht (lifesign=active haelt ihn warm)
GET /api/inference/pod?modelId=groot1.7&checkpoint=...&lifesign=active
-> { "status": "ready", "endpointUrl": "http://<host>:<port>", "error": null }
# 3. Auf dem Pod (inference_server.py, Port 8765)
POST {endpointUrl}/load_policy { "hf_repo_id": "s3://...", "model_type": "groot1.7" }
GET {endpointUrl}/health -> { "loaded": true, "inference_count": 1284 }
# 4. Lauf auf dem Arm starten; die Plattform reicht endpointUrl als sidecar_url
# an die robotapp (POST http://<robot>:8000/inference/start) durch
POST /api/inference
{ "robotId": "<robot>", "modelHfId": "s3://ay-robots/checkpoints/...",
"modelType": "groot1.7", "mode": "cloud",
"instruction": "pick up the cube and put it in the box" }
-> { "jobId": "<job>", "sidecarUrl": "http://<host>:<port>", "mode": "cloud" }
# 5. Status vom Roboter-Host: p50_ms, p99_ms, predictions, last_action, last_error
GET /inference/status
# 6. GPU bewusst freigeben (faellt in den Leerlauf-Abbau)
DELETE /api/inference/pod?modelId=groot1.7&checkpoint=...Technical data
Values from the platform code and the dashboard, not from a brochure.
- Models
- ACT, SmolVLA, Pi0.5, GR00T N1.5, GR00T N1.7
- model_type act, smolvla, pi0, groot1.5, groot1.7. Base models: nvidia/GR00T-N1.7-3B, nvidia/GR00T-N1.5-3B, lerobot/pi05_base.
- GPU tier per model
- RTX 3090/4090 class for ACT and SmolVLA, A100 80 GB for Pi0.5 and GR00T
- Selected by VRAM class; datacenter hosts only, reliability at least 0.97.
- Model latency per action chunk
- ACT 20 ms, GR00T N1.5 98 ms, GR00T N1.7 152 ms, SmolVLA 245 ms, Pi0.5 485 ms
- On the serving GPU; same values as the dashboard model picker.
- Network path
- 30 to 70 ms on top
- Arm to EU datacenter and back; p50 and p99 shown live.
- Cameras
- WebRTC to the dashboard, up to 5 cameras
- Including one wrist camera, 30 fps; observations go from robotapp to pod.
- Billing
- Per minute, no base fee
- Price fixed before start; warm pods count, released pods do not.
- Stop
- Any time, locally
- /inference/stop, /inference/pause, /estop/soft and /motors/limp run on the robot host.
- Checkpoint format
- LeRobot, config.json plus model.safetensors
- s3://ay-robots/checkpoints/<dataset>-<model>-r<run>; the resolver takes checkpoints/last/pretrained_model or the highest checkpoint-N.
- Cold start and lifetime
- About 7 minutes cold, warm reuse, 4 h maximum per pod
- Measured 402 s (GR00T N1.7), 450 s (Pi0.5); 25 minutes idle timeout.
Upload a checkpoint, choose the model, watch the arm. The first endpoint takes minutes.
Start an endpoint in minutesOwn GPU workstation vs. cloud inference
Both work. The difference is what you pay for the hours in which the arm is not moving.
| Criterion | Own GPU workstation | Cloud inference |
|---|---|---|
| Purchase | Four figures for a card that serves GR00T or Pi0.5, more for 80 GB. | None. You pay GPU minutes while the endpoint is running. |
| Utilisation | Idle most of the day; one arm rarely keeps a card busy. | Rented for the run, released 25 minutes after the last lifesign. |
| Model switch | Reinstall the environment per model family, then reload the weights. | New modelId, new pod with the prebuilt environment for that family. |
| Latency | No network path. ACT at 20 ms per step is best served here. | Model time plus 30 to 70 ms. Fine for chunked policies from GR00T upwards. |
| Maintenance | Drivers, CUDA, torch versions and disk space are your job. | Pod image, environment and checkpoint sync are part of the platform. |
| Scaling to several arms | One card per arm, or a queue in front of the card. | One pod per policy; a second arm on the same policy reuses it. |
Recommendation: keep ACT local in the desktop client, serve GR00T and Pi0.5 from the cloud.
Latency, honestly calculated
Model time plus network path, and why 485 ms still drives a 30 fps arm.
A 30 fps control loop has 33 ms per step. Read naively, only ACT fits, and only without a network in between. The numbers here are per action chunk, though: a sequence of future joint targets. With lerobot's defaults of chunk_size 100 and n_action_steps 100, one prediction carries about 3.3 seconds of motion; with n_action_steps 25 still 0.8 seconds.
The robotapp requests the next chunk while the current one runs. Pi0.5 needs about 485 ms plus 30 to 70 ms on the wire, roughly 550 ms end to end. As long as a chunk holds more motion than that, the arm never waits. It costs reaction time: the policy sees a scene up to half a second old.
p50 is the median of the round trip, p99 the slowest one percent. The dashboard shows both because the outliers decide whether a chunk arrives late: a p50 of 148 ms with a p99 of 212 ms means the buffer has to cover 212 ms. If p99 drifts toward the chunk length, the panel shows it before the arm stutters.
Pay when the arm moves - not otherwise
Cloud inference pays off because robotics GPUs would sit idle most of the time.
No base fee
You pay GPU minutes only while your endpoint is running. Stop it, and it costs nothing more.
Billed by the minute
The price of the chosen GPU class is fixed before you start - no surprises on the bill.
Instead of a €2,000 workstation
An inference-capable GPU costs four figures and mostly sits idle. Here you rent it only for the minutes the arm is actually moving.
No checkpoint yet? Train on your own or a marketplace dataset first.
Go to cloud trainingQuestions about cloud inference
Start with what your data supports. ACT trains fastest, runs locally at about 20 ms per step and needs no endpoint. For language conditioning and generalisation, GR00T N1.5 is the fastest large model at 98 ms per chunk; Pi0.5 the most accurate at 485 ms.
Related on AY-Robots
Teleoperation
Remote operators drive your arm and record demonstration data around the clock.
Read moreDesktop client
Record your own datasets with your arm and cameras, straight into LeRobot format.
Read moreCloud training
Fine-tune GR00T, Pi0.5, SmolVLA or ACT on a rented GPU, billed by the minute.
Read moreSimulator
Real teleop demos become varied simulation episodes, checked automatically and exported in LeRobot format.
Read moreSet up your first endpoint
Create an account, upload a checkpoint, start the GPU - the rest is just watching.