Inference latency

  • inference time
  • latency
  • ms per action step
  • policy control rate
Definition

Inference latency is the wall-clock time a policy needs to produce one action step, measured in the cloud pool for each model here. It runs from about 20 ms for ACT to about 485 ms for Pi0.5, which is the difference between an arm that moves continuously and one that visibly pauses between decisions.

Last updated 2026-08-09

The measured numbers

ModelPer action stepDecisions per secondHow it reads on the arm
ACTabout 20 msroughly 50Continuous motion, no visible gap between decisions
GR00T N1.7about 152 msroughly 6 to 7Fluent, with a perceptible rhythm
GR00T N1.5about 165 msroughly 6The same character as N1.7, marginally slower
SmolVLAabout 245 msroughly 4Deliberate and visibly stepped
Pi0.5about 485 msroughly 2Considered, with short settles between moves

Two things in that table are worth pausing on. Latency does not track parameter count: GR00T N1.7 is about 3 billion parameters at about 152 ms and is faster than SmolVLA at about 450 million and about 245 ms, because how the actions are generated matters more than how many weights are involved. And it does not track the GPU tier either, since ACT and SmolVLA train on the same 24 GB tier and are an order of magnitude apart when they run.

What the number does and does not decide

It is not the rate at which the arm moves. These policies emit chunks of actions, so the arm keeps executing while the model thinks. A 485 ms policy is not a two hertz robot, it is a robot that reconsiders the scene about twice a second. What latency really sets is how quickly the policy can respond to something changing.

  • Static scene, object where you left it: all five are fast enough, and the choice comes down to accuracy instead.
  • Object that can shift mid-episode: you want frequent replanning, which means low latency, which in practice means ACT.
  • Anything reactive, such as tracking a moving target or catching a slipping grasp: roughly two decisions per second is not enough, and no chunk length repairs that.
  • A repetitive cell: the pauses accumulate over hundreds of cycles even when the success rate is identical.
Where inference runs is a separate question

ACT and SmolVLA are not cloud-only, so the desktop client runs those checkpoints on your own machine and everything stays on the bench. GR00T N1.7, GR00T N1.5 and Pi0.5 are cloud-only, which puts a network round trip on top of the model’s own time.

/policies lists the figure for each model next to what it costs to train, and /learn/run-your-first-policy is where you first see what those milliseconds look like on hardware.