Scripted Motion or a Trained Policy on an SO-100 Arm

Two different jobs that happen to share an arm

There are two ways to make a small arm do something useful, and they are usually discussed as rivals. One is to work out where things are and write the motion: joint angles, in order, driven to and verified. The other is to demonstrate the task a few dozen times and train a network that maps what the cameras see to the next joint command, which is what imitation learning means and what the five models on this platform do.

Framed as a rivalry the question has no answer. Framed as a division of labour it usually answers itself in a minute: a written motion is a way of stating knowledge you already have exactly, and a policy is a way of acquiring knowledge you cannot state. Nobody can write down how hard to squeeze a soft object, what a half-occluded cube looks like from the wrist camera, or when a grasp has already failed. Equally, nobody should record fifty demonstrations to teach an arm to travel between two coordinates it could simply be told.

What follows is the decision, with the numbers this platform actually runs on rather than in the abstract: what the written path can and cannot do on an SO-100 or SO-101, what a policy costs in episodes, hours and dollars, and six questions that in practice settle which side of the line a task falls on. The workflow is identical on both arms, so read SO-100 throughout as SO-100 or SO-101.

What writing the motion actually means on this hardware

The concrete form of the written path here is the local MCP server that ships with the CLI, started by ay-robots mcp. It hands an agent nine tools on the arm that is physically attached to that machine, and it is worth reading them as a list rather than as a description, because the list is the argument: get_status, list_devices, get_joint_positions, move_home, move_joint, start_recording, stop_recording, get_recording_status and list_local_datasets. The copy-paste blocks for wiring it into a client live on the MCP page and the full command list on the CLI reference; the walk through what an agent actually does with those nine is in driving an SO-100 over MCP, and none of it is repeated here.

The only motion primitive in that set is a single joint moved to a single absolute angle in degrees. There is no Cartesian target, no inverse kinematics on the robot side, and no trajectory object that can be stored and replayed. A sweep is a sequence of calls, and everything between them, the ordering, the waiting, the checking, is the agent's own loop. That is the honest shape of it: the platform does not hold a program, it executes one command at a time and reports back what happened.

Every one of those commands passes the same set of guards in the robot backend before it reaches a servo. They matter for a written motion because they are what makes a naive script produce a different result than the one on paper:

One more property decides more than any of the above: none of the nine tools returns an image. The backend does serve camera frames and streams over HTTP, but no tool exposes them, so an agent driving through this surface is working from joint angles and whatever a human has told it. That is a deliberate boundary, and it is the single biggest reason a written motion cannot do a policy's job. The two motion tools also refuse to run at all without an explicit confirm flag, which is checked before any HTTP call leaves the process.

The run this article grew out of

In August 2026 Anthropic published the Model Hardware Standard, a specification for letting agents operate physical equipment, and opened a research preview. AY-Robots took part in that preview. In our run an agent measured the workspace of an SO-ARM101 itself and wrote the motion: no policy, no teleoperation, no demonstrations. Reported from that setup were 4.1 mm positioning accuracy and 3.0 mm placement accuracy, and the best run so far placed 12 bricks and formed all four colour groups.

A best run is not a success rate, and a fully hands-off run from start to finish has not happened yet. Nothing in the decision below rests on those numbers; they are here because they are the reason we started sorting tasks this way in the first place. The run itself is written up in moving an SO-ARM101 without a policy, the calibration procedure in our SO-ARM101 calibration report and its general form in hand-eye calibration without a checkerboard, and the specification itself in what MHS changes for your robot arm.

What a policy costs, in the units you actually pay in

The GPU bill is the part everyone asks about and the part that matters least. Five models can be trained from this platform, and the run itself is cheap on all of them. What differs is how much data each needs before it does anything useful, and how fast it answers once it is running.

Those hourly figures are spot market ranges rather than a quote, because the training pool rents by the hour and picks a card by the VRAM the model needs. The same numbers are on the pricing page and the same table is what the hosted MCP server returns from its cost estimator, so an agent asked what a run will cost gives the answer the page gives. Per-step latency is worth a second look before choosing: at 485 ms Pi0.5 is visible as hesitation on fast motions, which is a property of the deployment and not of the training bill. The term is explained under inference latency.

More episodes do not make the run longer

The question that follows the table immediately: if fifty episodes cost two to five hours, does a hundred cost twice that? No. Training here is step based rather than epoch based, so the episode count does not change how long a run takes; it changes what each step samples from. The cost estimator on the hosted server states that in its own answer rather than leaving it to be inferred, which is the reason to call it before renting anything. Asked about ACT with thirty episodes, it returns this:

{
  "policy":   { "slug": "act", "name": "ACT", "trainer": "act" },
  "gpu_tier": "RTX_4090",
  "gpu":      "RTX 4090 or any card with 24 GB",
  "steps":    { "used": 100000, "model_default": 100000, "scale_factor": 1 },
  "estimate": {
    "hourly_usd": "0.30 to 0.60 USD",
    "duration":   "2 to 5 hours",
    "total":      "about 1 to 3 USD"
  },
  "episodes": {
    "given": 30,
    "minimum_for_this_model": 50,
    "verdict": "30 episodes is below the 50 this model usually needs. The run costs the same either way; it is the success rate that suffers."
  },
  "assumptions": [
    "Training is step based, not epoch based, so the episode count does not change how long a run takes. It changes what each step samples from.",
    "Duration and total are for a single run at 100000 steps on the RTX 4090 or any card with 24 GB tier.",
    "Scaling with the step count is linear, which holds for the training loop but ignores queue time and checkpoint uploads."
  ]
}

The step count, not the episode count, is what the duration hangs on, and the five models do not share a default schedule. That is the number to look at before assuming two runs on the same card cost the same:

Two of those batch sizes need a footnote before anyone reads them as a hardware statement. The training form suggests a gradient accumulation value next to the batch size, 16 for Pi0.5 and GR00T N1.5 and 8 for SmolVLA, but it only actually reaches the trainer on the two GR00T entry points. For ACT, SmolVLA and Pi0.5 the batch size in the table is the batch size that trains.

The real cost is the row on the left. Fifty kept episodes means teleoperating the same task over and over, and the recording guide deliberately splits that across sittings: three sessions of twenty produce better data than one session of sixty, because demonstration quality drops measurably once you are bored. The discipline that goes with it is the rest of the cost: cameras fixed so they cannot be nudged, constant lighting, the same task sentence every time, the object position varied but the strategy not, and every failed attempt discarded rather than kept. The step-by-step version is in record your first dataset, and the reasoning behind the discipline in data collection for the SO-100. A failed grasp in the training set is not a neutral data point; it teaches the policy to fail.

The check answers in the dashboard's own words, which are German, and the three sentences it can return are worth recognising before a run rather than after one:

ACT kann diesen Datensatz nicht verwenden: Er liegt im LeRobot-Format v2.1,
ACT, SmolVLA und Pi0.5 benötigen v3.0. Nimm GR00T N1.7 oder N1.5 - oder
konvertiere den Datensatz nach v3.0.

GR00T N1.7 kann diesen Datensatz nicht verwenden: Er liegt im LeRobot-Format
v3.0, GR00T benötigt v2.1. Nimm die v2.1-Fassung dieses Datensatzes.

Liegt im LeRobot-v2.1-Format (GR00T) und nicht auf HuggingFace - ACT, SmolVLA
und Pi0.5 benötigen v3.0 vom Hub.

The first of the three is the one nobody expects, because it fires on datasets that are demonstrably on the Hugging Face Hub. What the check measures is codebase_version in meta/info.json, not whether the repository resolves: a v2.1 tree that exists on the Hub is exactly what a Pi0.5 run once died on, with lerobot raising a backward-compatibility error the moment it opened the dataset. The third message covers the mirror image, a dataset whose format could not be measured and that is provably not on the Hub, which is how a copy that lives only in the bucket fails four or five minutes in with a repository-not-found error, by which time the card is rented and billing. Everything unclear in between deliberately passes: when the format cannot be measured at all, the dataset counts as compatible and the guard in the start route stays the last line of defence.

Conversion runs in one direction on its own. The lerobot package converts upward, v2.1 to v3.0; going back down to the GR00T layout is a separate script on the training server rather than a button. The practical rule is to upload the converted copy under a new name instead of replacing the original, then re-read total_episodes on the copy, because a conversion that silently dropped episodes does not announce itself. The full procedure, both directions and the checks, is on the fix page for a rejected dataset version.

Training has no CLI command, which surprises people who look for one: it runs on the rented card and is started from the web dashboard or through the hosted server's training tool. The commands that do exist are the ones around it, recording, listing, uploading, and running a finished policy back on the arm.

The two paths are not allowed the same speed

Neither table above shows this, and for some tasks it decides more than the latency column does: the same backend holds a written motion and a running policy to different speed limits, and the policy's are the tighter ones. Per action step a policy is quick, 20 milliseconds for ACT. Per second of travel on the arm it is the slower of the two, and deliberately so.

Two details in that right-hand column are worth understanding rather than memorising. The degrees-per-second ceiling is invariant to the control rate on purpose: raising the loop from 30 Hz to 60 Hz shrinks the per-tick cap in the same proportion, so a faster loop buys smoother motion and never a faster arm. And the cap is measured against the last command sent rather than the last position read, because during inference the state read contends with the write path and can go stale. A clamp that compares against a stale position is a clamp that has stopped clamping, which is the failure this construction is built to avoid.

For the decision the consequence is narrow but real. If the uncertain part of your task has to happen faster than 40 degrees per second, that is a property of how the policy is deployed here and not of the policy you trained, and no amount of demonstrations changes it. If the geometric part of your task is a long transit, the written path will cross it faster than a policy will. It is one more reason why the seam described further down tends to fall where it does.

Six questions that settle it

Sorted by task property rather than by technology, most cases decide themselves. The last column is the part worth doing before the argument, because it is cheaper than either approach.

Two rows pulling in opposite directions is the normal case, not a failure of the table. It usually means the task has a geometric half and an uncertain half, and that the interesting question is where the seam runs rather than which column wins.

Where a written motion runs out

The limits are not a matter of effort. They follow from what the approach is:

• Anything compliant or deformable. There is no coordinate for how far a cable has slid in the gripper or how a cloth has folded. A joint angle cannot express it, so a written motion cannot aim at it.
• Anything that has to react to the camera at control rate. Reacting to an image is what a policy is; a sequence of joint targets is not a feedback loop, and on this tool surface the agent does not receive images at all.
• Variation the author did not anticipate. Every new case is another branch somebody has to write. The branch does not exist until it is written, and the arm does not know that.
• Perception the agent does not have. If a written motion is to pick up an object that moves, something else has to say where it is, and the quality of the whole thing is the quality of that step.

There is a useful external data point on how far the approach reaches when the perception problem is taken away. Demonstration-Free Robotic Control via LLM Agents runs an unmodified frontier agent framework on the LIBERO, ManiSkill3 and MetaWorld benchmarks and reports success rates of 84.9, 85.7 and 96 percent, with privileged environment state access, which the paper says approaches VLA models trained on fewer than 100 demonstrations per task. The qualifier is the whole point. Those are simulated benchmarks in which the agent is handed the state of the world. On a desk, nothing hands you the state; establishing it is the work.

The other half is written into the problem statement of the ACT paper, which opens by asking whether learning can enable low-cost, imprecise hardware to perform fine manipulation tasks such as threading cable ties or slotting a battery, tasks it describes as requiring precision, careful coordination of contact forces, and closed-loop visual feedback. Two of those three are things a written motion has no way of producing.

Where a policy is expensive

The costs run in the opposite direction and they are mostly human:

• Collecting the data. Thirty to fifty consistent episodes of one task, driven by hand. This is the bill, and it is paid in attention rather than money.
• Keeping the recordings clean. A nudged camera invalidates everything recorded before it was nudged, because the policy learns pixel positions. Lighting, background and the task sentence have to hold still too.
• The training run. Two to six hours during which you learn nothing about whether it worked, because the loss curve is a smoke signal rather than a measure of success.
• Evaluation on hardware. Success rate on physical attempts is the only metric that counts, and producing it means running the arm again and counting.
• The second round. Where it fails, you record more demonstrations of exactly that situation. Targeted data beats more data, but it is another session on the arm either way.

That last point deserves the qualifier that the papers themselves put on it. The DAgger paper states the underlying problem plainly: in sequential prediction, future observations depend on previous actions, which violates the assumptions of ordinary statistical learning and leads to poor performance in theory and often in practice. The ACT authors name the same effect from the practical side, that errors in the policy can compound over time. A trained policy drifts in exactly the states that were thin in your data, and no amount of GPU fixes that. Correcting a running policy and feeding those corrections back is the DAgger loop, which is the one part of this territory that runs on the platform today.

The platform's own guide to imitation learning on the SO-100 sorts the same ground into what works and what struggles, and its "struggles" column is worth reading before committing an afternoon: millimetre precision, long-horizon tasks with many sub-goals, situations far outside the demonstrations, and anything needing force feedback the arm cannot sense. The SO-100 has no force-torque sensing, which is the reason the last item is on the list. The data half of that bill, from the first recorded episode to a dataset somebody else can download, is in from first recording to a listed LeRobot dataset.

Splitting one task along the seam

Most tabletop tasks are not uniformly hard. Approach, retreat, moving between stations, homing after a fault, driving to a fixed start pose before an evaluation: geometry, all of it, and all of it cheaper to write than to demonstrate. The part where the arm touches something uncertain is usually a few seconds long and is the part worth spending demonstrations and GPU hours on. Recording an operator driving a straight line between two known coordinates fifty times spends the expensive resource on the cheap half of the problem.

Splitting deliberately along that seam also improves the dataset, because every episode then contains the part that actually needs to be learned. Whether the split is worth the extra machinery depends on how long the uncertain segment is relative to the whole task. If the whole task is uncertain, there is no seam and the policy does all of it. If none of it is, there was never a reason to record.

If you are unsure, do it in this order

The cheapest sequence is the one that spends the reversible effort first. Every step here either answers the question or produces something the next step needs.

1. 1
   Check what the machine actually sees

   Arms and cameras first. If a camera is missing here, everything downstream of it is guesswork, and it is worth fixing before anything else.

   bash

   ay-robots devices

2. 2
   Set the zero point

   Guide the joints to the neutral pose and press Enter. Every joint angle after this, written or learned, is relative to what you capture here, which is why it comes before both paths.

   bash

   ay-robots calibrate

3. 3
   Try the geometry by hand

   Drive the poses your task needs, one joint at a time, and read the positions back between commands. If the task falls out of this in an afternoon, you are done and no dataset was needed. If it does not, you now know precisely which part resisted, which is the part to record.

4. 4
   Record only that part

   Aim for fifty kept episodes, thirty if you are starting with SmolVLA. Record appends to a dataset that already exists, so create it first in the web interface, on the Manual Control recording card or on the Datasets page. Enter ends an episode, Ctrl-C throws away the one in progress.

   bash

   ay-robots record --dataset red_cube_pick --task "pick up the red cube and place it in the bowl" --episodes 10
   ay-robots datasets list --local

5. 5
   Upload and train the small model first

   Upload takes the dataset id, not a path. Then train SmolVLA or ACT before anything larger: a foundation model on thirty inconsistent episodes performs worse than ACT on the same data and takes far longer to tell you so. Training itself is started in the dashboard or through the hosted MCP server, not from the CLI.

   bash

   ay-robots datasets upload red_cube_pick --name "SO-100 red cube pick v1"

6. 6
   Run it back and count

   Without a checkpoint this runs the base model, which is a useful smoke test before your own policy exists. With one, it is the only measurement that means anything. ay-robots stop ends the policy and leaves the backend up.

   bash

   ay-robots run --model act --checkpoint <hf-repo-id-or-s3-uri> --task "pick up the red cube and place it in the bowl"

Two points where the sequence refuses instead of guessing

Most of what can go wrong on the hardware side is the same list for any way of driving this arm, and it is written out message by message in driving an SO-100 over MCP. Two refusals belong to this sequence in particular, because they sit at the two points where it commits to something that is awkward to undo.

The third thing that can go wrong here is not on the arm at all, and it is the expensive one: a dataset in the wrong LeRobot version. The compatibility check above catches it before a card is rented and the start route refuses it afterwards, which is worth using rather than trusting, because this is the one step in the sequence that rents a GPU by the hour.

If you have no arm yet, the order collapses to one step: a real SO-100 hangs on the live page and can be driven from the browser without an account or an install, which is enough to see what the geometric half of any of this feels like. The client itself is on the download page, and the walk through the whole chain from a bare arm to a running policy is under run your first policy.

What we have not established

We have not run the two approaches against each other on the same task under the same conditions, and until that exists any statement about which is more reliable on a given task is an opinion. The figures from our own run are a best case on one setup, not a success rate. The episode minimums in the table are the thresholds this platform uses, drawn from what these models need in practice, not measurements from a controlled study. And the seam described above is a way of thinking about a task, not something the software does for you.

What is solid is the shape of the trade: geometry is cheap to write and expensive to demonstrate, contact is impossible to write and learnable from demonstration, and the cost of finding out which one your task is amounts to an afternoon with the arm.

The next concrete step depends on which column your task landed in. If it is geometry, the local server and the command reference are what you need and nothing has to be recorded. If it is contact, start at recording a dataset and pick a model from the policy pages once you know what you have.