In our run inside Anthropic's Model Hardware Standard research preview, an agent measured the workspace of an SO-ARM101 itself and wrote the motion. No policy, no teleoperation, no demonstrations. Measured on that setup: 4.1 mm positioning accuracy and 3.0 mm placement accuracy; the best run so far placed 12 bricks and formed all four colour groups, and a fully hands-off run from start to finish has not happened yet. This article sets that against what a trained policy covers, and says plainly where each one stops.
- •In our own run inside Anthropic's Model Hardware Standard research preview, Claude Code drove a physical SO-ARM101. Nothing about it was trained: no policy, no teleoperation, no demonstrations. The agent measured the workspace itself and wrote the movement.
- •Calibration used the arm as its own ruler. Torque drops, a person sets the closed gripper down on 16 points the software draws into the camera view, and the robot reads back where those points are. No checkerboard, no camera intrinsics.
- •Measured on that setup: 4.1 mm positioning accuracy and 3.0 mm placement accuracy.
- •Best run so far: 12 bricks placed, all four colour groups formed. A fully hands-off run from start to finish has not happened yet.
- •Written motion is strong where the geometry is known and stays put. A learned policy is what covers variation, compliance, deformable material, and anything that has to react to what the camera sees at control rate.
- •A written motion is only exact if you read back what the arm did. On our own backend a target is clamped into the joint range and to 18 degrees per command, a second command inside 12 milliseconds is dropped, and the answer reports success either way.
- •One best run is not a reliability figure. Nothing here makes trained policies unnecessary, and we are not going to pretend otherwise on our own blog.
What actually ran
Anthropic previewed the Model Hardware Standard on 27 August 2026 and describes it on the announcement page as "a shared specification for AI agents to safely operate physical devices". We took part in that research preview. On our side, what ran was Claude Code driving a physical SO-ARM101 — the arm that the hardware repository calls "the next-generation version of the SO-100 robot arm", the same class of hardware the rest of our SO-100 material is about.
Nothing in that run was trained. No policy, no teleoperation, no demonstrations. The agent surveyed the workspace itself and wrote the motion. That is worth sitting with for a moment on this particular site, because what we sell is the opposite: recording episodes on an arm and training a policy on them. So this article is going to be about where each of the two approaches stops, and it is not going to end with the conclusion that the thing we sell just became obsolete — nor with the conclusion that the agent run was a parlour trick.
MHS itself is plumbing, not a model. Devices are exposed through a small set of primitives — the announcement gives "commands like 'read' (for example, 'get temperature') or 'write' (for example, 'set temperature')" — and an agent reaches them through one of three routes: "MCP, the command line interface, and code files (APIs)". Two limits are stated on the same page and are worth repeating before anyone gets carried away. It "doesn't yet work with hardware that lacks a programming interface", and: "As a large language model, Claude learns about the physical world through text and images, meaning its spatial and physical reasoning have limitations that still require expert oversight." The standard is also not public. Anthropic told The Register: "There's more to learn before we open source MHS. LLMs still lack physical intuition, having learned about the physical world from text and images." The preview site carries an application form and no specification. What the standard specifies in detail, and what it does and does not change for a small arm, is covered separately in our walkthrough of MHS; this article is about one run made with it.
The numbers in this article belong to our run and to no other. The Anthropic announcement itself quotes a PhD student at the University of Washington: "we used an open-source robotic arm built on LeRobot, instrumented with MHS, to safely coordinate sample loading across multiple instruments." Different lab, different task, different hardware, different results. Do not carry figures from one into the other. And to be explicit about our own standing: we took part in the research preview. We are not an Anthropic partner or sponsor, this article is not an Anthropic statement, and nothing below is endorsed by anyone but us.
The arm is its own ruler
The calibration is the part worth writing down, because it sidesteps the step that usually eats an afternoon. The conventional way from pixels to robot coordinates runs through a known target and a camera model. MathWorks' Camera Calibrator, to take one documented example, works from a "checkerboard, circle grid, AprilTag, ChArUco, and custom detector" pattern and estimates "the values of the intrinsic parameters, the extrinsic parameters, and the distortion coefficients". None of that was used here.
Instead: torque drops on the arm. A person takes the limp arm by its closed gripper and sets that gripper down on 16 points which the software has drawn into the camera image. At each point the robot reads back its own joint positions. The correspondence between image coordinates and arm coordinates therefore comes out of the arm's own encoders rather than out of a camera model. The arm is the measuring device. What that buys you is that the procedure needs no printed target, no lens parameters and no assumptions about the camera — and what it costs is a human hand for sixteen touches.
Two things follow from "torque drops", and neither is decorative. The arm is limp for the whole procedure. That is the same physical state this platform's own joint-calibration wizard — a different procedure, same precondition — refuses to enter quietly: it releases torque on all six servos as it opens, and the first line it puts on the screen is "Make sure you can safely support the robot arm. With torque disabled, the arm will move freely and may fall if not supported." Sixteen touch points is sixteen occasions to find out what an unsupported elbow does on the way down. The second thing is that a person decides, sixteen times, when the closed gripper is on the point. That judgement is inside the number that comes out at the end, not outside it.
Accuracy on that setup: 4.1 mm positioning, 3.0 mm placement. Both figures are ours, from that setup, on that table. Where those two numbers come from, what a single run can and cannot establish about them, and which error sources sit underneath them is the whole subject of the calibration report.
They are single figures from one arm, one camera and one table, reported by us. There is no repeatability protocol behind them here: no sample count, no spread, no statement about how they survive a power cycle, a knock against the table, or a camera that has been nudged. Read them as the order of magnitude — millimetres, not tenths of a millimetre and not centimetres — and not as a specification you can plan a fixture around.
| What gets called calibration | What it establishes | What it needs |
|---|---|---|
| Camera calibration (e.g. MathWorks Camera Calibrator) | Intrinsic parameters, extrinsic parameters and distortion coefficients of the camera | A known pattern — checkerboard, circle grid, AprilTag, ChArUco — and many views of it |
| LeRobot arm calibration | Joint ranges, so that leader and follower report the same values in the same physical pose. The LeRobot SO-101 guide notes it "allows a neural network trained on one robot to work on another" | Moving every joint through its full range once, per arm |
| The touch-point procedure in this run | A mapping between points drawn into the camera image and coordinates the arm can drive to | Torque off, a human hand, 16 points, and the arm's own encoders |
They are three different problems that share a word, which is why calibration is such a reliable source of confusion in threads about small arms. The LeRobot procedure and its rationale are documented in the SO-101 assembly guide; it is a prerequisite for the other two, not a substitute for either. Of the three rows above, only the middle one ships here: joint calibration is a wizard in the web cockpit, behind a button labelled Calibrate Arm. The touch-point mapping in the third row is a button nowhere on this platform. It is a recipe you build, and the buildable version of it — what to touch, in what order, and how to check the result instead of believing it — is in hand-eye calibration without a checkerboard.
Twelve bricks is a best run, not a success rate
The best run so far placed 12 bricks and formed all four colour groups. That is the number we have, and it is a ceiling, not an average. A fully hands-off run from start to finish — nobody touching anything between the start and the end — has not happened yet. It is the next thing on the list, which is a polite way of saying it is not done.
We are stating that because the characteristic failure of this whole genre of demonstration is quoting the best take. A best run establishes that something is possible. It says nothing about how often it works, nothing about the run after the one that got filmed, and nothing about what happens when a brick lands on its edge. Anyone who has had an arm on their desk for a week knows the distance between "it did it" and "it does it", and that distance is where all the actual work lives.
There is a yardstick for this on this site already, and it was not written for agent runs. The training guides here converge on the same protocol, and the GR00T N1.7 guide puts it under a heading that leaves little room: "Evaluate on hardware, because nothing else counts". What it asks for is twenty attempts, the object in a different position each time, "results written down as they happen. Not the six attempts that went well, and not a number you reconstruct from memory afterwards." The same guide adds the part that makes a number comparable rather than merely quotable: the protocol has to stay fixed between runs, because changing the attempt count or the object positions makes the second run look better for reasons that have nothing to do with what changed.
Held against that, our run reports a ceiling and nothing else. Setting the two side by side is not a confession of sloppiness. It is the difference between a demonstration and a measurement, and the demonstration comes first because it is the cheaper of the two.
| What we ask of a trained policy before quoting a number | What the twelve-brick run has |
|---|---|
| A fixed number of attempts, twenty, decided before the session starts | One best run. How many runs it was the best of is not something we have reported |
| The object in a different position each time, spread across the region the task is supposed to cover | Not established. How the arrangement varied between runs is not something we have reported |
| Every outcome written down as it happens, the failures included | A ceiling: twelve bricks placed, all four colour groups formed |
| The identical protocol repeated for the next version, so that two numbers mean the same thing | No second run under a stated protocol, so there is nothing to compare it against |
| A run nobody touches, because an intervention is a failure that got rescued | Not yet. A fully hands-off run from start to finish is the next thing on the list |
Running that protocol on the agent side is not hard, it is work: fix the layout, fix the attempt count, run it, and write down what happened including the attempts that ended early. Until somebody does that, twelve is a ceiling, and the only honest way to quote it keeps the word "best" attached to it.
Where a written motion holds, and where it does not
The useful comparison is not "agent versus policy". It is: which properties of a task does each approach actually cover. Sorted that way, most of the argument dissolves, because the two lists barely overlap.
| Property of the task | Written motion (agent) | Learned policy |
|---|---|---|
| Fixed, known geometry — bins, fixtures, a grid of slots | Strong. If a pose can be computed, it can be written down and driven to exactly, every time | Works, but you pay in demonstrations to teach something a coordinate could have stated outright |
| Sorting by a visible, discrete property (colour, presence, count) | Workable, given a perception step that returns a label and a position | Works, and extends to properties that are awkward to write a rule for |
| Object pose varies from run to run | Only as good as the perception feeding it. Each new case is another branch somebody has to write | This is what the training data is for |
| Contact-rich, compliant or deformable material | Hard. There is no coordinate for how far a cable tie has slid or how a cloth has folded | The class of task these models are aimed at; pi-zero lists laundry folding and table cleaning among its evaluated tasks |
| Closed-loop reaction to the camera image, at control rate | Not what a written trajectory does | The entire premise. ACT reports six real-world tasks learned from roughly ten minutes of demonstrations |
| Recovering from a state nobody anticipated | The branch does not exist until somebody writes it | Not solved here either — off-distribution states are precisely where cloned policies drift |
The two rows with claims in them carry sources. ACT, the policy behind the ALOHA work, reports that "only 10 minutes worth of demonstrations" sufficed to learn six difficult real-world tasks — among them opening a translucent condiment cup and slotting a battery — "with 80-90% success". pi-zero puts "a novel flow matching architecture" on top of a pre-trained vision-language model and names "laundry folding, table cleaning, and assembling boxes" among the tasks it covers. SmolVLA is the small end of the same family, "designed to be trained on a single GPU and deployed on consumer-grade GPUs or even CPUs", and GR00T N1 goes the other way, trained on "a heterogeneous mixture of real-robot trajectories, human videos, and synthetically generated datasets". All four are policies you train on data, and all four are trainable from the cloud training side of this platform.
Notice what none of those four papers claims: that the model will hit a fixed slot to the millimetre because you asked nicely. Precision at a known coordinate is not what imitation learning is good at, and it is exactly what a written motion does for free. The reverse holds just as firmly.
On our own arm backend, a commanded angle is a request
The MHS run did not go through this platform's backend, so none of what follows describes it. It describes what a written motion meets on the arm we ship for, and it belongs in this article because the strongest entry on the written side of the ledger — that a coordinate can be read, argued about and corrected — only holds if you also read back what the arm did with it. Every command that reaches a servo through our backend passes three guards first, and all three are silent.
The target is clamped into that joint's range: shoulder pan and wrist roll to plus/minus 150 degrees, shoulder lift to plus/minus 90, elbow flex to plus/minus 120, wrist flex to plus/minus 100, the gripper from minus 160 to 100. A single command may then move a joint at most 18 degrees, and the gripper at most 12, which makes a 90-degree sweep five commands rather than one. And two commands to the same joint closer together than 12 milliseconds: the second is dropped, not queued, so a tight loop loses commands instead of moving faster. None of the three raises an error. What they produce instead is a field.
{
"success": true,
"motor": "shoulder_lift",
"position": 120.0,
"robot_id": null,
"applied": true,
"applied_position": 90.0
}So the rule for a written motion on this stack is short: the number you sent is a request, the number that came back is the fact. Reading it costs one line and it is the whole difference between a script that reports it did something and a script that did it. An agent that surveys the workspace and then drives it has to close that loop for the same reason a human does, and nothing about the survey closes it for free.
Two further guards sit outside those three. A thermal watchdog polls servo temperatures every two seconds, parks the arm and blocks motion once a servo passes 65 degrees, releasing only when it falls back to 55; while it is blocked, a motor command comes back as HTTP 423 with the message "Roboter kühlt ab - Steuerung kurz gesperrt" — the robot is cooling down, control briefly locked. The self-collision check, by contrast, is a function that currently returns safe for every pose by explicit decision: joint ranges are enforced, the arm meeting itself or the table is not. Clear the workspace before an agent drives, exactly as you would before your own script does. The full ledger of what a policy costs against what a written motion costs, in episodes, hours and dollars, is in scripted motion or a trained policy.
- Nothing to record and nothing to train. The path from idea to moving arm is short enough to iterate inside an afternoon
- Every decision is inspectable. A coordinate can be read, argued about and corrected — as long as you also read back the angle that actually reached the servo
- Behaviour does not shift underneath you. The same file does the same thing tomorrow, with no checkpoint to compare against
- The agent that surveyed the workspace also wrote the motion, so the survey and the movement cannot silently disagree about where the table is
- Coverage equals what somebody thought of. A case that was not written down is a case that is not handled
- Compliance and deformation have no coordinate representation, so there is nothing to write
- Perception remains a separate problem. Nothing about an agent writing code solves finding the object
- One best run is not a reliability figure, and we do not have a hands-off run yet
- Anthropic's own text puts a ceiling on it: Claude's "spatial and physical reasoning have limitations that still require expert oversight"
- A commanded angle is not automatically a movement. On our own backend it is clamped into the joint range and to 18 degrees per command, and the answer says success either way
Why we are not putting the policies away
The tempting story is that one of these replaces the other, and the tempting story is wrong in both directions. A written motion is a way of encoding knowledge you already have in an exact form. A policy is a way of acquiring knowledge you cannot state — how hard to squeeze, when the grasp has failed, what a half-occluded object looks like from the wrist camera. Those are different jobs. An arm that does a fixed pick-and-place into a fixture and an arm that has to deal with whatever is on the table are not the same problem, and it has never been a scandal that they need different tools.
The interesting arrangement is the one where they sit next to each other. Approach, retreat, tool changes, moving between stations, homing after a fault: geometry, all of it, and all of it cheaper to write down than to demonstrate. The part in the middle where the arm touches something uncertain is the part worth spending demonstrations and GPU hours on. Splitting a task along that seam is a decision you can make deliberately, and it is a better use of a dataset than recording an operator driving a straight line between two known points fifty times.
If you have never watched the learned half of this, the fastest way is to drive a real SO-100 yourself at the live arm, then compare it against what a trained policy does with the same hardware in running your first policy. The recording side is in data collection and the training side in the training documentation. If you would rather put an agent on your own arm than a policy, the route that ships today is MCP rather than MHS, and it is documented in letting an agent drive a physically connected SO-100.
The part an agent cannot write down
Recording episodes on an SO-100 or SO-101, curating them, and training ACT, SmolVLA, pi-zero or GR00T on rented GPUs is what this platform does. It is the tool for the half of the task where there is no coordinate to write — contact, compliance, and reacting to what the camera actually shows.
See how data collection worksWhat an agent that surveys a workspace might do for data collection
None of what follows exists in the platform. There is no date attached to any of it, no beta, and no roadmap commitment. It is written down because it is the honest reason we find the run interesting beyond the demonstration value, and because a reader thinking about their own setup may get there faster than we do.
The capability that stands out in the run is not that the arm moved. It is that something measured the workspace and then acted on its own measurement. Recording sessions fail for reasons that live in exactly that layer, and they fail quietly: a camera nudged between episodes, a fixture that has walked two centimetres over an afternoon, a wrist view that no longer contains the gripper. None of that shows up in a loss curve. It shows up weeks later as a policy that behaves oddly for reasons nobody can reconstruct.
That failure already has its own page here, written for policies rather than for datasets: it worked yesterday and fails today. The reasoning on it is the part worth borrowing. A checkpoint is a fixed function of its observations, so if the output changed, the input did, which leaves four candidates: a camera that moved, changed lighting, a changed table, mount or object, and a recalibrated arm. Only the last is repairable without new data, and only if the original neutral pose can be reproduced. A recording session has the same four candidates and none of the diagnosis, because a dataset does not fail visibly. It simply teaches something slightly wrong, and pays it out months later.
One of the four deserves naming, because it is the one a survey pass would catch first and the one people rule out last. Camera ids on this platform are pinned to the hardware id, so the id you saw yesterday belongs to the same physical camera today. The operating-system index underneath is what moves: devices are enumerated in the order the system hands them over, so a replug, a webcam plugged in for a call, or a boot with a different USB device attached can shift which camera sits on which index. A setup that pinned the wrist view to an index rather than to an id now feeds the wrist stream where the scene stream is expected, and both previews look entirely plausible.
ay-robots --json devices
# cameras[].id stays with the physical camera
# cameras[].device_path /dev/video<index>, and that index can move
# after a replug or a rebootWhat exists today in that layer on the arm side is one automatic check, and its narrowness is the point. The backend watches the gripper reading: if it sits outside the gripper's calibrated range for forty consecutive polls, roughly two seconds at 20 Hz, it files a warning against that arm — it turns up in the backend's status answer under safety.calibration_warnings — naming the measured value, the limits it violated, and the instruction to recalibrate the gripper. That catches exactly one fault, a calibration that belongs to a different arm or to a different day, and it catches it in two seconds. Nothing else in the loop watches geometry. No check notices that the tripod moved, that the table is two centimetres further away than it was, or that the wrist view no longer contains the gripper. Those are the ones that end up in the dataset.
- A survey pass before a session that records where the camera and the work area actually are, so a later shift is detectable instead of merely suspected.
- A re-check between episodes that compares the current geometry against the survey and stops the session when it has moved, rather than filling the dataset with episodes recorded under a changed setup.
- Scripted scene resets between takes on tasks where the reset is pure geometry, which is the tedious half of every recording afternoon and the half where operator attention drifts first.
- Deterministic evaluation poses: driving to the same set of start positions before every evaluation run, so two checkpoints are compared under the same conditions rather than under whatever the last person left on the table.
- A written baseline for the geometric part of a task, purely to establish what the arm can reach and hold before anyone spends an afternoon recording it.
Every item on that list is about the setup around the data, not about replacing the data. The list is also easy to write and hard to build, and we have not built any of it. Treat it as a set of open questions we would rather state out loud than quietly imply we have solved. The human-in-the-loop side of the same problem — correcting a policy while it runs, and feeding those corrections back — is the one part of this territory we do have running, and it is described under the DAgger loop.
What is still open
- A hands-off run from start to finish. Until that exists, everything above is about a supervised process, not an autonomous one.
- Repeatability figures for the calibration. Two single numbers do not describe a distribution, and we have not measured one.
- How the touch-point procedure behaves after a knock, a power cycle, or a camera that was moved by 5 mm. Unknown, and it is the practical question.
- Whether the approach transfers to tasks that are not bricks on a table. Discrete rigid objects with clear colour classes is a friendly case, chosen because it is friendly.
- What MHS looks like once it is public. The specification is not open source; the announcement states the intent to open-source it, without a date, and the preview site publishes no specification.
- Where the sensible seam between written motion and learned policy sits for a given task. We have opinions above, not measurements.
- Whether the guards on our own backend would have to change for a written motion to be driven through it rather than through MHS. The step clamp and the rate limit were tuned against a human at a control, not against a program stepping a trajectory, and nobody has run that comparison.
Does this mean you can skip collecting data and training a policy?▾
For a task whose geometry is known and stays put, and where the perception problem is easy or already solved, a written motion may well be the shorter route — that was true before this run and remains true. It does not extend to contact-rich work, deformable material, or anything that has to react to the camera image at control rate. Those are the cases the policies exist for, and nothing in our run touched them.
Is 4.1 mm good?▾
It depends entirely on the tolerance of your task, and it is one figure from one setup rather than a specification. For dropping a brick into a marked area it is plenty. For inserting a connector it is not. We have no repeatability data behind the number, which is the first thing you would want before designing anything around it.
Why not use a checkerboard and camera intrinsics like everyone else?▾
The touch-point procedure gets the mapping the task needs — image points to arm coordinates — without a printed target or a camera model, because the arm's encoders supply the ground truth. That is a genuine simplification for this kind of setup. It is not a general replacement for camera calibration: if you need undistorted images or metric depth from that camera for something else, you still need the camera model.
Is ay-robots a partner of Anthropic?▾
No. We took part in the Model Hardware Standard research preview. Access during the preview is by application, as stated on the preview site. That is the whole of the relationship, and this article is not an Anthropic statement.
Was this the same demonstration that circulated with a LeRobot arm?▾
No. The Anthropic announcement describes a run by a PhD student at the University of Washington using an open-source arm built on LeRobot to coordinate sample loading across instruments. That is a different lab, a different task and different hardware. Our figures belong only to our run.
Which policy would you train for a task like this one?▾
That depends on what makes the task hard, not on the arm. The trade-offs between the architectures are laid out in our comparison of the arms and models rather than here, and picking one of them is a separate decision from deciding whether the task needs a learned policy at all.
Can I run the sixteen-point procedure on my own arm today?▾
Not as a feature. On this platform joint calibration ships as a wizard in the web cockpit, behind a button labelled Calibrate Arm, and the camera-to-arm mapping ships nowhere. Building the touch-point routine yourself is a recipe rather than a button: the sequence, the points, and the checks that tell you whether the result is any good are laid out in our article on hand-eye calibration without a checkerboard. Nothing about it needs MHS.
What would it take to turn twelve bricks into a success rate?▾
The same protocol we ask of every policy trained on this platform: a fixed number of attempts decided in advance, twenty being the usual figure, the arrangement varied across the region the task is meant to cover, every outcome written down as it happens rather than remembered afterwards, and the identical protocol repeated for the next version so that two numbers mean the same thing. It is an afternoon of work and it is not glamorous. Until it exists, the honest quote keeps the word best in front of it.
One run does not settle an argument, and we would rather publish the ceiling with its caveats attached than a clean number with the caveats removed. What the run did establish, for us, is that the boundary between "write it down" and "learn it" is further towards the written side than we had assumed for tasks with stable geometry — and exactly where we assumed it was for everything else. If you want to see where your own task falls, the hardware comparison is in the arm comparison, and the recording client is at the download page.
Sources
- Anthropic (27 Aug 2026): Previewing the Model Hardware Standard
- Model Hardware Standard — research preview site
- The Register (28 Aug 2026): Anthropic proposes plumbing spec to link AI agents to lab kit and robots
- Zhao, Kumar, Levine, Finn (2023): Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware (ACT / ALOHA)
- Physical Intelligence (2024): pi-zero — A Vision-Language-Action Flow Model for General Robot Control
- Shukor et al. (2025): SmolVLA — A Vision-Language-Action Model for Affordable and Efficient Robotics
- NVIDIA (2025): GR00T N1 — An Open Foundation Model for Generalist Humanoid Robots
- Hugging Face LeRobot — SO-101 assembly and calibration guide
- Hugging Face LeRobot — models, datasets and tools for real-world robotics in PyTorch
- TheRobotStudio: SO-ARM100 / SO-ARM101 hardware repository
- MathWorks: Using the Single Camera Calibrator App
Ready for high-quality robotics data?
AY-Robots connects your robots to skilled operators worldwide.
Get Started