Our SO-ARM101 Calibration Run and What 4.1 mm Establishes

This is a report from one session rather than a procedure to follow. We took part in Anthropic's research preview of the Model Hardware Standard, and ran Claude Code on a physical SO-ARM101. Nothing about that run was trained: no policy, no teleoperation, no demonstrations. The agent surveyed the workspace itself and wrote the motion — that part is covered in the companion piece on the agent-written motion, and what the standard itself changes is covered in the overview of MHS. This article is about the step before both of them, where the arm and the camera were brought into a common frame without a checkerboard and without camera intrinsics. The general version of that step — what hand-eye calibration is, what the textbook route asks for, and how to build the checkerboard-free variant on an SO-100 or SO-101 of your own — is the guide to hand-eye calibration without a checkerboard. What follows here is the measurement report that guide refers back to: one run, two numbers, and everything around them we did not measure.

One separation first, because these runs are already being conflated. Anthropic's announcement names Hugging Face as an early adopter, “adding MHS support in LeRobot, their robotics library”, and describes a University of Washington group that “used an open-source robotic arm built on LeRobot, instrumented with MHS, to safely coordinate sample loading across multiple instruments”. Those are other people's runs on other people's benches. Every number in this article is from ours, and none of it transfers in either direction.

What a classical hand-eye calibration asks for first

The standard route is two calibrations stacked on each other, and the first one is about the camera alone. You solve intrinsics and distortion from many views of an object whose geometry you already know. OpenCV calls that object a calibration rig and ships built-in support for a chessboard; its calib3d header describes the requirement as “an object with known geometry and easily detectable feature points” and returns “the overall RMS re-projection error” for the fit. The same header tells you how to shoot the views: “use multiple board poses with significant tilt, avoid collecting all views at a single working distance, span the expected working-distance range”. For the planar case the calibration engine cites Zhang's 2000 flexible technique for camera calibration.

The second calibration is the hand-eye transform proper, the one that answers where the camera sits relative to the gripper. OpenCV's calibrateHandEye wants two stacks of poses: the gripper-to-base transform for each pose, “recorded using for instance the robot kinematics”, and the target-to-camera transform for each pose, “recorded using for instance a pose estimation method (PnP) from 2D-3D point correspondences”. It solves what the header writes as the AX=XB equation and hands back the camera-to-gripper transform. On sample size it is explicit: “A minimum of 2 motions with non parallel rotation axes are necessary to determine the hand-eye transformation. So at least 3 different poses are required, but it is strongly recommended to use many more poses.” The five implemented solvers are named after their authors, the default being the one after Tsai and Lenz, 1989.

Read that as a list of things that must already be true before the first solver runs, and the shape of the problem changes. You need a printed pattern whose square size you have measured, because that measurement is where the millimetre physically enters the system. You need intrinsics, because PnP cannot run without them, and you needed the pattern to get the intrinsics. And you need forward kinematics you are willing to trust, because gripper-to-base is computed from link lengths and joint angles rather than observed. On a printed desk arm the third one is the assumption worth arguing with: both the link lengths and the joint angles carry the tolerances of printed parts and geared bus servos.

Rows above are read off OpenCV's calib3d header, not from memory.

Joint calibration is not camera calibration

There is a layer below all of this that gets called by the same word, and mixing the two up is the most common way to talk past each other about an SO-100 class arm. In LeRobot, calibrating an SO-101 is purely a joint-space procedure: the assembly guide says to “move the robot to the position where all joints are in the middle of their ranges” and then “move each joint through its full range of motion”. The stated purpose is interoperability between machines: “The calibration process is very important because it allows a neural network trained on one robot to work on another.” No camera appears anywhere in it.

Our own client does the same class of thing. The calibration wizard in the cockpit first releases the servos so the arm can be moved by hand, then captures two poses, a centre pose with the arm extended straight forward and the gripper fully closed and a second pose with the base rotated left, and saves both for the six joints shoulder_pan, shoulder_lift, elbow_flex, wrist_flex, wrist_roll and gripper. It carries its own safety notice for the obvious reason: with torque disabled the arm moves freely and may fall if nobody supports it. The result fixes where the arm believes its own joints are. It says nothing whatsoever about where the camera is. If you want the longer version, the calibration entry in the glossary covers what the zero point is for, and the desktop client documentation covers where the wizard sits.

POST /motors/torque/disable        release the servos, arm is hand-movable
POST /motors/calibration/capture   read all six joint positions once
POST /motors/calibration/save      store centre pose and second pose

The sixteen-point procedure from the run

The route in our run skips the pattern entirely, and with it both the intrinsics step and the PnP step. What is left is four moves.

1. 1
   Torque falls away

   The servos are released so the arm can be moved by hand. Same precondition as the joint calibration above, and the same hazard: an untorqued arm is held up by whoever is holding it.

2. 2
   The software draws sixteen points in the camera view

   The marks are placed in the image, not on the table. Nothing is printed, nothing is stuck down, and nothing is measured beforehand.

3. 3
   A person guides the closed gripper onto each mark

   Closed, because an open two-finger gripper has two jaw tips and therefore no single point you could put on a mark.

4. 4
   The robot reads its own position back

   Each mark now has a partner: a pixel in the camera image, and the arm's own reading of where its gripper was while it sat on that pixel.

This is where the phrase earns its keep. In the classical route the millimetre enters through the printed pattern — you tell the solver how wide a square is, and every metric quantity downstream descends from that one number you measured with a ruler. Here there is no pattern, so the only thing in the room that knows a distance is the arm. Its joint readings, run through its own geometry, are the ruler. The camera is never asked how long anything is. It is only ever asked where something is in its own image.

It is worth being clear about what a set of sixteen correspondences can and cannot carry, separately from what our implementation does with them. If the marks lie on one flat surface, the relationship between the image and that surface is a plane-to-plane perspective transform. OpenCV's homography estimator draws random subsets “of four pairs each”, which is the algebraic minimum, notes that the result “is determined up to a scale”, and with all pairs supplied fits by least squares so that the back-projection error is minimised. Sixteen is four times that minimum, which is the difference between a fit you can average over and an exact interpolation through the fewest points that will do. What our 4.1 mm figure is measured against — that residual, or a separate check with the arm driving itself — is not something a single number tells you, and we have not written it down.

What 4.1 mm and 3.0 mm are

The run reports 4.1 mm positional accuracy and 3.0 mm placement accuracy. Those are two different quantities and the run reports one number for each of them.

What is not in the report: how many attempts each figure summarises, whether it is a mean, a maximum or an RMS, what the spread around it was, and whether the whole procedure was torn down and repeated to see whether a second calibration lands where the first one did. Without those, this is one run and not a measurement series. Nobody should quote 4.1 mm as a specification for the method, and that includes us. It is a result from a session, and a result from a session is the weakest kind of number there is.

Whether roughly four millimetres is enough is a question about your part, not about the arm. If the feature you are grabbing is large compared with four millimetres — a block you can approach anywhere along one face, a bin you are dropping into — the budget is comfortable and the calibration is not what will limit you. If your task tolerance is of the same order as four millimetres, the calibration figure alone is not the thing to look at anyway; you would want the spread, and we do not have it. The concrete task behind these numbers was placing blocks: the best run so far placed twelve of them and formed all four colour groups.

Where this can go wrong

Backlash between teaching and driving

The points are taught with the servos released and a human hand pushing the joints into place. Later the arm drives to those same positions under its own torque. Backlash is the play between meshing teeth: for one input position the output can sit anywhere within that play, and which end of it the joint rests against depends on how it was last loaded. Hand-led and self-driven are not the same loading. The SO-101 follower runs six STS3215 servos, and the LeRobot guide gives the follower gearing as 1/345 throughout, with the leader mixing 1/191, 1/345 and 1/147. How much play that particular train has, and whether the hand-led to self-driven difference sits inside or outside 4.1 mm, our run does not measure. It is the first thing we would measure next.

Deflection under a load that changes

The structure is printed: TheRobotStudio's bill of materials ships with the instructions to 3D print the parts, and the SO-101 is described there as the next-generation version of the SO-100 with improved wiring and updated leader motors (the arm comparison lays out what differs between the low-cost arms). A printed link deflects under load, and the load differs across the two situations this calibration straddles. While teaching, a person is carrying part of the arm's weight. While executing, the servos hold it against gravity on their own. During a place, the gripper is additionally carrying something. If deflection differs between teaching and execution, it appears as an offset in exactly the axis you care about, and it will look like calibration error. We have not separated it from the other sources.

A camera that moves

This is the sharpest practical difference from the classical route, and it cuts both ways. Intrinsics describe the lens and the sensor: move the camera and they still hold, only the extrinsics change, and re-solving extrinsics is the cheap half. A direct image-to-arm map has no such split. It is intrinsics, extrinsics and kinematics fused into one fit, so nudging the camera voids all of it at once, silently and with no error message. The mitigation is genuinely cheap — run the sixteen points again — but nothing in the procedure notices that you should. On a bench where the camera is on a clamp that people knock, that is a real operational cost and not a footnote.

Outside the points

Sixteen correspondences constrain the region they span, on the surface they lie on. A mark set that covers the near half of the table says little about the far corner, and a mapping fitted on the table surface says nothing about a point 100 mm above it unless the geometry carrying it up there is itself sound. If the task uses the whole end-effector workspace, the marks should too. This is not a criticism of the method so much as a note that the sixteen points are a design decision, not a constant.

What sixteen points cost in practice

The procedure needs no printed pattern, no measurement of a pattern, no camera datasheet and no separate pose-capture script. It needs the arm untorqued, one person with a free hand, the camera already in its final position, and sixteen touches. There is no skill in it beyond putting a closed gripper on a mark. That is the honest advantage over a chessboard session, because the usual failure of classical calibration on a desk arm is not the mathematics — it is a hurried set of views taken at one working distance with too little tilt, which produces a plausible-looking result that is quietly wrong. Sixteen points fail more visibly: either the gripper was on the mark or it was not.

We did not time it, so we are not going to quote a duration. What we can state is the standing cost. Move the camera and you pay sixteen touches again. Change the surface the task happens on and you pay them again. Nothing about it accumulates into an asset you keep, which is the trade you are making in exchange for not owning a calibration target. If you are working through the same steps on your own arm, the getting-started walkthrough and the SO-100 hub cover the layers below this one, and the SO-101 page covers what differs on the newer arm.

What MHS supplies here, and what it does not

The Model Hardware Standard, which Anthropic previewed on 27 August 2026, is “a shared specification for AI agents to safely operate physical devices”. A driver exposes a small set of primitives, “commands like ‘read’ (for example, ‘get temperature’) or ‘write’ (for example, ‘set temperature’)”, and can be reached three ways: “MCP, the command line interface, and code files (APIs)”. It is plumbing. It is not a model, not a controller and not a perception stack. If you have already worked with the MCP tooling on this platform, the shape will be familiar.

That distinction matters for reading a demo like ours, because it would be easy to see an agent driving an arm and conclude that the standard did the geometry. It did not. MHS is what lets an agent read a joint position and write a joint target without a bespoke translator sitting between them. Deciding to draw sixteen marks, deciding where to draw them, and deciding what to do with sixteen readings is code and prompt on top of that. The standard removed an integration problem, not a robotics problem.

MHS is not open source yet. The announcement describes the preview as running “ahead of making the standard open source” and names no date, so anything you read about availability — including from us — is stated intent and not a schedule. Access during the preview runs through an application on the standard's own site.

What we have not done

1. A fully hands-off run from start to finish. The best run so far is twelve blocks placed with all four colour groups formed, and the sixteen taught points are one of the places where a person is still required.
2. A repeatability figure. The same calibration torn down and run twice, and the disagreement between the two.
3. A separation of error sources. Backlash, deflection and the fit itself are currently one number.
4. A camera-displacement test. Move the camera by a known amount, do not recalibrate, and measure what breaks and by how much.
5. A side-by-side against a classical chessboard hand-eye calibration on the same arm, the same camera and the same task. Until that exists we can say the direct route worked. We cannot say it is better.

None of that is a plan with dates attached, and we are not going to pretend otherwise. It is the list of measurements that would turn a session result into something you could rely on, written down so that the next person to read 4.1 mm knows exactly how much weight it will carry.