The stack around the model

The policy is the smallest part of a working robot.Everything around it is the harness.

Recording, one dataset format, training on rented GPUs, a warm pod, an evaluation record, an agent interface: six layers between arm and policy.

See the six layers What you bring

5: Models you fine-tune through one form, ACT to GR00T
2: Arms supported end to end: SO-100 and SO-101. Related arms differ in driver only.
2: Incompatible LeRobot dataset formats the trainers demand: v2.0/v2.1 for GR00T, v3.0 for the rest.
6: Layers on this page. Miss one and the work stops there.

The core claim

Choosing a model is a decision. The rest is a project.

Fine-tuning is the step with a progress bar. It takes the least time.

What decides whether the arm works sits around training: how demonstrations were recorded, whether two sets merge, where the card comes from, how runs are compared.

In test engineering a harness is the rig that holds the thing under test. Robot learning needs one too. If only training interests you, the cloud training page covers it on its own.

None of this replaces the model. GR00T, Pi0.5, SmolVLA and ACT keep their strengths and limits; the harness only feeds and hosts them.

Layer by layer

Six layers, and what breaks when one is missing

Each layer exists because something went wrong without it. The bottom line is the failure.

Recording

The local cockpit drives the arm, shows the cameras and writes episodes while you teleoperate.

Without one recorder for arm and cameras, no two datasets match.

How data collection works

One dataset format

Episodes are LeRobot datasets, the layout the ecosystem uses, so recordings from two labs match.

Without one format two datasets cannot merge, and the trainers disagree: GR00T reads v2.0 or v2.1, the others v3.0.

Read the documentation

Training on rented GPUs

One form for 5 models, GR00T N1.7 and N1.5, Pi0.5, SmolVLA and ACT: pick dataset and model, the job runs on a rented pod.

Without rented compute your graphics card picks the model, and 80 GB cards are rare.

Cloud training

Serving the trained policy

A trained policy runs as a warm GPU pod: the arm sends an observation, gets an action.

Without a warm pod every start pays the cold-start wait, and slow starts go unevaluated.

Cloud inference

An evaluation record

Every policy keeps an evaluation folder: which runs were attempted, on which task, with what outcome.

Without a shared record you compare feelings instead of runs.

Browse the model pages

An agent interface

A local MCP server beside the cockpit exposes the arm as 9 tools: read state, record episodes, park the arm.

Without a machine-readable surface every repetition needs a hand on the keys.

Let Claude control your robot

Two pieces hang off the rig

A dataset marketplace

Recorded episodes can be listed and bought instead of thrown away.

Open the marketplace

Teleoperation from the browser

An arm can be driven from a browser, which is how remote collectors record.

How teleoperation works

Not sure this fits your arm? The hosted demo needs no account and no hardware.

Try it first

The honest comparison

Build it yourself, or take the harness

Both are legitimate. The right column is the same work, done once.

Layer	Build it yourself	With this harness
Recording	Your own script around the recorder, cameras wired, one stable episode definition.	The cockpit records episodes locally, desktop or browser.
Dataset format	Your own layout, plus a converter once trainer versions drift.	LeRobot throughout, with the version each trainer needs on its page.
Compute	Buy a card, or rent one and keep driver and torch alive.	A rented pod per job, started from a form.
Serving	Wrap the checkpoint in a server, keep it warm, expose it.	A warm pod with an endpoint the arm speaks.
Evaluation	A spreadsheet, if anyone keeps it current.	An evaluation folder per policy, beside its checkpoints.
Agent access	Usually nothing, because it is the layer people reach last.	A local MCP server with 9 tools, gated on motion.

The left column is right when the harness is what you want to build, wrong when the task is.

Before you start

What you bring, what you skip, where this stops

Prerequisites and limits, both short.

What you bring

An arm. SO-100 and SO-101 are supported end to end; related arms differ in driver, not pipeline.
Cameras, at least one, mounted the same way every episode.
A task worth repeating, and patience to record it often. Nothing later repairs a thin dataset.
A computer for the cockpit, with a USB port and a screen.

What you do not need

A GPU. Training and inference run on rented cards.
A Python environment, CUDA or a matching torch build.
A serving stack of your own; the warm pod is the endpoint.
A second machine. The MCP server runs beside the cockpit.

Where it stops

Evaluating on real hardware needs real hardware, the arm in front of the task.
A language model does not produce joint angles at control rate; motion comes from the policy.
Trainers disagree on the dataset version, so a set may need converting.

The agent layer

What an agent does here, and what it does not

This is the layer that gets oversold first, so it carries its limits.

The local MCP server is called ay-robots-local, speaks over stdio and runs beside the cockpit, offering 9 tools: 5 read only (status, devices, joints, recording state, datasets), 2 move the arm, 2 record.

Motion is gated by the server, not by good manners

The two motion tools refuse to run without an explicit confirmation flag, and return this:

move_joint moves a physical robot arm. Ask the user to confirm, then call again with confirm: true.

An exploring agent cannot move hardware by accident. Only a second call with the flag passes.

No language model streams joint angles here. Motion comes from the trained policy; the agent reads state, records episodes and parks the arm.

Letting Claude control your robot walks through that layer, and what an LLM can and cannot do with an arm is the wider version of the argument.

A second, hosted MCP server answers questions about models, arms and guides, and with an API key it also reads your account and starts a cloud run. What it never has is a path to hardware. The MCP page covers that one. To stay in a terminal, the command line tool drives the same local backend by hand.

Let Claude control your robot Get the local stack

Ready to record? The cockpit and the local MCP server come from the same download page.

Get the local stack

The model is the easy part. Take the rest off your desk.

Install the local stack, record a first episode, and see how far the harness carries it.

Download the local stack Try it without an account