MCP for Robots: What an AI Agent Can Do Through Tools

An MCP server is a list of functions plus a promise about what each one does. When you point an agent at a robot through one, you are not giving it a body. You are giving it a menu, and the menu is the entire safety story. Everything the agent can do to your arm is something you wrote down as a tool, gave a JSON Schema to, and decided whether to gate. Everything you left out simply does not exist for it. That is a better position than it sounds, and it is also the reason this goes wrong so often: the menu gets written the way an internal API gets written, and then a model that has read a poisoned web page starts ordering from it.

This page is about where the line goes. What an agent can usefully do through a tool interface for an SO-100 class arm, what the Model Context Protocol actually specifies about consent and confirmation, which operations belong behind an explicit yes, and which ones should never be tools at all. The protocol numbers here are from the 2026-07-28 revision, checked on 24 August 2026. The robot numbers are from the five policies this platform trains, listed on the policies page.

What an MCP server actually gives an agent

MCP is a JSON-RPC 2.0 protocol between three roles: a host (the LLM application), clients inside it, and servers that provide context and capabilities. A server can offer three things, and the difference between them is who is in charge.

The spec's own phrasing is that tools are model-controlled: the language model discovers and invokes them automatically based on its contextual understanding and the user's prompts. Resources and prompts need a human or an application to reach for them. Tools do not. So when people ask what an MCP server gives an AI agent controlling robots, the honest answer is: whatever you put in the tool list, invoked at the model's discretion, with arguments the model made up from your JSON Schema.

The chain between a sentence and a servo

Before deciding what belongs behind a confirmation, draw the hops. A user types a sentence. Somewhere at the other end an STS3215 servo turns. Between those two things there are more layers than people expect, and the confirmation has to sit at the layer that knows what the action costs.

1. The user writes a request in the host application.
2. The model reads the tool list from tools/list and picks one.
3. The client (optionally) shows the call to the user and asks.
4. The client sends tools/call over stdio or Streamable HTTP.
5. The server validates arguments against its own schema, checks authorization, and decides whether it needs more consent.
6. The server calls the real API: a training endpoint, a robot backend, a serial bus.
7. A joint moves, or a GPU gets rented, or nothing happens and an error comes back for the model to read.

Step 3 is the one everyone assumes is guaranteed. It is not. The tools specification says that for trust and safety there SHOULD always be a human in the loop with the ability to deny tool invocations, and that clients SHOULD prompt for user confirmation on sensitive operations and show tool inputs to the user before calling the server. Every one of those is a SHOULD. A client running in a CI job, a client the user has put into an auto-approve mode, or a client someone wrote in an afternoon will call your tool without asking anybody. Design as if step 3 does not happen, because for some fraction of your callers it does not.

Sort your tools before you write them

The useful first pass is not technical. Take every operation you were about to expose and put it in one of four buckets: it reads, it writes something reversible, it spends money or physical state, or it moves a joint. The bucket decides the annotations, the confirmation and, more often than people expect, whether the tool should exist.

The four annotation fields are defined in the protocol schema and their defaults are worth memorising, because omitting them is not neutral. readOnlyHint defaults to false. destructiveHint defaults to true and is only meaningful when readOnlyHint is false. idempotentHint defaults to false. openWorldHint defaults to true. In other words, an unannotated tool is assumed to be a destructive, non-idempotent, open-world write. That is the right default, and it means a careful client will nag about every tool you did not think about.

{
  "name": "stop_training",
  "title": "Stop a training run",
  "description": "Stop a run that is still queued or running. The rented GPU pod is destroyed and the run is recorded as cancelled. This cannot be undone and it does not refund GPU time already used; checkpoints written before the stop remain in object storage.",
  "annotations": {
    "readOnlyHint": false,
    "destructiveHint": true,
    "idempotentHint": true,
    "openWorldHint": true
  },
  "inputSchema": {
    "type": "object",
    "properties": {
      "job_id": {
        "type": "string",
        "description": "Job id of a queued or running run"
      }
    },
    "required": [
      "job_id"
    ]
  }
}

What belongs behind a confirmation, and how to actually enforce one

There are exactly two places a confirmation can live, and only one of them is yours. The client-side prompt is a courtesy the host application may or may not offer. The server-side gate is code you wrote, and it runs whether or not anybody is watching. Anything that costs money, destroys state or moves hardware needs the second kind.

The cheapest server-side gate is a required boolean in the input schema whose description says what agreeing means. It costs one extra round trip, it works on every client back to 2025-06-18, and it turns a vague instruction into an explicit one. Called without the flag, the tool starts nothing: it returns the cost estimate and an error telling the model to show the user that number and come back with an explicit yes. Make the refusal carry the estimate rather than a bare "confirmation required", because an agent handed a bare refusal will guess, and it will guess yes. That matters because a training run on the 80 GB tier is 4 to 12 USD and on the 24 GB tier 1 to 3 USD, and neither is a number an agent should commit on its own reading of "train something on this."

// The number your gate should refuse with. This is a real response from the
// public estimate tool on the hosted server, called with no API key at all:
//   tools/call estimate_training_cost {"policy": "groot1.7"}
{
  "policy":  { "slug": "groot-n1-7", "name": "GR00T N1.7", "trainer": "groot1.7" },
  "gpu":     "A100 80 GB or H100 80 GB",
  "steps":   { "used": 20000, "model_default": 20000 },
  "estimate": {
    "hourly_usd": "1.20 to 2.00 USD",
    "duration":   "3 to 6 hours",
    "total":      "about 4 to 12 USD"
  },
  "episodes": { "minimum_for_this_model": 50 },
  "disclaimer": "These are spot market ranges, not a quote. The pool rents by the hour and selects a card by VRAM rather than by name, so the hourly price moves with availability."
}

The richer gate is elicitation, a client feature the server can invoke in the middle of handling a call. The server returns an input-required result carrying an elicitation/create request; the client renders it, the human answers, and the client retries the original call with the answer attached. The response is one of three actions: accept, decline or cancel. The distinction is worth respecting. Decline means no. Cancel means the dialog was dismissed and you may reasonably ask again later. Form-mode schemas are deliberately limited to flat objects with primitive properties, and servers must not use form mode to request passwords, API keys, tokens or payment credentials; that is what URL mode is for.

Least privilege beats a bigger confirmation dialog

A confirmation prompt asks a human to be the access control system, several times an hour, while they are doing something else. That works for a while and then it stops working, because people click through the prompt they have already seen fifty times. The durable fix is not a louder dialog. It is that the credential the agent is holding cannot reach the dangerous thing at all.

The security best practices document has a section on exactly this under scope minimisation, and its list of common mistakes reads like a description of most first drafts: publishing every possible scope in scopes_supported, using wildcard or omnibus scopes, and bundling unrelated privileges to preempt future prompts. The recommended shape is a minimal initial scope covering low-risk discovery and reads, with incremental elevation the first time a privileged operation is attempted. Applied to a robot, that means the agent you leave running overnight holds a credential that reads telemetry and nothing else, and the credential that can move a joint is one a person hands over deliberately, for a session, in a room they are standing in.

On this platform the first row is literal. Connect to the hosted server with no key at all and the public tools still answer, which is enough for an agent to compare the five trainable policies, read every guide, run a symptom through diagnose_problem and estimate what a run costs from the pricing page. Nothing account-scoped appears until a bearer token does, and no key of any kind gets the hosted server closer to a servo, because it has no path to one. Compare that with the failure-mode index, which an agent can read freely and which is where most of its usefulness actually comes from.

Build a minimal robot MCP server yourself

The Python SDK is the shortest path, and version 2 changed the entry point, so older tutorials will not run. The class is MCPServer; type hints are the input schema; the docstring is the description. Start read-only, prove the agent can see your robot, and only then add anything that moves.

1. 1
   Install the SDK

   The cli extra adds the mcp command with mcp dev, mcp run and mcp install. Note the upper bound if you are still on the 1.x API: pip install mcp now installs 2.x.

   bash

   uv add "mcp[cli]"      # or: pip install "mcp[cli]"

2. 2
   Expose telemetry first, annotated as read-only

   One function, one docstring, two annotations. open_world_hint=False says this tool talks to one known arm, not the internet. A good client will stop asking about it, which is the point: you want the agent reading state constantly and writing state rarely.

   python

   from mcp.server import MCPServer
   from mcp.types import ToolAnnotations
   
   mcp = MCPServer("so100-bench")
   
   
   @mcp.tool(
       title="Read joint state",
       annotations=ToolAnnotations(read_only_hint=True, open_world_hint=False),
   )
   def joint_state() -> dict:
       """Current angle in degrees for each of the six joints, plus the gripper span."""
       return read_joint_state_from_bus()

3. 3
   Gate the one tool that moves, with elicitation

   A motion tool that returns before asking is a motion tool with no gate. Ask, check the action, and treat anything other than accept as a no. The message should say what will physically happen, not what function is being called.

   python

   from pydantic import BaseModel, Field
   from mcp.server.mcpserver import Context
   
   
   class Confirm(BaseModel):
       proceed: bool = Field(description="Move the arm now?")
   
   
   @mcp.tool(
       title="Move to home pose",
       annotations=ToolAnnotations(read_only_hint=False, destructive_hint=True),
   )
   async def go_home(ctx: Context) -> str:
       """Move the arm to its home pose. The arm moves immediately."""
       result = await ctx.elicit(
           message="The arm will move to its home pose now. Clear the workspace first.",
           schema=Confirm,
       )
       if result.action != "accept" or not result.data.proceed:
           return "Not moved."
       return go_home_on_hardware()

4. 4
   Open it in the Inspector before any client

   The Inspector shows you the tool list exactly as a model would receive it, including every description string. Read them. This is also where you notice that your docstring says something you would not want an agent acting on.

   bash

   uv run mcp dev server.py

5. 5
   Connect a client over stdio, not over the network

   A stdio server is reachable only by the process that spawned it, which is the cheapest isolation you will ever get. The security best practices document recommends exactly this for local servers, and recommends an authorization token or a unix domain socket if you must use HTTP.

   bash

   claude mcp add so100-bench -- uv run mcp run /path/to/server.py

If your robot already speaks ROS, do not rewrite that layer. The open-source ros-mcp-server takes the bridge route: an Apache-2.0 server that works across ROS 1 and ROS 2 distributions through a rosbridge node you add to your existing setup, and exposes publishing and subscribing to topics, calling services and actions, setting parameters and discovering what exists, all without changing the robot's own source code. The design decision it makes for you is that the tool surface is generic, which is convenient and also means the agent can publish to any topic it can enumerate. That is a fine debugging tool and a poor production boundary. Narrow it to the topics your task actually needs before it goes anywhere near a run that matters, and read the API reference if you would rather drive the platform side from your own code than from a tool call.

Two ways to give an agent the same robot workflow

# what you are actually maintaining
uv add "mcp[cli]"
uv run mcp dev server.py          # read the tool list as the model sees it
claude mcp add arm -- uv run mcp run server.py

# remote: read the catalog, no key needed, no hardware reachable
claude mcp add --transport http ay-robots https://www.ay-robots.com/api/mcp

# same endpoint, raw, to prove it is reachable
curl -sS https://www.ay-robots.com/api/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

# local: stdio, and the only server that can move a joint
ay-robots mcp

An agent is not a control loop

The most common design error in robot MCP servers is a tool called something like move_to that the author expects to be called in a loop. It will be, and it will be terrible. A tool call is a JSON-RPC round trip mediated by a language model deciding what to send. The model's own turn latency is measured in hundreds of milliseconds to seconds. Compare that to what the control loop actually needs.

Read that table as a budget. ACT leaves you 20 ms per step, which a single public-internet round trip eats whole. Even GR00T N1.7 at 152 ms leaves nothing for an agent turn on top. This is the honest limit and it is worth stating plainly: remote inference works for slow pick-and-place and does not work for fast reactive motion, and adding an agent in the loop makes it strictly worse. See inference latency for the arithmetic.

So the correct shape is: the agent starts and stops closed-loop behaviours; it does not participate in them. One tool call says "run this policy on this task until it finishes or I stop you", and a real control loop written in normal code does the 20 ms work. The agent watches the result, reads telemetry, decides what to try next. That is where a language model is genuinely good and where the round trip does not matter.

Long-running work does not fit inside a tool call

A fine-tuning run takes hours. A tool call that blocks for that long dies on a transport timeout, and one that returns immediately leaves the agent with nothing to poll. The protocol's answer is Tasks, an official opt-in extension identified as io.modelcontextprotocol/tasks: the server returns a durable task handle instead of a result, the client polls tasks/get, and the task carries a status through working, input_required, completed, failed or cancelled. The last three are terminal: once reached, the task's state does not change.

Two details matter for robots. First, input_required is how a long job asks a human something halfway through, which is the natural home for a mid-run confirmation. Second, tasks/cancel is cooperative: the server acknowledges the intent and is not obligated to stop the work. If your cancel has to actually stop a GPU or an arm, do not rely on the protocol's cancel semantics; expose an explicit stop tool that has a defined effect, and say in its description that it cannot be undone.

What goes wrong in practice

Three failure classes account for nearly everything. The first is the model simply calling tools badly. The ROSBag MCP Server paper (arXiv 2511.03497, November 2025) benchmarked eight state-of-the-art LLMs and VLMs, proprietary and open-source, on robotics tool calling. It reports a large divide in tool-calling capability between models, with Kimi K2 and Claude Sonnet 4 clearly ahead of the rest, and finds success rates varying with the tool description schema, the number of arguments and the number of tools available. That is a design lever, not just a model property: fewer tools, fewer arguments and better descriptions measurably improve the outcome.

The second is context cost. Anthropic's engineering write-up from 4 November 2025 notes that agents connected to thousands of tools process hundreds of thousands of tokens before reading a single request, and shows one workflow going from 150,000 tokens to 2,000 by moving the work into code instead of routing every intermediate result through the model. A robot server that exposes one tool per ROS topic hits this immediately.

The third class is adversarial, and robotics is where it stops being an abstraction. RoboPAIR (arXiv 2410.13691, October 2024) attacked three LLM-controlled systems: NVIDIA's Dolphins self-driving model in a white-box setting with full access, a Clearpath Robotics Jackal ground robot with a GPT-4o planner in a gray-box setting with partial access, and a GPT-3.5-integrated Unitree Go2 robot dog in a black-box setting with only query access. It reports that RoboPAIR and several static baselines find jailbreaks quickly and effectively, often at 100% attack success, and describes the Go2 result as the first successful jailbreak of a deployed commercial robotic system. Combine that with the lethal trifecta framing from Simon Willison (June 2025), where private data plus untrusted content plus an external communication channel is enough for exfiltration, and the robot version is obvious: private data plus untrusted content plus actuators.

Try the shape before you wire it to hardware

The cheapest way to develop an intuition for where the boundary goes is to drive the loop by hand once. Record a LeRobot dataset with the desktop client, train something small, and watch which steps you would have been comfortable delegating. Most people come out of it wanting the agent on the reading and the planning, and their own hand on the motion.

If you have no arm, /live streams a physical SO-100 you can drive from the browser with no signup, queue-based. If you want the model side first, the arena has 85 VLA models with 332 benchmark results, each value linked to its paper or model card, and the dataset directory lists public datasets you can train on without recording anything. When you do start a run, the training walkthrough and the GR00T N1.7 on SO-100 guide have the defaults the trainer actually sends. And if the first thing that breaks is a rejected dataset, that is the v3.0 format mismatch, not your server.

For the wider background on why these policies behave the way they do, the VLA overview covers the model family, the SO-100 guide covers the hardware end to end, and the data collection guide covers the part an agent genuinely cannot do for you.