The Open Source Robot Learning Ecosystem: Who Ships What

Robot learning is not one open source project. It is four or five projects with different owners, release cadences and licences, plus one shared data format that nobody formally standardised. If you have an SO-100 arm on your desk and want to train a policy on it, you will touch at least three of them, and the seams between them are where the day goes.

This is a map of who publishes what. Every command, flag, default and licence below came out of the actual repository, model card or paper.

The four layers, and who owns each

These are layers, not competing frameworks. Open X-Embodiment is data. LeRobot is the runtime and the format. Isaac-GR00T and openpi are model publishers who also ship training code. Only LeRobot tries to be all of it at once.

The five policies you can actually train here come from three of those publishers: GR00T N1.7 and GR00T N1.5 from NVIDIA, Pi0.5 from Physical Intelligence, SmolVLA from Hugging Face, and ACT out of the ALOHA line at Stanford. Those are the groups shipping usable open weights for tabletop manipulation.

LeRobot is the substrate everything plugs into

LeRobot started in January 2024 and now has the widest surface: robot drivers, teleoperation devices, the dataset format, policy implementations in pure PyTorch, and the CLI verbs that do the work. Version 0.6.1 landed 3 August 2026 and needs Python 3.12 or newer.

pip install lerobot
lerobot-info

# the four verbs that matter
lerobot-record   # drive the arm, write a LeRobotDataset
lerobot-train    # fine-tune a policy on that dataset
lerobot-eval     # score a checkpoint in a sim benchmark
lerobot-rollout  # run the checkpoint on real hardware

What is easy to miss is how much LeRobot has absorbed. Its policy table now lists ACT, Diffusion Policy, VQ-BeT and a multitask DiT for imitation learning; HIL-SERL and TDMPC for reinforcement learning; and for vision-language-action models it lists Pi0, Pi0Fast, Pi0.5, GR00T N1.7, SmolVLA, XVLA, EO-1, MolmoAct2, WALL-OSS and EVO1. Several are reimplementations of models whose authors publish their own repo. You can fine-tune GR00T through NVIDIA's code or LeRobot's, and the two do not take the same flags.

The defaults lerobot-train really uses

Read that last row twice: out of the box you get no validation loss at all. A training step count and a falling training loss tell you the optimiser works, not that the policy will move the arm. The loss falls but the policy does nothing failure mode is common enough to have its own page.

The dataset format is the real interoperability layer

Nobody standardised the LeRobot dataset format. It won by being the thing every tool already reads. openpi's documented path for your own data is to convert it into a LeRobot dataset first, using their convert_libero_data_to_lerobot.py as a template. Isaac-GR00T reads what its README calls a flavour of the LeRobot v2 format. The Hub carries tens of thousands of datasets tagged LeRobot.

# v2.1 -> v3.0, inside a lerobot environment
python -m lerobot.scripts.convert_dataset_v21_to_v30 \
  --repo-id=<HF_USER>/<DATASET_ID>

# v3.0 -> v2, inside the Isaac-GR00T repo, in its own venv
cd scripts/lerobot_conversion
uv venv && source .venv/bin/activate
uv pip install -e . --verbose
python convert_v3_to_v2.py --repo-id <DATASET_REPO_ID>

One more v3.0 detail: it writes Parquet incrementally, so a script that builds a dataset with LeRobotDataset.create() must call dataset.finalize() before pushing, or the footers never flush and the files will not load. Recording through lerobot-record handles this. The dataset docs cover the field names, and record your first dataset walks through producing one from a real arm.

Isaac-GR00T: NVIDIA's model with two front doors

Isaac-GR00T is the reference implementation of NVIDIA's GR00T N-series. The code is Apache 2.0; the weights are not, and they are not all under the same terms. The base nvidia/GR00T-N1.7-3B model card names the NVIDIA Open Model License Agreement. The older nvidia/GR00T-N1.5-3B card points at NVIDIA's OneWay Noncommercial License instead, which is a materially different document if you intend to sell anything. The release tags are public, so the timeline is checkable:

What actually changed in N1.7

• The VLM backbone moved from a vendored Eagle model (nvidia/Eagle-Block2A-2B-v2) to nvidia/Cosmos-Reason2-2B, built on Qwen3-VL.
• State and action dimensions expanded from 29 to 132, and the action horizon from 16 to 40.
• The action head stays a flow-matching DiT but drops from 32 to 16 diffusion layers.
• The rollout flag was renamed from --action-horizon to --execution-horizon, a silent break in anyone's shell scripts.
• Full-pipeline ONNX and TensorRT export landed. The model card reports 27.9 ms end to end and 35.9 Hz on an H100 80 GB, a 3.08x speedup over PyTorch eager.

The fine-tuning entry point is a tyro CLI, and the repo ships an SO100 modality config plus a five-episode SO100 demo dataset, which makes it the most directly relevant of the four projects if you own a low-cost arm.

CUDA_VISIBLE_DEVICES=0 uv run python \
    gr00t/experiment/launch_finetune.py \
    --base-model-path nvidia/GR00T-N1.7-3B \
    --dataset-path demo_data/cube_to_bowl_5 \
    --embodiment-tag NEW_EMBODIMENT \
    --modality-config-path examples/SO100/so100_config.py \
    --num-gpus 1 \
    --output-dir /tmp/test_finetune \
    --max-steps 2000 \
    --global-batch-size 32 \
    --dataloader-num-workers 4

The defaults inside FinetuneConfig explain how GR00T behaves during fine-tuning. It trains only the projector and the diffusion head by default: tune_llm and tune_visual are false, tune_projector and tune_diffusion_model are true. That is why a 3 B model fine-tunes on a single card at all.

The second front door is LeRobot, where GR00T N1.7 is available as --policy.type=groot. That path does expose a seed. It also carries a breaking change: GR00T N1.5 support was removed from LeRobot, and current releases reject N1.5 checkpoints with a migration note. The documented workaround is to pin lerobot==0.5.1, released 7 April 2026. The two generations are compared on the N1.7 versus N1.5 page.

openpi: Physical Intelligence's JAX stack

openpi is the odd one out. It is JAX-first where everything else is PyTorch, it has no tagged releases (the repo is a rolling main branch), and its base checkpoints live in a Google Cloud Storage bucket rather than on the Hub. It publishes pi0, a flow matching VLA; pi0-FAST, an autoregressive variant using the FAST action tokenizer; and Pi0.5, the open-world upgrade trained with knowledge insulation. The base checkpoints were pretrained on what the README calls 10k+ hours of robot data.

That table is the honest reason Pi0.5 sits in the 80 GB tier: a LoRA fine-tune fits on a 4090, a full one does not.

1. 1
   Compute normalisation statistics

   openpi refuses to train without them, and the error is not obvious the first time. Once per config.

   bash

   uv run scripts/compute_norm_stats.py --config-name pi05_libero

2. 2
   Train

   XLA_PYTHON_CLIENT_MEM_FRACTION lets JAX use 90 percent of the card instead of the 75 percent default. On a rented 80 GB GPU that difference is real.

   bash

   XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 uv run scripts/train.py pi05_libero \
     --exp-name=my_experiment --overwrite

3. 3
   Serve the checkpoint

   Serving is a separate process speaking websocket on port 8000. A thin openpi-client goes on the robot side, so the robot environment never imports JAX.

   bash

   uv run scripts/serve_policy.py policy:checkpoint \
     --policy.config=pi05_libero \
     --policy.dir=checkpoints/pi05_libero/my_experiment/20000

4. 4
   Or convert to PyTorch

   If you would rather not run JAX in production, openpi ships a converter and a PyTorch trainer that take the same config names.

   bash

   uv run examples/convert_jax_model_to_pytorch.py \
     --checkpoint_dir /path/to/jax/checkpoint \
     --output_path /path/to/converted/pytorch/checkpoint

One detail from openpi's remote inference guide is worth copying whichever model you run: resize camera frames to 224 by 224 on the client, before they go over the wire. The pretrained pi0 models expect 224 anyway, and pushing full-resolution frames across a network link is the fastest way to make a working policy laggy.

Open X-Embodiment: the data pool underneath all of it

Open X-Embodiment predates the current wave. The paper went up on arXiv on 13 October 2023 and answers one question: what happens if you pool everyone's robot data instead of each lab training on its own.

• 60 existing robot datasets from 34 research labs, 21 institutions on the paper.
• 22 robot embodiments, from single arms to bimanual setups to quadrupeds.
• More than one million real robot trajectories, 527 skills across 160,266 tasks.
• Distributed in RLDS episode format via TensorFlow Datasets and a public GCS bucket.
• Code Apache 2.0, materials CC-BY 4.0, per-dataset licences in a separate spreadsheet to check before commercial use.
• RT-1-X beat per-dataset baselines by 50 percent in the small-data regime; RT-2-X tripled RT-2 on emergent skill evaluations.

# via TensorFlow Datasets
python -c "import tensorflow_datasets as tfds; tfds.load('bridge')"

# or pull the raw RLDS shards
gsutil -m cp -r gs://gdm-robotics-open-x-embodiment/{dataset_name} \
  ~/tensorflow_datasets/

The catch for anyone with a hobby arm: RLDS is not LeRobot format, and almost none of the 22 embodiments is an SO-100. It is what the foundation models were pretrained on, not something you fine-tune against. We covered its structure and limits in the Open X-Embodiment deep dive, and the wider model lineage in the VLA overview.

How the pieces actually fit together

As a chain: Open X-Embodiment and its descendants feed the pretraining of GR00T, pi0 and Pi0.5. You record your own episodes in LeRobot format, fine-tune one of those checkpoints, and serve the resulting checkpoint back to the arm. LeRobot sits at three of those four steps, which is why it feels central despite publishing only one of the models.

For the model decision itself, GR00T N1.7 against Pi0.5 and ACT against SmolVLA lay out the trade-offs with numbers, and the Arena carries 85 VLA models with 332 benchmark results, each value linked back to its source.

Licences: what you can actually ship

This is the part people skip and then regret. All four projects use Apache 2.0 for source. The weights are a different question, and each publisher answers it differently.

Two paths to the same fine-tune

One goal: fine-tune GR00T N1.7 on 50 episodes recorded with an SO-100, then run it back on the arm. Assembled by hand, and on this platform.

Running the open stack yourself: the honest ledger

Where this ecosystem, and this platform, do not help you

Three limits, stated plainly, because the marketing around robot foundation models skips them.

First, latency. Inference latency here runs from 20 ms per action step for ACT to 485 ms for Pi0.5, with GR00T N1.7 at 152 ms and SmolVLA at 245 ms. Those are compute numbers. Add a public-internet round trip and a policy that worked on a local GPU becomes hesitant. Remote inference is viable for slow pick-and-place, not for fast reactive motion, and no amount of action chunking fully hides it. For tight control the inference has to sit next to the servos.

Second, data. None of these projects collects it for you. You cannot fine-tune on someone else's Franka data and expect transfer to your SO-100 gripper. You need your own episodes, and the platform minimum is 30 for SmolVLA and 50 for the others, which is a floor rather than a target. Recording those is manual work, and nobody automates it away.

Third, a platform cannot fix a bad dataset. If camera streams are swapped between episodes, if the wrist camera drifted, if one joint sat at its mechanical limit for half the recording, training converges happily and the policy does something confident and wrong. The failure-mode pages exist because these all look the same from the loss curve. Reading the data collection guide before recording 50 episodes is cheaper than recording them twice.

Where to start if all of this is new

The shortest useful path does not begin with a foundation model. It begins with the smallest thing that closes the loop, because the loop is where you learn whether your data is any good.

1. 1
   Drive an arm before you buy one

   You can teleoperate a physical SO-100 from the browser with no signup at /live, queue-based. It costs nothing and tells you what the latency feels like.

2. 2
   Record 30 to 50 episodes of one task

   Same object, same lighting, same starting region. Vary the object position, not the scene. This is the highest-leverage decision in the whole pipeline.

   bash

   lerobot-record \
     --robot.type=so100_follower \
     --teleop.type=so100_leader \
     --dataset.repo_id=${HF_USER}/cube-pick \
     --dataset.num_episodes=50 \
     --dataset.single_task="Pick up the red cube and place it in the bowl"

3. 3
   Train ACT first, not GR00T

   ACT is roughly 80 M parameters, trains from scratch on a 24 GB card, and runs at 20 ms per action step. It has no base model, so nothing depends on a gated download or a model licence. If ACT cannot learn your task, a 3 B VLA probably will not rescue it.

   bash

   lerobot-train \
     --policy.type=act \
     --dataset.repo_id=${HF_USER}/cube-pick \
     --batch_size=8 \
     --steps=100000

4. 4
   Then move up to a VLA

   Once ACT works, SmolVLA at roughly 450 M parameters is the next step on the same 24 GB card. GR00T N1.7 and Pi0.5 need the 80 GB tier. The training matrix has a guide for every model and arm combination, and run your first policy closes the loop on hardware.

Starting from the hardware end instead? The SO-100 complete guide covers build, calibration and first teleop, the SO-100 and LeRobot page covers the software side, the desktop client records datasets locally, and the training docs describe what the backend does with them.