KIT

MoDE

Karlsruhe Institute of Technology, Germany, United States 路 December 2024

Karlsruhe Institute of Technology (Intuitive Robots Lab) and MIT CSAIL

Robot foundation model 路 Open weights, MIT 路 Also written Mixture of Denoising Experts, MoDE Diffusion Policy, MoDE_Pretrained

Parameters
not published
The paper's figures disagree: Table 3 lists 685M parameters for the pretrained model, while Table 8 lists 740M total and 436M...
GPU memory
not published
weights at bf16, computed
Inference latency
12.2 ms
an NVIDIA A6000, average per action
Weights
MIT
MIT for the repository and the model card.

What it is

MoDE is a diffusion transformer policy from KIT and MIT that replaces dense feed-forward layers with a mixture of denoising experts routed by the noise level, so each denoising step uses only part of the network and the chosen experts can be cached. Pretrained on 196K trajectories from six Open X-Embodiment sources, it reaches an average length of 4.01 on CALVIN ABC to D and 95 percent on LIBERO-90, and it averages 26.3 percent on SimplerEnv, ahead of OpenVLA (23.7) and Octo (17.75) in the authors' table. Language enters through a frozen CLIP text encoder and images through FiLM-conditioned ResNets, with no vision language model. Inference took 12.2 ms per action on an NVIDIA A6000 in the paper.

Architecture

Backbone
No VLM backbone. Frozen CLIP ViT-B/32 text encoder and FiLM-conditioned ResNet-50 image encoders (ResNet-18 in some benchmark models) feeding a mixture-of-experts transformer with 12 layers, 4 experts and width 1024 in the pretrained model.
Action head
Diffusion transformer with noise-conditioned expert routing and noise-conditioned self-attention; DDIM sampling with 5 steps according to the model card, action window of 10, 7-dimensional delta end-effector actions.
Action chunk
10 steps per forward pass
Parameters
The paper's figures disagree: Table 3 lists 685M parameters for the pretrained model, while Table 8 lists 740M total and 436M active. The benchmark models without pretraining are listed at 460M.
Pretraining data
196K trajectories from six sources: BC-Z, LIBERO-10, Bridge, CMU Play-Fusion, Google Fractal and DOBB-E.
Embodiments
Franka (CALVIN and LIBERO simulation), Google robot and WidowX (SimplerEnv)

What hardware it needs

This is the section most people came for, so it is worth being precise about which numbers are measured and which are arithmetic.

Published figures. These come from the model authors, not from this site.
Inference latency, reported12.2 ms on an NVIDIA A6000, average per action source

No memory figure. Pretraining took 6 NVIDIA A6000 GPUs for 3 days (300K steps).

Reported results

Grouped by what the evaluation actually asked. Values from different suites are not comparable with each other, so each one keeps its suite and split.

Simulation

Reproducible benchmark suites. Everyone runs the same episodes, so the numbers can be compared inside a suite.

  • LIBERO LIBERO-90, pretrained model
    95%
    success rateAuthors' own evaluation, reported as 0.95 in the abstract. Without pretraining 91.0 percent.source
  • SimplerEnv average over the authors' SIMPLER task set
    26.3%
    success rateAuthors' own evaluation. OpenVLA 23.7 and Octo 17.75 on the same task set.source
  • CALVIN ABCD to D, pretrained model
    4.39 tasks (out of 5)
    average length (tasks completed in sequence)Authors' own evaluation, plus or minus 0.04. Without pretraining 4.30.source
  • CALVIN ABC to D, pretrained model
    4.01 tasks (out of 5)
    average length (tasks completed in sequence)Authors' own evaluation, plus or minus 0.04. Without pretraining 3.39.source

Real world

No results in this category are published for this model.

Fine tuned tasks

No results in this category are published for this model.

On real hardware

No real robot experiments are reported.

Fine tuning it yourself

Start from mbreuss/MoDE_Pretrained, pretrained for 300k steps on the Open X-Embodiment subset, and fine-tune on your task; the model takes a static and a gripper RGB camera plus a language instruction. Fine-tuned CALVIN and LIBERO checkpoints are published by the same author.

Where it helps, where it does not

Strengths

  • Efficient: 12.2 ms per action, and expert caching cuts inference cost by about 90 percent, according to the authors.
  • Strong simulation results for its size: 4.01 on CALVIN ABC to D and 95 percent on LIBERO-90.
  • Open weights under MIT, with pretrained and fine-tuned checkpoints.

Limits

  • Simulation only; no real robot evaluation.
  • Parameter counts are inconsistent across the paper's tables.
  • Small pretraining mixture: 196K trajectories against 800k for Octo.

Sources

Everything on this page was taken from these documents. Where they disagree with what you read here, they win.

Entry last checked 2026-11-12.

Try a policy on a real arm

A real SO-100 runs on /live and can be driven from the browser when it is online. Five of the models in this arena can be fine tuned on a dataset you record with your own arm.