AY-ROBOTSAI Teleoperation
Sign Up

CogACT-Large

Microsoft, United States · November 2024

Microsoft Research

Foundation VLAOpen weights, MITAlso written CogACT DiT-L
Parameters
not published
The paper gives 7.6B only for the default DiT-Base configuration and never states a total for the Large variant.
GPU memory
30 GB
reported by the authors
Inference latency
not published
per action step
Weights
MIT
downloadable checkpoint

What it is

CogACT-Large keeps the same roughly 7B parameter Prismatic backbone as the other variants and scales only the diffusion action module, from 89M to 308M parameters. In the authors' action module ablation it is the strongest of the three, averaging 64.8 percent across the three SIMPLER settings against 62.5 percent for Base, with the biggest gain on the WidowX suite (58.3 versus 51.3 percent). The gain is not uniform: on Google Robot variant aggregation it actually falls behind Base, 59.3 versus 61.3 percent. It is the least downloaded of the three checkpoints and the paper's headline claims, real robot experiments and inference timing were all produced with the Base configuration, so evidence for this variant rests on one ablation table.

Architecture

Backbone
Prismatic VLM at 7B scale: DINOv2 ViT-L/14 plus SigLIP ViT-So400M/14 vision encoders with a Llama-2 language module
Action head
Diffusion transformer (DiT-Large, 308M parameters) conditioned on a single cognition token, sampled with DDIM and classifier-free guidance
Action chunk
16 steps per forward pass
Parameters
The paper gives 7.6B only for the default DiT-Base configuration and never states a total for the Large variant. The action module itself is 308M parameters (Table 7). The released checkpoint file is 31.40 GB in fp32, which is 0.88 GB larger than the Base checkpoint, consistent with the 219M parameter difference between DiT-B and DiT-L.
Pretraining data
A subset of Open X-Embodiment, about 0.4M trajectories according to the project page.
Embodiments
Google Robot (SIMPLER simulation), WidowX / Bridge (SIMPLER simulation)

What hardware it needs

This is the section most people came for, so it is worth being precise about which numbers are measured and which are arithmetic.

Published figures. These come from the model authors, not from this site.
Inference memory, reported30 GB
Checkpoint download31.4 GB

No variant specific figure. The family wide statement is about 30 GB in fp32 for inference; the Large checkpoint file alone is 31.40 GB.

Reported results

Grouped by what the evaluation actually asked. Values from different suites are not comparable with each other, so each one keeps its suite and split.

Simulation

Reproducible benchmark suites. Everyone runs the same episodes, so the numbers can be compared inside a suite.

  • SIMPLER Google Robot, visual matching
    76.7%
    success rateFrom the authors' action module ablation table, not the main results table.source
  • SIMPLER average over Google Robot visual matching, Google Robot variant aggregation and WidowX visual matching
    64.8%
    success rateHighest of the three variants. Small 58.5, Base 62.5 in the same table.source
  • SIMPLER Google Robot, variant aggregation
    59.3%
    success rateLower than CogACT-Base at 61.3 in the same table.source
  • SIMPLER WidowX / Bridge, visual matching
    58.3%
    success rateBest WidowX result in the family, versus 51.3 for Base.source

Real world

No results in this category are published for this model.

Fine tuned tasks

No results in this category are published for this model.

On real hardware

No real robot evaluation of the Large variant is published. The real hardware experiments in the paper (Realman and Franka) use the default DiT-Base configuration.

Fine tuning it yourself

Identical pipeline to CogACT-Base: RLDS or Open X-Embodiment data with 7-DoF end effector delta actions, Python 3.10, PyTorch 2.2 or newer, CUDA 12 or newer, Flash-Attention 2 for training, reference config of one node with 8 A100 GPUs.

Where it helps, where it does not

Strengths

  • Best average across the three SIMPLER settings in the authors' ablation, 64.8 percent
  • Clear gain on the WidowX suite, 58.3 versus 51.3 percent for Base
  • The extra 219M parameters cost only 0.88 GB of checkpoint size relative to Base
  • Same MIT license and same tooling as the rest of the family

Limits

  • Not uniformly better than Base: 59.3 versus 61.3 percent on Google Robot variant aggregation.
  • All evidence comes from a single ablation table. No real robot results and no separately measured latency exist for this variant.
  • No published total parameter count for this specific variant.
  • No latency measurement of its own. The 181 ms per inference figure in the README refers to CogACT-Base, and a 3.5 times larger action module will not be free.
  • Same single camera, single arm restriction as the rest of the family.

Sources

Everything on this page was taken from these documents. Where they disagree with what you read here, they win.

Entry last checked 2026-08-11.

Try a policy on a real arm

A physical SO-100 is online and free to drive in the browser, and five of the models in this arena can be fine tuned on a dataset you record with your own.

Download the clientDrive a real arm now