CogACT-Small
Microsoft, United States · November 2024
Microsoft Research
- Parameters
- not published
- The paper gives 7.6B only for the default DiT-Base configuration and never states a total for the Small variant.
- GPU memory
- 30 GB
- reported by the authors
- Inference latency
- not published
- per action step
- Weights
- MIT
- downloadable checkpoint
What it is
CogACT-Small is the smallest of the three released CogACT checkpoints and differs from the Base variant only in the diffusion action module, which shrinks from 89M to 13M parameters while the roughly 7B parameter Prismatic backbone stays the same. Because the backbone dominates, the checkpoint is still a 30.22 GB fp32 file and the memory requirement is essentially identical to Base, so this variant buys almost no deployment savings. The paper's action module ablation shows it costs real accuracy: 58.5 percent average across the three SIMPLER settings against 62.5 percent for Base and 64.8 percent for Large. Its numbers come from the ablation table rather than the main results tables, and no real robot evaluation of this variant is published.
Architecture
- Backbone
- Prismatic VLM at 7B scale: DINOv2 ViT-L/14 plus SigLIP ViT-So400M/14 vision encoders with a Llama-2 language module
- Action head
- Diffusion transformer (DiT-Small, 13M parameters) conditioned on a single cognition token, sampled with DDIM and classifier-free guidance
- Action chunk
- 16 steps per forward pass
- Parameters
- The paper gives 7.6B only for the default DiT-Base configuration and never states a total for the Small variant. The action module itself is 13M parameters (Table 7). The released checkpoint file is 30.22 GB in fp32, which is 0.30 GB smaller than the Base checkpoint, consistent with the 76M parameter difference between DiT-S and DiT-B.
- Pretraining data
- A subset of Open X-Embodiment, about 0.4M trajectories according to the project page.
- Embodiments
- Google Robot (SIMPLER simulation), WidowX / Bridge (SIMPLER simulation)
What hardware it needs
This is the section most people came for, so it is worth being precise about which numbers are measured and which are arithmetic.
| Inference memory, reported | 30 GB |
| Checkpoint download | 30.22 GB |
No variant specific figure. The family wide statement is about 30 GB in fp32 for inference, with a bf16 path mentioned but not measured.
Reported results
Grouped by what the evaluation actually asked. Values from different suites are not comparable with each other, so each one keeps its suite and split.
Simulation
Reproducible benchmark suites. Everyone runs the same episodes, so the numbers can be compared inside a suite.
- SIMPLER Google Robot, visual matching73.3%
- SIMPLER average over Google Robot visual matching, Google Robot variant aggregation and WidowX visual matching58.5%
- SIMPLER Google Robot, variant aggregation51.3%success rateFrom the authors' action module ablation table. Base reaches 61.3 in the same table.source
- SIMPLER WidowX / Bridge, visual matching51%
Real world
No results in this category are published for this model.
Fine tuned tasks
No results in this category are published for this model.
On real hardware
No real robot evaluation of the Small variant is published. The real hardware experiments in the paper (Realman and Franka) use the default DiT-Base configuration.
Fine tuning it yourself
Identical pipeline to CogACT-Base: RLDS or Open X-Embodiment data with 7-DoF end effector delta actions, Python 3.10, PyTorch 2.2 or newer, CUDA 12 or newer, Flash-Attention 2 for training, reference config of one node with 8 A100 GPUs.
Where it helps, where it does not
Strengths
- Same MIT license and same download path as the other two variants
- Smallest action module in the family, useful as a control point when studying action head scaling
Limits
- Almost no practical benefit over CogACT-Base: the 76M parameter saving is invisible next to the roughly 7B backbone and the checkpoint is only 0.30 GB smaller.
- Measurably worse than Base on SIMPLER: 51.3 versus 61.3 on Google Robot variant aggregation.
- Only ablation table numbers exist for this variant. No real robot results and no separately measured latency were published.
- No published total parameter count for this specific variant.
- Same single camera, single arm restriction as the rest of the family.
Sources
Everything on this page was taken from these documents. Where they disagree with what you read here, they win.
- https://arxiv.org/abs/2411.19650
- https://arxiv.org/html/2411.19650v1
- https://github.com/microsoft/CogACT
- https://huggingface.co/CogACT/CogACT-Small
- https://cogact.github.io/
Entry last checked 2026-08-11.
Try a policy on a real arm
A physical SO-100 is online and free to drive in the browser, and five of the models in this arena can be fine tuned on a dataset you record with your own.