Seer-Large
Shanghai AI Laboratory, China · December 2024
Shanghai AI Laboratory, with Peking University and the Chinese University of Hong Kong
- Parameters
- 566 M
- The paper reports 315M trainable parameters for Seer-Large plus the same 251M frozen parameters as the base model, giving 566M...
- GPU memory
- 1.1 GB
- weights at bf16, computed. Fits an 8 GB card
- Inference latency
- not published
- per action step
- Weights
- closed
- no public checkpoint
What it is
Seer-Large is the scaled version of Seer, with 315M trainable parameters on top of the same 251M frozen MAE ViT-B and CLIP encoders, 566M in total. It holds the paper's headline result of 4.28 average successful task length on CALVIN ABC-D, which was state of the art at ICLR 2025. The scaling is real rather than cosmetic: the benefit of pre-training grows from 0.34 to 0.45 tasks when moving from the base model to this one. The caveat for anyone choosing between the two sizes is that Seer-Large is only evaluated on CALVIN, with no LIBERO and no real-robot numbers, so paying 1.8 times the memory is only justified if your task resembles CALVIN.
Architecture
- Backbone
- Same architecture as Seer with a larger transformer trunk. MAE-pretrained ViT-B image encoder plus frozen CLIP ViT-B/32 text encoder (251M frozen parameters combined), Perceiver Resampler token compression, MLP state encoder.
- Action head
- Predictive inverse dynamics: forecast future visual states, then regress actions conditioned on the forecast, trained end to end.
- Parameters
- The paper reports 315M trainable parameters for Seer-Large plus the same 251M frozen parameters as the base model, giving 566M in total. Trainable capacity is therefore roughly 4.8 times that of Seer while total size grows only 1.8 times, because the frozen encoders are shared.
- Pretraining data
- Same pipeline as Seer. Pre-training for CALVIN ABC-D takes about 40 hours on eight RTX 4090 GPUs.
- Embodiments
- CALVIN simulation benchmark, LIBERO simulation benchmark
What hardware it needs
This is the section most people came for, so it is worth being precise about which numbers are measured and which are arithmetic.
| Precision | Weights | Note |
|---|---|---|
| fp32 | 2.1 GB | Training master weights, rarely used for inference |
| bf16 or fp16 | 1.1 GB | The usual way these checkpoints are served |
| int8 | 0.5 GB | Quantised, expect some loss of precision on fine motions |
| int4 | 0.3 GB | Aggressive quantisation, verify success rate before trusting it |
Smallest real card that fits the bf16 weights plus a 40 percent runtime allowance: 8 GB. That is an estimate for inference, not for fine tuning. Full fine tuning also holds optimiser state and gradients, which typically costs several times the weights unless you use LoRA or a comparable adapter.
The authors state eight RTX 4090 GPUs were used for pre-training and fine-tuning, with about 40 hours of pre-training for CALVIN ABC-D. No VRAM figure and no single-GPU minimum is published. With 566M total parameters this variant is roughly 1.8 times the memory footprint of the base model.
The authors publish no memory or latency figure for this model. Everything above is computed from the parameter count.
Reported results
Grouped by what the evaluation actually asked. Values from different suites are not comparable with each other, so each one keeps its suite and split.
Simulation
Reproducible benchmark suites. Everyone runs the same episodes, so the numbers can be compared inside a suite.
- CALVIN ABC to D, released Seer-Large checkpoint as shipped4.3 tasksaverage successful sequence lengthValue stated for the downloadable checkpoint in the repository README, which differs slightly from the 4.28 in the paper table.source
- CALVIN ABC to D, long-horizon chains of 5 language instructions4.28 tasksaverage successful sequence lengthAuthors' own evaluation, the state-of-the-art figure quoted in the paper abstract. The released checkpoint in the GitHub README is listed at 4.30, a small discrepancy against the published table.source
- CALVIN ABC to D, Seer-Large trained from scratch without pre-training3.83 tasksaverage successful sequence lengthAuthors' own ablation. Pre-training is worth 0.45 tasks at this model size, more than the 0.34 it is worth for the base model.source
Real world
No results in this category are published for this model.
Fine tuned tasks
No results in this category are published for this model.
On real hardware
No real-world results are published for Seer-Large. The paper's real-world section reports the base Seer model fine-tuned from the DROID pre-trained checkpoint.
Fine tuning it yourself
The repository releases training code for Seer-Large on the CALVIN ABC-D experiments and a downloadable checkpoint via Google Drive. Fine-tuning follows the same recipe as the base model. Budget for roughly 566M parameters in memory, of which 315M are trainable.
Where it helps, where it does not
Strengths
- Highest CALVIN ABC-D average length reported in the paper at 4.28 out of a maximum of 5, above 3D Diffuser Actor at 3.27 and GR-1 at 3.06.
- Scaling behaves as intended: the from-scratch to pre-trained gap widens from 0.34 to 0.45 tasks when going from Seer to Seer-Large, so the larger trunk actually exploits the pre-training.
- Still small enough to train on eight consumer RTX 4090 cards, unlike VLA-scale models built on 7B backbones.
- Apache-2.0 code and weights.
Limits
- The paper reports Seer-Large only on CALVIN. No LIBERO-LONG and no real-world numbers are published for this size, so its advantage outside CALVIN is unproven.
- 1.8 times the memory of the base model for a benchmark gain that is only demonstrated on one suite.
- The released checkpoint reports 4.30 while the paper table says 4.28, so the shipped artifact and the publication are not exactly the same model.
- Same structural limits as the base model: no language model, no reasoning, checkpoints on Google Drive rather than Hugging Face, and no published latency or VRAM figures.
Sources
Everything on this page was taken from these documents. Where they disagree with what you read here, they win.
- https://arxiv.org/abs/2412.15109
- https://arxiv.org/html/2412.15109v1
- https://github.com/InternRobotics/Seer
- https://raw.githubusercontent.com/InternRobotics/Seer/main/README.md
- https://openreview.net/forum?id=meRCKuUpmc
Entry last checked 2026-08-11.
Try a policy on a real arm
A physical SO-100 is online and free to drive in the browser, and five of the models in this arena can be fine tuned on a dataset you record with your own.