GR-2
ByteDance, China · October 2024
ByteDance Research
- Parameters
- 230 M
- The default model used in the main experiments contains 230M parameters of which 95M are trainable, corresponding to the...
- GPU memory
- 0.4 GB
- weights at bf16, computed. Fits an 8 GB card
- Inference latency
- not published
- per action step
- Weights
- closed
- No public weights or code release found
What it is
GR-2 is ByteDance Research's generative video-language-action model, and its defining choice is to predict future video first and actions second. A GPT-style transformer is pre-trained on 38 million internet video clips and over 50 billion tokens to learn world dynamics, then fine-tuned jointly for video generation and action prediction on robot trajectories, with a conditional VAE producing whole action trajectories rather than single steps. The default configuration is remarkably small at 230M total parameters with 95M trainable, and it still reaches a CALVIN ABCD-D average length of 4.64 and 97.7 percent across 105 real tasks in the simple setting. The honest caveats are that performance drops to 87.0 percent in unseen environments and 73.9 percent with 50 trajectories per task, and that no code or weights were ever released, so none of this is reproducible outside ByteDance.
Architecture
- Backbone
- GPT-style autoregressive transformer over VQGAN-tokenized video plus a frozen text encoder. Not a VLM-derived architecture.
- Action head
- Conditional VAE that generates an action trajectory rather than single-step actions, which the paper credits for both trajectory smoothing and real-time performance
- Parameters
- The default model used in the main experiments contains 230M parameters of which 95M are trainable, corresponding to the GR-2-B variant. The scaling study covers GR-2-S at 30M trainable, GR-2-B at 95M, GR-2-L at 312M and GR-2-XL at 719M trainable parameters. The frozen VQGAN video tokenizer and the frozen text encoder account for the gap between 95M trainable and 230M total.
- Pretraining data
- 38 million video clips and more than 50 billion tokens of internet video for the video-prediction pre-training stage, followed by fine-tuning for joint video generation and action prediction on robot trajectories.
- Embodiments
- single-arm manipulator with a parallel gripper (ByteDance internal platform), evaluated on CALVIN in simulation
What hardware it needs
This is the section most people came for, so it is worth being precise about which numbers are measured and which are arithmetic.
| Precision | Weights | Note |
|---|---|---|
| fp32 | 0.9 GB | Training master weights, rarely used for inference |
| bf16 or fp16 | 0.4 GB | The usual way these checkpoints are served |
| int8 | 0.2 GB | Quantised, expect some loss of precision on fine motions |
| int4 | 0.1 GB | Aggressive quantisation, verify success rate before trusting it |
Smallest real card that fits the bf16 weights plus a 40 percent runtime allowance: 8 GB. That is an estimate for inference, not for fine tuning. Full fine tuning also holds optimiser state and gradients, which typically costs several times the weights unless you use LoRA or a comparable adapter.
Nothing published. The paper does not state training hardware or inference GPU requirements.
The authors publish no memory or latency figure for this model. Everything above is computed from the parameter count.
Reported results
Grouped by what the evaluation actually asked. Values from different suites are not comparable with each other, so each one keeps its suite and split.
Simulation
Reproducible benchmark suites. Everyone runs the same episodes, so the numbers can be compared inside a suite.
- CALVIN ABCD-D, 1000 evaluated sequences4.64 tasks (out of 5)average length (tasks completed in sequence)Authors' own evaluation. Per-step success is 98.6 percent for the first task and 85.9 percent for five in sequence.source
Real world
Rollouts on physical hardware. The setups differ, so read these as evidence, not as a ranking.
- Real robot, multi-task (ByteDance platform) 105 manipulation tasks, simple setting97.7%average success rateAuthors' own evaluation and the abstract's headline number. Measured in the simple setting, not in the generalization settings.source
- Real robot, multi-task (ByteDance platform) unseen environments with data augmentation87%average success rateAuthors' own evaluation. This is the more decision-relevant number than the 97.7 percent simple-setting figure.source
- Real robot, end-to-end bin picking (ByteDance platform) 122 objects (55 seen, 67 unseen), seen-object setting79%average success rateAuthors' own evaluation. The predecessor GR-1 scores 33.3 percent on the same setting, which is the clearest measure of what the video pre-training buys.source
- Real robot, multi-task (ByteDance platform) limited data, 50 trajectories per task, simple setting73.9%average success rateAuthors' own evaluation. Useful as the data-scarcity floor for anyone planning their own demonstration budget.source
Fine tuned tasks
No results in this category are published for this model.
On real hardware
97.7 percent average success across 105 real manipulation tasks in the simple setting, dropping to 87.0 percent in unseen environments with data augmentation and to 73.9 percent when limited to 50 trajectories per task. End-to-end bin picking on 122 objects (55 seen, 67 unseen) reaches 79.0 percent on seen objects, up from 33.3 percent for the predecessor GR-1.
Fine tuning it yourself
No public fine-tuning path exists. There is no released code or checkpoint, so GR-2 cannot be fine-tuned or reproduced outside ByteDance.
Where it helps, where it does not
Strengths
- Very small for its results: the default configuration is 230M total parameters with 95M trainable, an order of magnitude below the 7B-class models in this comparison
- CALVIN ABCD-D average length of 4.64 out of 5, which was state of the art at publication
- Clean scaling evidence across four sizes from 30M to 719M trainable parameters
- Video pre-training on 38M clips and 50B tokens transfers measurably: bin picking improved from 33.3 percent for GR-1 to 79.0 percent
- Reports the degradation under data scarcity (73.9 percent at 50 trajectories per task) rather than only the best case
Limits
- No code and no weights were found in any official ByteDance repository, so nothing here is reproducible and the model cannot be deployed by third parties
- The 97.7 percent multi-task figure is measured in what the paper calls the simple setting; the same policy falls to 87.0 percent in unseen environments
- No published VRAM, latency, action chunk size or model inference frequency
- The 200 Hz figure in the paper refers to the whole-body controller, not to the policy, and is easy to misread as model throughput
- Superseded by GR-3 in July 2025
Sources
Everything on this page was taken from these documents. Where they disagree with what you read here, they win.
- https://arxiv.org/abs/2410.06158
- https://arxiv.org/html/2410.06158v1
- https://gr2-manipulation.github.io/
Entry last checked 2026-08-11.
Try a policy on a real arm
A physical SO-100 is online and free to drive in the browser, and five of the models in this arena can be fine tuned on a dataset you record with your own.