The training job is stuck in QUEUED and never starts

QUEUED means no card has been bought yet

The platform does not own GPUs. Each run rents one by the hour on a spot market, and the pool selects by VRAM rather than by GPU model name, because what matters is whether the weights and activations fit. The queue is the interval between you pressing Start and an offer appearing that matches the requested amount of memory at an acceptable price. It is a procurement step, and it is the only part of the pipeline whose duration is set by other people.

That has one immediately useful consequence. A queued job has no machine, so it has no logs, no telemetry and no loss chart. The absence of all three is expected and is not evidence of a fault. Nothing is billed either, because there is nothing rented yet.

The bands are ranges rather than prices because they move daily. That volatility is the same mechanism as the queue: when the market is tight, prices are high and offers at a given price are scarce, and both of those show up on your run page as a status that does not change.

Why the big tier waits longer

Cards with 80 GB of memory are datacenter parts. There are fewer of them in the rental market than 24 GB consumer cards, and they are the ones every large training job competes for, not just robotics work. A request for 24 GB matches a large, deep pool of hardware. A request for 80 GB matches a small one, and when that small pool is busy your job waits for someone else to finish.

The market also has busy and quiet periods, so the same request can be filled in minutes at one hour and sit for a long stretch at another. This is worth knowing before you plan an evening around a GR00T run: the wait is not part of your control surface, and the only lever you have is which tier you ask for.

Tell a supply problem from a broken job

1. 1
   Read the status and whether it ever started

   A job that has never left QUEUED has no start time. That distinction is the whole diagnosis: if it never began, no configuration of yours has been evaluated yet, so nothing you wrote in the form can be the cause.

   bash

   curl -X POST https://www.ay-robots.com/api/mcp \
     -H "Authorization: Bearer $AY_ROBOTS_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "jsonrpc": "2.0",
       "id": 3,
       "method": "tools/call",
       "params": {
         "name": "list_training_jobs",
         "arguments": { "status": "QUEUED", "limit": 10 }
       }
     }'

2. 2
   Rule out the failure that looks similar

   A job that went QUEUED, then RUNNING, then FAILED inside a couple of minutes is a different animal and belongs to the dataset version or memory diagnosis, not here. Check whether the status ever changed at all before you conclude you are waiting on hardware.

3. 3
   Check whether you queued it twice

   Pressing Start again does not improve your position. Two queued copies of the same run are two independent rentals, and if both find a card you pay for both. List your queued jobs and cancel the duplicate.

4. 4
   Decide a patience budget before you start waiting

   Pick a number of hours in advance. Waiting is free, but a decision made at midnight about whether to keep waiting is not a decision, it is a mood.

5. 5
   If you are iterating, switch tiers instead of waiting

   Cancel the queued job and start the same dataset on the 24 GB tier. You get an answer today, and the question at this stage is usually about the data rather than about the model.

// Two jobs on the same dataset. Only the tier is different.
{
  "jobs": [
    { "id": "job_c7a13d", "modelType": "groot1.7", "status": "QUEUED",
      "startedAt": null,                    "errorMessage": null },
    { "id": "job_c7a1f0", "modelType": "smolvla", "status": "RUNNING",
      "startedAt": "2026-08-09T09:14:02Z",  "errorMessage": null }
  ],
  "count": 2
}

The practical workaround while you iterate

SmolVLA sits on the 24 GB tier, needs 30 episodes rather than 50, and exists for exactly this situation: it is cheap enough to run repeatedly and good enough to tell you whether a dataset is worth scaling. Running it first is not a compromise, it is the sensible order. See /policies/smolvla for what it is, /train/smolvla-on-so-100 for the run itself, and /compare/groot-n1-7-vs-smolvla for what you are deferring rather than losing.

ACT is the other model on the small tier and it is a different kind of answer. It trains from scratch on your episodes, runs its default 100 000 step schedule, and gives you the fastest inference of anything here. If your task is one fixed repetitive motion, it is often the model you end up deploying rather than a stepping stone. /policies/act and /train/act-on-so-100 cover it.

Once the data has proven itself on the small tier, the 80 GB run is one queued job rather than five, and a long wait costs you a lot less when you only have to have it once. The overall sequence, from recording to evaluation, is at /learn/record-your-first-dataset and /learn/run-your-first-policy.

Frequently asked questions