Rate Limits

How per-minute and concurrent limits are enforced, what a 429 looks like, and how to back off without losing requests.

Updated Aug 2026

Rate limits protect the shared inference fleet by preventing any single key from crowding out others. They are applied per key, measured independently from monthly usage quotas, and shaped so that well-behaved clients rarely encounter them.

Limits are exposed in every response through the X-RateLimit-* headers, so a client can track its remaining budget without making an extra request.

Plan limits

Tier	Requests / minute	Concurrency	Burst
Developer	60 RPM	4 concurrent requests	No burst allowance
Professional	300 RPM	24 concurrent requests	120 requests burst
Enterprise	Custom	Custom	Custom, agreed in contract

How limits are enforced

Requests per minute are evaluated over a sliding one-minute window per API key. Concurrent requests are counted separately and cap in-flight work rather than arrival rate, which matters most for long-running document parses and vision jobs.

RPM is a hard ceiling on arrivals within any rolling 60-second window.
Concurrency caps simultaneous in-flight requests, independent of RPM.
Enterprise agreements may raise both limits and add regional burst pools.

The 429 response

Exceeding a limit returns HTTP 429 with a rate_limit_error code. The Retry-After header states how many seconds to wait before the next attempt, and the limit headers describe the ceiling that was hit.

429.json

{
  "error": {
    "code": "rate_limit_error",
    "message": "Rate limit exceeded: 60 requests per minute on the Developer plan.",
    "request_id": "req_9f2c41d8a1b04e7b9c3e5d6f7a8b9c0d"
  }
}

Exponential backoff

When you receive a 429, wait before retrying, and increase the wait on each consecutive failure rather than hammering the endpoint. Honor the Retry-After header when it is present, and cap the delay so a persistent limit cannot stall a job forever.

backoff.py

import time

def retry_on_429(fn, max_attempts=5):
    delay = 1.0
    for attempt in range(1, max_attempts + 1):
        resp = fn()
        if resp.status_code != 429:
            return resp
        retry_after = float(resp.headers.get("Retry-After", delay))
        time.sleep(min(retry_after if retry_after > 0 else delay, 60.0))
        delay *= 2
    raise RuntimeError("Rate limit persisted across retries")

Concurrency vs quota

Three independent controls are easy to conflate, and each needs a different mitigation.

Rate limit — arrivals per minute. Back off or move to a higher plan.
Concurrency — in-flight requests. Use a bounded worker pool, not a tighter arrival rate.
Usage quota — the monthly call or token allowance. Watch the usage endpoint and alert before exhaustion; exceeding it returns 402, not 429.

Was this helpful?

Your feedback shapes how we improve this documentation.

YesNo