Rate limits

Rate limiting is enforced at the edge load balancer, not in the application. When you exceed a limit, the gateway returns HTTP 429.

Current limits (Power Beta)

Bucket	Limit
Per Authorization value (any auth)	600 req / 60 s
Per IP (unauthenticated)	60 req / 60 s

These limits apply across the entire /v1/* surface — not per-endpoint.

The 429 response

The 429 response body is generated by the load balancer as a short HTML page rather than the standard JSON error envelope.

Do not parse the body. Rely on the status code alone to detect throttling.

Recommended client behaviour

1. Catch 429 responses explicitly.
2. Back off exponentially with jitter — e.g. start at 1 s, double on each retry up to a 30 s ceiling, plus 0–500 ms jitter.
3. Cap total retries at a number that matches your SLO — typically 3 to 5 attempts.
4. Do not retry on 400, 401, or 403. Those will not succeed on a second attempt.

Streaming and long requests

Streaming chat completions count as a single request against the rate limit regardless of duration. If you exceed the limit while a stream is in progress, the in-flight stream completes; only the next request is throttled.

Upstream requests are also subject to a server-side timeout (currently 120 seconds). Requests that exceed this return 502 rather than 429.