How limits apply
- Limits apply to your organization, shared across all of its projects and API keys. Adding keys or projects doesn’t raise them.
- Limits are measured in requests and in tokens, per minute and per UTC calendar day. Token usage includes input tokens, output tokens, and reasoning tokens.
- The number of concurrent requests, those in progress at once, is also limited, for your organization and for each API key.
- Your organization’s plan can also set daily allowances of uncached input tokens, the input not served from cache, and of output tokens, reasoning tokens included. They are separate from the daily token limit, so an allowance can run out while the daily token limit still has capacity.
When you exceed a limit
The API returns429 Too Many Requests. The error code is daily_allowance_exhausted when a daily allowance is used up, and rate_limit_exceeded for any other limit; the message says which one:
429 response
Retry-After header, it gives the minimum number of seconds to wait before retrying.
Daily limits and daily allowances renew at 00:00 UTC. When one is spent, the response also includes x-should-retry: false: don’t retry automatically. Retry-After then gives the seconds until 00:00 UTC. The OpenAI SDKs honor this header and don’t retry.
The service can also return 503 with code infrastructure_rate_limit_exceeded or inference_capacity_unavailable when inference capacity is temporarily exhausted. These responses also include Retry-After; retry them the same way. A 503 with code overloaded means inference capacity was exceeded, not your organization’s limits; retry it after a short delay.
Rate limit headers
Once a request is authenticated and accepted for rate limiting, its response reports your organization’s limits and remaining capacity in these headers. Error responses carry them too, including a429.
The same headers with -day appended, such as x-ratelimit-remaining-tokens-day, report your daily limits. Daily capacity resets at 00:00 UTC, and the daily reset headers give the time until then. The daily token limit counts input tokens, cached input included, and output tokens.
When your organization’s plan sets daily allowances, these headers report them, with one set for each allowance the plan sets:
x-ratelimit-reset-uncached-input-tokens-day and x-ratelimit-reset-output-tokens-day give the time until the allowance renews at 00:00 UTC. An allowance’s remaining value counts the usage requests have reported, not an estimate of the current request, so requests already running can use more than remains.
- Request headers appear on every endpoint, and token headers only on requests that generate tokens, such as chat completions. Apart from the daily allowances, token values reflect an estimate of the request’s input tokens, made when the request is accepted, not the request’s final usage.
- For a rate limit, unlike an allowance, a remaining value counts the current request as of when it was accepted. A rejected request uses no capacity.
- A reset value is the time until capacity fully replenishes if you send no more requests. It can be longer than a minute. It isn’t when you can retry: after a
429, wait forRetry-After. - The headers are omitted when capacity can’t be reported, so don’t depend on them being present.
Stay within your limits
- Retry with backoff. Wait at least
Retry-Afterseconds when it is present, and back off exponentially with jitter when it isn’t. The OpenAI SDKs do this automatically; see Retrying. - Limit concurrency. Cap the number of requests in flight rather than sending bursts.
- Watch the rate limit headers to slow down before you reach a limit.
- Set
max_completion_tokensto what you need, and lowerreasoning_effortfor simple tasks. - Trim conversation history you don’t need, to use fewer input tokens.
Request higher limits
First, register and sign in at platform.reflection.ai. Then email developers@reflection.ai with:- Your registered email address and the organization that needs the increase.
- Your use case and whether the traffic is for evaluation, development, or production.
- The request and token rates you need, expected total usage, and how long you need the higher limits.
- Any data retention requirements and whether you’re interested in paid usage.