Fixing Azure OpenAI 429 errors: tokens-per-minute, quota, and what actually works

The 429 is the most-hit error in Azure AI work, and most of the advice about it is "add retries" — which works right up until it quietly becomes your architecture. Here is what the error actually means, why you hit it far sooner than the math suggests, and the fixes in order of effort, from a header you are probably ignoring to capacity you might genuinely need to buy.

New to cloud? CAMPUX is a free, build-first course. Start here →

A 429 from Azure OpenAI means your deployment ran out of its tokens-per-minute or requests-per-minute allowance — the service is fine, your quota is spent. The immediate fix is to honor the Retry-After header and back off; the durable fix is to right-size max_tokens and buy capacity to match what you actually measure.

Let me walk through it the way I would with someone on my team: what the limiter actually counts, the cheap fixes, the expensive ones, then the uncomfortable question of whether you have a retry problem or a capacity problem. They feel the same at 2am. They are not.

What a 429 actually means here

First, the most common misread: a 429 does not mean Azure OpenAI is down, degraded, or having a bad day. It is a deliberate answer from a rate limiter that is working exactly as designed. Your deployment was given a throughput budget, your traffic exceeded it inside the current window, and the platform said "not right now" instead of "no."

The budget has two dials:

• TPM — tokens per minute. The total token throughput your deployment may consume in a minute. This is the main dial.
• RPM — requests per minute. A cap on call count regardless of size. You do not set RPM separately; it rides along with your TPM allocation at a fixed ratio. The ratio varies by model and Microsoft adjusts these over time, so check the current quota documentation rather than trusting a number in a blog post, including this one.

Now the part that explains half the confused Slack threads I have seen: where the quota lives. Quota is held per subscription, per region, per model. Think of it as a pool: your subscription might hold, say, some amount of GPT-4o TPM in East US 2. Every deployment you create for that model in that region carves its slice out of that one pool. Create three deployments of the same model in the same region and they are splitting the same budget, not tripling it. Deploy the same model in a different region, and you are drawing from a different pool entirely. This is why "just make another deployment" sometimes helps (different region or model) and sometimes does nothing at all (same region, same model, same exhausted pool).

One more mechanical detail worth knowing: the 429 response carries a Retry-After header — the platform telling you, in seconds, when it expects to have room for you again. A surprising amount of production code never reads it. We will fix that below.

Why you hit the limit sooner than you think

Here is where the arithmetic most people do in their heads goes wrong. You look at your TPM number, estimate your average request at a few hundred tokens, multiply, and conclude you have enormous headroom. Then you get rate limited at a fraction of that volume. Three reasons, in descending order of how often they are the culprit.

The limiter counts max_tokens at request time, not actual usage

This is the single most common surprise, and as documented as of 2026 it works like this: when your request arrives, the limiter estimates its cost as your prompt tokens plus the full max_tokens you asked for — before the model has generated a single token. Set max_tokens to 4,000 "just to be safe" and every call reserves roughly 4,000 tokens of your minute budget, even when the completion comes back at 80 tokens. The unused reservation is not the bill you pay in dollars — billing is on actual usage — but it absolutely is the budget you burn against the rate limiter.

The practical consequence: a lazy default max_tokens copied from a quickstart can cut your effective throughput by 10x or more. If you take one thing from this article, audit that parameter first.

Bursts inside the minute matter

The limit says "per minute," but enforcement does not politely wait for the minute to end. The platform evaluates your rate over short sub-minute windows — behavior Microsoft has described in terms of one-second and ten-second buckets, though I would hedge on the exact mechanics since they are an implementation detail that can change. What it means for you: firing your entire minute's worth of traffic in the first two seconds will draw 429s even though your total for the minute was technically under budget. Smooth traffic passes; spiky traffic of the same volume gets throttled. If your app fans out twenty parallel calls the instant a user clicks, that burst is the problem, not your monthly volume.

Streaming does not exempt you

A streamed response feels lighter, and people half-consciously assume it is metered differently. It is not. A streaming request is estimated and counted the same way at admission — prompt plus max_tokens, reserved up front. Streaming improves perceived latency for your user; it does nothing for your quota.

Billing charges you for the tokens you used. The rate limiter charges you for the tokens you asked permission to use. Those are very different numbers.

The immediate fixes, in order of effort

These are the same-day fixes, cheapest first. Most teams need the first three and never think about them again.

1. Read Retry-After and honor it

The 429 tells you exactly how long to wait. Honoring it beats any backoff schedule you invent, because it reflects the limiter's actual state rather than your guess. If you are using a recent official SDK, good news: the OpenAI Python and JavaScript libraries retry 429s automatically with backoff, and they respect Retry-After. Before writing retry code, check whether your client already does it — many hand-rolled retry loops are duplicating (and fighting) the SDK's built-in behavior.

2. Backoff with jitter, done right

If you do roll your own — a raw REST integration, or a framework that swallows the SDK's retries — the shape that works is exponential backoff with jitter, capped, with Retry-After taking precedence when present:

import random, time

def call_with_backoff(fn, max_attempts=6):
    for attempt in range(max_attempts):
        try:
            return fn()
        except RateLimitError as e:
            if attempt == max_attempts - 1:
                raise
            retry_after = getattr(e, "retry_after", None)
            if retry_after:                      # server told us; believe it
                delay = float(retry_after)
            else:                                # else: exponential + jitter
                delay = min(60, (2 ** attempt) + random.uniform(0, 1))
            time.sleep(delay)

The jitter is not decoration. Without it, every throttled worker in your fleet sleeps the same interval and wakes at the same instant — a synchronized stampede that re-triggers the very limit you backed off from. The tenacity library gives you the same pattern declaratively (wait_random_exponential) if you prefer not to hand-write it.

3. Right-size max_tokens per call

Given what section two established, this is the highest-value cheap fix. Stop using one global max_tokens. A classification call that returns one word does not need the same ceiling as a report generator. Set each call type's limit to a sane maximum for what it actually produces, and your effective TPM can multiply without touching quota at all.

4. Get background work off the interactive path

If nightly summarization jobs, embedding backfills, and user-facing chat all share one deployment, your batch work is spending the quota your users need at the worst moments. Push background work through a queue with a controlled concurrency and let it drain slowly — it does not care about latency, so make it the traffic that waits. If the volume is large, Azure OpenAI's Batch API runs it at lower priority (and, at the time of writing, lower cost) without touching your interactive limits at all.

The capacity fixes

If you have done all of the above and still see daily 429s, you have outgrown your allocation. Now it is a capacity conversation, and there are four levels.

Raise the deployment's TPM allocation

Remember the pool model: your deployment holds a slice of the subscription's regional quota for that model. Often the slice is small while the pool still has room — especially if the deployment was created with a default allocation. In the Azure AI Foundry portal (or the quota blade), check the model's quota for your region and drag the deployment's TPM up to what the pool allows. This is a two-minute change and it is genuinely common for it to be the whole fix.

Request a quota increase

If the pool itself is exhausted, you can file a quota increase request from the same quota view. Be honest with yourself about timelines: these are reviewed, not automatic, and approval depends on regional capacity for that model. Days, sometimes longer, sometimes declined for constrained models. File it early, but do not make it your only plan.

Split across a second region

Since quota pools are per region, a deployment of the same model in a second region is new budget. The pattern is a thin router in front: send traffic to region A, spill to region B on a 429 (or round-robin if you prefer). Azure API Management can do this with policy, or twenty lines in your own gateway does it fine. Two caveats that deserve respect: data residency — if your compliance posture requires prompts and outputs to stay in a geography, confirm the second region qualifies before you route a single request; and consistency — model versions and feature availability can differ slightly by region, so pin versions explicitly.

PTU: provisioned throughput, and what it is actually for

Everything above is pay-as-you-go, where you share pooled capacity and the TPM/RPM limits are how the platform keeps sharing fair. Provisioned throughput units (PTUs) are the other model: you reserve dedicated capacity for your deployment. What you get is predictable throughput and, just as valuable in production, predictable latency — you are no longer sharing a lane. What you pay is a commitment: PTU pricing is a monthly-reservation style of economics that only makes sense at sustained volume, and I will hedge on figures because they change — check current Azure pricing and run your own arithmetic.

The honest note, because vendors will not say it plainly: PTU is not a fix for a development annoyance. If your 429s come from a demo, a test suite, or a bursty internal tool, PTU is a truck you are buying to deliver one letter. It is for measured, sustained production load where the commitment is cheaper than the equivalent pay-go tokens — or where latency predictability is worth the premium on its own. There is also a useful middle pattern: spillover, where your PTU deployment handles the base load and overflow spills to a pay-go deployment instead of failing. Microsoft has been building this out as a managed feature; availability and mechanics have been shifting, so verify the current state in the docs before you design around it.

The part the retry tutorials skip

Here is the honest gap. Retries do not create capacity. Every retry is the same work submitted again, later — which means under sustained overload, retries convert your capacity shortage into a latency problem and then hide it. The request eventually succeeds, your error rate looks fine, and meanwhile your p95 latency has tripled because half your traffic is sleeping in backoff loops. I have watched teams tune retry parameters for weeks when the true situation was simple: they needed more TPM than they owned.

The test is frequency. A 429 during a traffic spike, absorbed by backoff, invisible to users — that is the system working. 429s every day, at predictable hours, with users feeling the slowdown — that is not a transient error, that is a capacity-planning problem wearing an error code. Retrying harder at that point is denial with jitter.

The way out is to measure before you buy. Two numbers, from your logs, this week: tokens per request (prompt and completion separately — and compare completion actuals against your max_tokens settings, because the gap between them is free throughput you are leaving on the table) and requests per user session at peak. Multiply out to peak TPM demand, add sensible headroom, and now you know whether the answer is a bigger allocation, a second region, or PTUs — and you can defend the spend with arithmetic instead of vibes. Cost and capacity are the same conversation here; what AI workloads actually cost on Azure walks the money side of the same measurement.

This measure-then-provision habit is the difference between operating AI infrastructure and being surprised by it — the same discipline we drill in the free classes, and in more depth in the live cohort, where students size and deploy real workloads rather than reading about them.

A diagnosis checklist you can run today

1. Confirm it is really a rate limit. 429 with a Retry-After header, error text about token rate limits. (A 429 for exceeding a spending cap is a different beast with a similar face — the error body tells you which.)
2. Check your max_tokens defaults. If a global value is 4x your typical completion, you found it.
3. Look at burst shape. Are the 429s clustered in the same seconds? Smooth the fan-out.
4. Separate traffic classes. Is batch work sharing the users' deployment?
5. Check the quota blade. Is the deployment's TPM slice small while the regional pool has room?
6. Only then talk about quota increases, second regions, or PTUs — with measured tokens-per-request in hand.

If you work through those six in order, you will fix the large majority of 429 situations without spending a dollar. And if you want the surrounding context — what Azure AI Foundry is and how deployments fit into it — the Foundry explainer is the right companion read, with the AI infrastructure overview behind it.

Questions people also ask

What does error 429 mean in Azure OpenAI?

A 429 means your requests exceeded the rate limits on your Azure OpenAI deployment — either tokens per minute (TPM) or requests per minute (RPM). The service is up and your key is valid; you have simply asked for more capacity in that minute than your deployment was allocated. The response usually includes a Retry-After header telling you how long to wait before trying again.

How do I fix the 429 rate limit error in Azure OpenAI?

In order of effort: honor the Retry-After header and retry with exponential backoff and jitter; lower max_tokens to what your responses actually need, since the limiter counts it against your quota at request time; move background work into a queue so it stops competing with interactive traffic; then raise the deployment's TPM allocation in the portal, request a quota increase, or add a deployment in a second region. If you are retrying constantly every day, treat it as a capacity problem, not a transient error.

Does max_tokens affect Azure OpenAI rate limits?

Yes, and more than most people expect. As documented as of 2026, the rate limiter estimates a request's cost using your prompt size plus the max_tokens you set, at the time the request arrives — before the model generates anything. A request with max_tokens set to 4,000 reserves roughly that much from your tokens-per-minute budget even if the completion comes back at 80 tokens. Right-sizing max_tokens per call is often the single cheapest fix for 429s.

What is the difference between TPM and RPM in Azure OpenAI?

TPM is tokens per minute — the total token throughput a deployment may use in a minute, counting prompt tokens plus reserved completion tokens. RPM is requests per minute — a cap on how many calls you can make regardless of size. When you assign TPM to a deployment, an RPM limit comes with it at a fixed ratio that varies by model, so check the current Microsoft documentation for the ratio on your model. You can hit either limit first: many small requests trip RPM, a few large ones trip TPM.

Should I use provisioned throughput (PTU) to fix 429 errors?

Only if you have sustained production load. Provisioned throughput units reserve dedicated capacity, which gives you predictable latency and throughput, but they come with a monthly-commitment style of pricing that only pays off at steady volume — check current Azure pricing, because the economics change. For a development annoyance or bursty low traffic, backoff, right-sized max_tokens, and a modest TPM increase are the right tools. PTU is a capacity purchase, and it should follow measurement of your real token usage, not frustration.