Last updated: September 2026
You are halfway through a long session when the tool stops: “you’ve reached your limit.” Or a script starts throwing errors even though your account has plenty of budget left. Or a big document makes the model quietly forget its own first page. These are three different limits, and people constantly confuse them, which is why the fix so often misses. This guide explains the three kinds of limit on AI tools, why they exist, and how to stop hitting them, without pretending the exact numbers stay still, because they do not.
This is the mechanics of limits, not their price. For what the tiers cost, see our AI tools pricing comparison and how much AI engineering tools cost; for whether a specific plan is worth buying, is ChatGPT Plus worth it for engineers; and for the legal usage rules, AI tool terms of service for engineering. It all sits under our hub on the best AI tools for engineers.
The short version: Three different things all get called “limits.” A usage cap is how much you can send per rolling window on a subscription. An API rate limit is a requests-per-minute and tokens-per-minute ceiling that returns an HTTP 429 error. The context window is how many tokens a single request can hold at once. The first two are about how much you use over time; the context window is about how much the model can read at once, so hitting a limit mid-session usually means one specific thing, and the fix depends on which. Exact numbers change constantly and often are not published, so treat any figure as a snapshot and check the vendor’s live usage page.
Three different things called limits
The single most useful thing to get straight is that “limit” means three unrelated mechanisms. Two of them govern how much you can use over a stretch of time; the third governs how much you can put in front of the model in one go. Conflate them and you will try to fix a context-window problem by upgrading your plan, or blame a rate limit on a cap that has nothing to do with it.
Usage caps: how much over time
On the consumer subscriptions, ChatGPT, Claude, and Gemini apps, a usage cap governs how much you can send before the tool pauses you. These usually reset on a rolling window measured in hours rather than at midnight, and heavier “thinking” models often carry a separate, tighter weekly allowance. Anthropic, for example, documents that a Claude plan’s included usage resets every five hours once you reach it, and that usage is shared across the Claude app and Claude Code ([Anthropic Help Center](https://support.claude.com/en/articles/12429409)). The exact message counts are the most volatile numbers in this whole area and often are not published at all, so the practical move is to watch the in-app usage indicator rather than trust any figure you read online. This bites an engineer mid-task: you are deep in a refactor and the session stalls for a few hours, which is an availability problem, not a billing one.
API rate limits: requests and tokens per minute
On the developer API, the limit is throughput. Providers cap how many requests per minute (RPM) and tokens per minute (TPM) your organization can send, and exceeding either returns an HTTP 429 error, typically with a Retry-After header telling you how long to wait. Anthropic measures requests per minute plus input and output tokens per minute for each model class and returns a 429 with retry-after when you exceed any of them ([Anthropic API docs](https://platform.claude.com/docs/en/api/rate-limits)); OpenAI’s API uses the same RPM and TPM metrics across usage tiers that rise as your cumulative spend grows ([OpenAI API docs](https://developers.openai.com/api/docs/guides/rate-limits)); Google’s Gemini API applies per-minute and per-day limits per project and returns a 429 RESOURCE_EXHAUSTED ([Google Gemini API docs](https://ai.google.dev/gemini-api/docs/rate-limits)). This is the limit that bites automation: a batch job or an agent loop fires too many calls per minute and gets throttled even though plenty of budget remains.
The context window: how much at once
The context window is the one people misfile as a usage cap. It is the number of tokens, your prompt plus the conversation history plus any files and tool output, that the model can hold in a single request. It is a memory ceiling, not an allowance that resets. The newest models hold very large windows, on the order of hundreds of thousands to over a million tokens depending on the model, and those numbers are tied to specific model releases, so confirm the current figure on the vendor’s own model page rather than assuming. When you exceed it, the model does not error politely; it may drop or summarize the earliest turns, so a very long thread or a large document makes it “forget” the top. Regenerating repeatedly inside a bloated thread burns both the window and your usage cap faster, which is how a context problem masquerades as a cap problem.
Why the limits exist
None of this is arbitrary. Providers state plainly that limits are there to prevent abuse and to manage capacity so that one heavy user cannot degrade the service for everyone. Anthropic frames its rate limits as designed to prevent API abuse while minimizing impact on normal usage, and Google describes its Gemini limits as mitigating misuse and managing capacity ([Anthropic](https://platform.claude.com/docs/en/api/rate-limits); [Google](https://ai.google.dev/gemini-api/docs/rate-limits)). Running a frontier model is expensive per token, so caps and rate limits are how a provider keeps a flat subscription price viable and the API responsive. Reading them as capacity management rather than a penalty makes the workarounds obvious.
How to stop hitting them
Match the fix to the limit you are actually hitting:
- Scope the context. Paste only the relevant file or section, not the whole repository or document. This keeps you well under the context window and sends fewer tokens against your caps and TPM.
- Split long tasks into stages. Break a big analysis into sequential smaller prompts so no single request approaches the window ceiling and each one is cheaper.
- Start a fresh chat when a thread gets long. Long threads silently accumulate tokens; a new chat resets the per-request load. Carry a short summary forward by hand rather than relying on auto-truncation.
- Move heavy or automated work to the API, or a higher tier. API throughput tiers scale with your usage history and spend, whereas consumer caps do not, so scripted loops belong on the API.
- Back off on 429s. Respect the Retry-After header, add jitter, and ramp traffic up gradually instead of spiking it. Use a batch endpoint for bulk work.
- Cache repeated context. On APIs that support prompt caching, cache a large system prompt or reference document so it does not count against your throughput on every call.
Frequently asked questions
Why do I keep hitting a limit on ChatGPT or Claude?
Almost always a usage cap: consumer subscriptions allow a certain amount of use per rolling window, often a few hours, with a tighter separate allowance on the heavier reasoning models. It resets on its own after the window passes. If instead you are seeing an HTTP 429 from a script, that is an API rate limit, which is a different mechanism. Check the in-app usage indicator to see which one you are against.
What is the difference between a rate limit and a context window?
A rate limit is about how much you use over time: requests per minute and tokens per minute, enforced with a 429 error. The context window is about how much the model can read at once: the tokens in a single request, prompt plus history plus files. You can be nowhere near a rate limit and still overflow the context window on one huge document, and vice versa. They are unrelated ceilings.
Do usage limits change?
Constantly. Providers adjust caps, rate-limit tiers, and context-window sizes with new model releases and capacity changes, and consumer message counts are often not published at all. Treat any specific number you see, including in this article’s linked sources, as a snapshot with a date, and confirm the current figure on the vendor’s own usage or model page before relying on it.
Does a bigger context window mean I will hit fewer limits?
Not necessarily. A larger context window lets you put more in front of the model at once, but filling it with a huge thread still consumes tokens that count against your usage cap and, on the API, your tokens-per-minute. A big window used carelessly can make you hit the over-time limits faster, which is why scoping the context still matters even on a model that could technically hold everything.
Sources
- Anthropic API documentation, Rate limits (RPM/ITPM/OTPM, 429, retry-after)
- Anthropic Help Center, usage limits and rolling-window reset
- OpenAI API documentation, Rate limits and usage tiers
- Google Gemini API documentation, Rate limits
Written by the CognitiveFuture editorial team. The structural facts here (rate-limit metrics, the 429 and Retry-After behavior, rolling-window resets, and the meaning of the context window) are drawn from the vendors’ own documentation linked above, reached at the time of writing. Specific caps, tier thresholds, and context-window sizes change frequently and are often unpublished, so we describe the mechanisms rather than commit to numbers that will date; confirm current figures on the vendor’s live pages. This is general information, not legal or purchasing advice.
Richard Johnson
Richard Johnson is an AI specialist at one of the world's largest technology companies, where he has spent the past three years helping organizations adopt AI. CognitiveFuture extends that work publicly: gathering the available evidence on each tool, from vendor documentation to independent reviews and user feedback, and cutting a crowded market down to the right choice for the job in front of you.
