ChatGPT System Analysis: A Metered Compute Service Running Into a Capacity Ceiling

ChatGPT works as a metered compute service. Every request passes through a plan-based limit check, a model choice, optional tools, and then a shared pool of data-centre capacity that is the scarcest resource in the whole system. OpenAI does not publish its model architecture, so this article describes the layers that public sources make visible, not the internals of the models.

The request path

A request starts with the user's plan (Free, Go, Plus, Pro, Business or Enterprise) and arrives through an app: web, mobile, desktop, or Codex. It then meets the metering and routing layer, which applies plan caps and decides whether to fall back to a lighter model. From there it reaches a model (a family running from the deep Sol to the fast Luna) and any tools it needs, such as Deep Research, image and voice generation, or agents. Everything lands on a compute pool spread across Azure, Oracle, AWS and CoreWeave. Each layer can throttle the one after it, and the metering layer is where a power user first feels a limit.

Layer by layer

Apps and access. Demand is spiky and tied to launches. Weekly Codex usage in India rose four times after the Codex Mac app arrived.

Plans and metering. This layer assigns text, reasoning, file, voice and Codex allowances, resets them on rolling windows and sells credits beyond them. The caps are deliberately opaque and change often.

Model routing. OpenAI offers a model family from deep to fast and falls back to a lighter model when a cap is reached. The cost of that fallback is that quality drops silently for users who hit caps.

Tools and agents. Search, Deep Research, image and video generation, voice, file analysis, connectors and cloud coding tasks. Multi-step agents multiply tokens per request.

Compute and hosting. Reported Stargate capacity is about 7 GW through 2028, alongside a $300 billion Oracle contract, Microsoft Azure, and a $50 billion AWS agreement for OpenAI's agent platform. Compute spend is about $50 billion in 2026 and exceeds revenue, and partner disagreements have delayed data-centre governance.

Regional infrastructure. In India, a first 100 MW of capacity is planned with Tata Group's TCS HyperVault, scalable to 1 GW, to meet data-residency needs. It is not yet at scale.

The core tension

Demand is elastic and capacity is not. OpenAI's stated reason for tighter limits is that higher caps weaken demand smoothing and hit a capacity ceiling faster. Free promotions such as ChatGPT Go in India push demand up without adding revenue, so limits, fallbacks and credits are the levers that keep the system stable while OpenAI builds more capacity.

Failure modes users actually see

A quality drop after the fallback to a lighter model. A sudden change in caps after a new model launch. A retired model with short notice. A higher bill when Codex switches from included usage to credits.

What would change the picture

Three things: more local capacity in India, cheaper inference from newer model tiers such as Luna, and better caching, which already prices cached input at one tenth of normal input on Codex. Until they arrive, the system will keep managing scarcity through limits, and users at the heavy end will keep feeling it first.