OpenAI's Agents API Unleashes Long-Running Tasks as Astra Strains Capacity

On Thursday, OpenAI made its Agents API available in public beta, granting developers access to the infrastructure powering Codex so they can deploy agents that operate continuously for days at a time. Previously, builders had to construct their own mechanisms to maintain agent operation, but the new API handles job tracking and provides execution space for agents, allowing them to function even when tasks exceed a single context window. This development arrives on the same day OpenAI halted fresh enrollments for its $200-monthly Pro plan, citing strain on infrastructure from demand surrounding GPT-6 Astra.

Thibault Sottiaux, the engineering lead responsible for Codex, stated on X that Pro subscriptions put the most strain on our systems, and noted that OpenAI was expanding capacity as fast as we can. Although the Agents API and ChatGPT Pro operate as distinct offerings with no inherent reason to compete for resources, the simultaneity of these announcements raises questions about infrastructure allocation: the company is simultaneously simplifying agent deployment for developers while restricting access to its most resource-intensive consumer offering.

https://x.com/thsottiaux/status/2098113585683808624?ref_src=twsrc%5Etfw

Agent inference adds up fast

The API employs context compression to prevent agents from halting when they approach the model's context boundary. Developers can also invoke tools selectively or distribute segments of complex jobs across parallel subagents. Work execution occurs either within OpenAI's environment or on developer-controlled systems.

As agents advance through tasks, they repeatedly query the model for subsequent steps. A job spanning multiple hours can generate substantially more inference than a standard API request, with usage accelerating further when multiple agents operate concurrently. OpenAI's own experience illustrates this pattern. In a research report released September 6, the organization disclosed that by mid-August, its research division was logging 3.1 agent-workdays for every human workday, calculated in standard eight-hour blocks. The median researcher, ranked by agent consumption, was incurring over $600 daily in inference expenses at API rates, while those at the 90th percentile surpassed $7,000.

Prior to June, OpenAI's researchers contributed more labor than their deployed agents. By mid-August, however, agents were executing three times the volume of work. While OpenAI's research teams represent an outlier scenario, these figures demonstrate the implications when agent deployment reaches scale. A single individual can rapidly generate inference volumes far exceeding what traditional headcount metrics would predict.

Friction limited compute demand

The Agents API reduces experimentation costs by excluding the orchestration layer from billing. Developers pay exclusively for the models, tools, and hosted compute their agents consume.

The tradeoff involves reduced barriers to higher inference consumption. Context compaction exemplifies this dynamic. Previously, a full context window forced developers to choose between discarding information or condensing accumulated work. The API now manages this automatically, allowing agents to persist beyond context limits. This benefits developers but also means workloads continue expanding when context windows would previously have imposed a hard stop.

Astra demand hit the ceiling

The Astra rollout provides insight into potential scaling challenges. OpenAI suspended new Pro subscriber registrations fewer than two weeks after the model's September 3 launch, attributing the pause to the strain these accounts impose on infrastructure. The Agents API incorporates its own rate restrictions and consumption tiers, so the Pro suspension does not directly impact developers leveraging the API. Nevertheless, the company is already managing capacity constraints around its newest model.

Infrastructure outweighs benchmarks now

As developers deploy additional agents and extend their runtime, consumption accelerates. A single developer might operate multiple agents simultaneously, each returning to the model repeatedly throughout execution. Consequently, headcount alone provides limited insight into actual compute utilization. Cloudflare advanced a comparable thesis this summer, contending that the systems supporting AI workloads would eventually rival the significance of the models themselves.

Source: The New Stack