OpenAI's Brockman argues AI agents should use computers like humans, not custom integrations

During an appearance on the a16z podcast, Greg Brockman, president and co-founder of OpenAI, argued that the technology sector has spent considerable effort "retooling the world" to accommodate AI agents. He suggested that computer use capabilities could provide a more straightforward alternative to this approach.

For Brockman, the guiding principle should be straightforward: "We really should be one AI that's unified, that makes it so easy and smooth for you to be less engaging with the computer and less wrapping yourself around the computer."

Despite this vision, other organizations in the AI space continue developing connector infrastructure to support agent workflows.

Rethinking the connector-heavy approach

MCP servers, command-line interfaces, APIs, and similar agent integrations enable AI systems to access a broader range of tools, data sources, and applications. However, this expansive connectivity carries costs, as Brockman suggested during his conversation with a16z hosts Ben Horowitz and Erik Torenberg.

What if it's more behaving like a human? Can it just use a computer?

Greg Brockman

Brockman elaborated on the current state of integrations: "People have been building these MCP servers and these CLIs and just really sort of taking the world of software and making it accessible in this almost stilted way that is not really meant for humans. It's like we're kind of retooling the world."

The prevalence of these connectors stems from practical necessity. "For agentic use cases, it really comes down to the tools," Brockman explained, accounting for why MCP servers and similar solutions have proliferated. If AI models could learn to operate computers as humans do, developers might no longer need to construct specialized integrations for each application.

Computer use has been part of OpenAI's strategy since 2015

According to Brockman, this concept emerged during OpenAI's early years. The team established a three-step strategic plan at an offsite gathering in November 2015, which the organization largely followed over the subsequent decade.

During that same period, OpenAI explored the possibility of applying reinforcement learning directly to computer interfaces. Brockman recalled the discussion: "We also talked about, what if we could do reinforcement learning where the environment is screen pixels, keyboard, mouse, right? Same interface as a human."

From Brockman's perspective, this capability could unlock the computer for virtually any task. He emphasized that computer use for agents could enable the broader industry to advance AI capabilities without requiring custom integrations at every stage. Rather than constructing and maintaining numerous specialized connectors, an agent could leverage the standard computer interface that humans use.

"There's so much software that you don't even think about," he told the a16z hosts. "How much of your life is clicking around menus and typing things into a spreadsheet and things like that? None of that is what we should be doing."

Brockman pointed to computer use capabilities as one reason he considers GPT-6 Astra, OpenAI's latest flagship model, "pretty reasonable to call" AGI.

Structured integrations remain central to industry practice

Despite Brockman's optimism about computer use, much of the industry continues prioritizing structured integrations. Several organizations are actively expanding their connector offerings.

Amazon Web Services recently unveiled a managed consent portal, a managed web experience, and session binding endpoint for AgentCore Gateway, a feature of Amazon Bedrock AgentCore designed to link agents with external tools and services. AWS published a comprehensive guide to the capability on Monday.

Using the consent portal, administrators can designate GitHub and Slack as gateway targets and distribute the portal URL to developers. Those developers can then access the URL, authenticate through their corporate identity provider, and establish connections to GitHub and Slack independently.

At OpenAI itself, plugins continue to play a significant role. Though OpenAI Codex launched in the browser via a new Chrome extension in May, enabling agents to function within active browser sessions and authenticated workflows spanning multiple tabs, plugins remained the preferred integration method. This approach allowed Codex to interact directly with services including Slack, Gmail, and GitHub without requiring manual navigation of their user interfaces.

Brockman told the a16z hosts that simplicity should serve as the industry's guiding principle. Dependable computer use could represent one pathway toward achieving this goal. For the present, however, connectors maintain their position in the technology landscape.