Developers

Gemini 4 Neon's coding performance triples with better harness, raising questions about model versus infrastructure

Google's latest Gemini model achieved dramatically different benchmark scores depending on which coding harness ran it, suggesting that infrastructure choices matter as much as the underlying model for AI coding agents.

3 min read
Google’s Gemini 4 Neon tripled its coding score without changing the model

Google unveiled Gemini 4 Neon on Wednesday with a striking demonstration: the model resolved 12.8% of the SWE-sweep coding benchmark when paired with the mini-SWE harness, but that figure climbed to 39.0% when the identical model operated within Antigravity.

The more-than-threefold improvement raises questions about how much coding-agent performance comes from the underlying model and how much depends on the harness…

The disparity highlights a fundamental tension in evaluating AI coding systems: distinguishing between gains attributable to model improvements versus gains from the infrastructure orchestrating the model's tools, context management, and execution flow. A separate benchmark released the day before showed similar infrastructure effects, with the same open model incurring costs that varied by up to five times depending on which harness deployed it.

Same model, different scores

Google's Gemini 4 Neon model card benchmarked three frontier models on SWE-sweep, testing each against both their native provider harnesses and a shared mini-SWE baseline. Claude Opus 5.5 improved from 3.6% with mini-SWE to 10.1% when running through Claude Code, while GPT-6.2 Sol declined from 11.6% to 8.3% under the same transition from mini-SWE to Codex.

Neon and Opus performed substantially better with their providers' harnesses, but Sol did not.

Google's testing did not isolate which specific changes drove the performance differences, nor did it verify whether these results would replicate across other benchmarks or code repositories.

The harness does the work

Antigravity's harness underwent co-training alongside Gemini, while Neon received updates to reduce superfluous file reads and strengthen context handling during extended coding tasks. The Antigravity infrastructure also enables subagents to operate in isolated Git worktrees and validate their modifications before integration.

Google only tested Neon against mini-SWE, so there's no data yet on how it performs with other harnesses.

Without separate testing of individual components, determining which improvements contributed most to Neon's SWE-sweep gains remains unclear. The absence of cross-harness evaluation leaves open the question of how Neon would perform in alternative environments.

Neon plans, Hydrogen builds

Neon joins the model-switching architecture Google introduced with Gemini 4 Hydrogen, where Neon tackles planning and complex coding challenges while delegating routine tasks to Hydrogen. The system can spawn multiple Hydrogen subagents running in parallel through the /swarm capability that debuted with Carbon.

Google has not disclosed how this multi-model orchestration affects Neon's SWE-sweep performance or what cost implications it carries for task completion.

Counting the cost per task

Neon achieved 65.2% on Terminal-Bench 4.1 and 46.2% on FrontierCode 2.0 when operating at its default high reasoning setting. It also ranked first on the Artificial Analysis Coding Agent Index with 58.6 points.

At its standard high thinking configuration, Neon costs 32% less per task than Gemini 4 Carbon based on Google's internal cost comparison. API pricing stands at $3 per million input tokens and $15 per million output tokens, with cached input priced at $0.15 per million tokens and a 1 million-token context window.

Neon is accessible via the Gemini API, Google AI Studio, and Gemini for Business. Google AI Pro and Ultra subscribers gain access through Antigravity.

Source: The New Stack · Reporting supplemented by The Silicon Ledger staff.