Google's Gemini 4 Argon Undercuts Claude on Price, But Independent Testing Reveals Accuracy Gaps

When Google unveiled Gemini 4 Argon to a select group of testers on September 30, the company presented benchmark data showing the model ahead of Claude Opus 5.5 across 14 of 18 tests. Performance gaps included DeepSWE, where Argon scored 77.9% versus Opus's 74.2%, and GraphWalks at 256K to 1 million tokens, where Argon achieved 84.2% compared to Opus's 66.8%. The model became available to paid Gemini API developers two weeks later, on October 13.

Google acknowledged where Opus 5.5 maintains an edge, winning on four benchmarks including Terminal-bench 4.0, which evaluates agentic command-line operations, where Opus scored 66.4% versus Argon's 57.4%. The pricing advantage proved more immediate: through November 30, Argon costs $2 per million input tokens and $10 per million output tokens—half Opus 5.5's rate. Starting December 1, Argon's pricing rises to $4 and $20 respectively, matching Anthropic's model. This creates a narrow window for developers to assess whether Argon delivers superior performance or merely undercuts on cost.

To evaluate whether the model lives up to its benchmark claims, independent testing examined three demanding tasks where Google claimed advantages. Each test ran once per model through OpenRouter on both vendors' servers using default reasoning settings, with answers graded against pre-built and verified answer keys designed to measure quality and comprehensiveness.

The tests

• Bug hunt – A 3,500-line Python library containing 14 intentional logic bugs and five decoys. Each model reviewed the code and attempted fixes.
• Billing spec – A subscription billing engine specification evaluated against 80 hidden tests covering daylight saving transitions, month-end dates, and rounding. A follow-up added two currencies and 20 additional tests.
• Long context – A simulated wiki with 400 services and scattered dependencies, tested at three different sizes. Each model answered three graph questions containing planted traps.

Bug hunt

The bug hunt produced a tie. Both models identified all 14 bugs, including four that only emerge when tracing a value across three or more files. Neither model incorrectly flagged any decoys. In the follow-up test, both successfully corrected all 14 bugs without breaking any of the 93 existing tests.

Cost efficiency favored Argon. Its average review expense came to 1 cent per bug discovered, compared to Opus's 2.4 cents, calculated from total per-call costs rather than per-bug measurement. Argon would maintain a cost advantage even at its full December 1 pricing, given its lower token consumption.

• Review: Argon 53,031 input, 3,964 output, $0.15, 43.0 seconds; Opus 64,572 input, 4,053 output, $0.34, 38.9 seconds
• Fix: Argon 54,923 input, 24,292 output, $0.35, 186.6 seconds; Opus 66,465 input, 34,263 output, $0.95, 219.8 seconds

Billing specs

Accuracy diverged significantly on this task. Opus passed all 80 hidden tests, then all 100 following the currency expansion. Argon passed 11 of 80, then 20 of 100. The failure stemmed not from flawed logic but from a single early mistake: Argon's code passed an extra argument to Python's datetime.combine(), which raises an error, and most of the module depended on that helper function. When that single line was manually corrected, Argon scored 80/80 and 100/100. Official scores reflect the graded results, but the root cause was a discrete implementation error rather than fundamental reasoning problems.

While Argon remained cheaper on this test, the pricing advantage will evaporate once the promotional period ends, though Argon's reduced token usage would preserve some cost benefit.

• Build: Argon 3,907 input, 3,727 output, $0.05, 54.1 seconds; Opus 5,236 input, 7,763 output, $0.18, 63.0 seconds
• Follow-up: Argon 8,881 input, 4,673 output, $0.06, 41.5 seconds; Opus 11,807 input, 7,267 output, $0.19, 51.9 seconds

Long context

Token counting diverged between vendors. A 500K document registered as 549,483 tokens on Argon but 784,087 on Opus. A 900K document reached 1.4 million tokens on Claude's tokenizer, causing Anthropic's API to reject it. A third document of approximately 600K tokens was added as the largest size both models could process.

Both models answered two of three questions flawlessly across all sizes they processed. Performance split on the blast radius question, which requires identifying all 33 services that fail when one service goes down. Opus retrieved all 33 at 500K and 32 at 600K, with no incorrect entries. Argon retrieved 20 correct and 8 incorrect at 500K; 19 correct and 16 incorrect at 600K; and 23 correct and 2 incorrect at 900K. At 600K and 900K, Argon treated a dependency as removed after the wiki reinstated it.

Unlike the billing bug, this wasn't a small slip. Argon got the reasoning wrong across the entire answer. This is a big red flag.

Argon's speed on the long-context task masked underlying problems. The model generated answers rapidly without showing step-by-step reasoning, while Opus invested more time in reasoning first. Token usage reflected this difference: Opus consumed over 5,000 tokens, mostly for reasoning, versus Argon's 243–425. This disparity likely connects to Argon's inaccurate answers, though causation cannot be definitively established from a single run. Argon's lower cost on this task does not justify its use for similar work given its unreliability and careless execution.

• 500K: Argon 549,483 input, 264 output, $1.10, 24.2 seconds; Opus 784,087 input, 5,532 output, $3.25, 68.3 seconds
• 600K: Argon 652,492 input, 425 output, $1.31, 32.4 seconds; Opus 935,332 input, 5,158 output, $3.84, 69.1 seconds
• 900K: Argon 982,367 input, 243 output, $1.97, 41.5 seconds; Opus not run

Results

Independent testing produced findings at odds with Google's published benchmarks. Argon matched Opus on code review and repair at less than half the cost. It stumbled on the specification test due to a single faulty line of code, and it consistently missed or fabricated services when answering long-context blast-radius questions.

Assessment

These results argue against adopting Argon. Although it delivered cost savings and matched Opus's accuracy on the bug hunt, extended work with the model across other tasks revealed reliability concerns. Artificial Analysis's independent ranking places Opus 5.5 at 58, ahead of Argon's 53.

I wouldn't unquestioningly trust Argon on any task.

Google continues advancing its competitive position, but the company has not yet closed the gap with Anthropic's leading offerings.