How Anthropic Tests Claude Duet's Judgment on What Matters Most

Anthropic released Claude Duet on October 21, an AI assistant that scans connected applications and executes work in the background while determining which items warrant user attention without being prompted. The frontier AI lab, which has IPO ambitions, disclosed to The Silicon Ledger how it validates whether Duet correctly identifies priorities.

The challenge runs deeper than engineering: Anthropic has built internal evaluation systems to measure how well Duet ranks competing tasks, yet the company acknowledges that determining what is truly urgent remains fundamentally subjective. When an agent operates across email, calendars, and business software, the stakes of misjudging priorities grow considerably. Duet can stay within its authorized permissions while still getting the ranking wrong.

Running on Claude Opus 5.5, Duet can prepare work and surface items it believes require attention without awaiting explicit direction. The company has been validating this capability through internal testing with its own staff, observing how the agent navigates competing demands in real working conditions.

Testing Duet's judgment

An Anthropic representative explained the evaluation approach: "After using Duet for our day-to-day work at Anthropic, we calibrated graders that assess correctness in task prioritization, and we run them offline against cases representative of internal use and early external testing."

These evaluation systems measure whether Duet can separate tasks deserving immediate action from those that can be deferred, though Anthropic recognizes the inherent difficulty in making such judgments. The company stated: "Urgency is an inherently subjective call, so we found building an eval for it imperfect but quite helpful."

Duet can identify certain obvious signals—approaching deadlines or emails awaiting responses—but struggles with determining which task should take precedence. Anthropic acknowledges that the agent sometimes misinterprets context or misgauges urgency. The company's launch materials and public guidance do not disclose how frequently these errors occur.

Testing Duet's safeguards

Two days after launch, on October 23, Anthropic published details about Duet's safety, privacy, and security measures. Karan Ahuja, who leads the Duet team, referenced this documentation while responding to concerns raised through various channels.

https://x.com/ahuja_karan/status/2113768265562620109?ref_src=twsrc%5Etfw

Anthropic has isolated the component that performs tasks from the component that validates whether those tasks align with user intent. According to the company representative: "A separate, automated check runs alongside each action, built on the auto mode classifier from Claude Code and extended for Claude Duet. It assesses whether the action matches your intent."

Because this verification operates independently from the model executing the task, Anthropic contends that hidden instructions embedded in web content cannot directly influence the classifier. However, the company's public materials do not reveal how consistently the separate check identifies actions outside the user's scope. Academic research on related systems shows the difficulty of this enforcement: studies found that removing prompt sections defining authorized versus prohibited actions in Claude Code tests led to increased out-of-scope behavior, though that research did not evaluate Duet's separate classifier specifically.

Limiting prompt injection

The architectural separation aims to mitigate prompt injection attacks, in which malicious instructions hidden in emails, documents, or websites attempt to redirect an agent's behavior. Researchers have already investigated this vulnerability by planting concealed messages for rogue agents and documenting their responses. Anthropic's guidance still advises users to review drafts and approval requests, acknowledging that Duet may encounter prompt injection attempts.

Duet accesses only applications already connected to a user's Claude account. Removing an app connection blocks future access, but information Duet has stored in memory persists until the user clears it. Anthropic states that Duet never obtains passwords, browser cookies, or authentication tokens, and cannot store them in memory.

The agent is designed to request approval before executing certain actions such as payments or deletions. Sharing sensitive health or financial data requires consent, though an explicit user request to share such information can constitute that consent. Trading securities, transferring funds between accounts, and accessing another person's account remain completely prohibited.

Summaries, not surveillance

Users can track Duet's completed work and ongoing tasks through a task list, with individual pages for each job. Users can also provide new instructions while the agent is operating.

Visibility into background operations remains limited. During the research preview phase, users cannot observe Duet working in its cloud browser or assume manual control. Users can review task activity and approve actions, but cannot examine each browser step as it unfolds. The launch and support materials do not describe a mechanism for reversing completed actions across connected applications.

What background work costs

Duet draws from the same usage allowance as standard Claude conversations, meaning background work counts against a subscriber's plan. The consumption varies based on task complexity.

An Anthropic representative noted: "Larger tasks, like building a financial model or a deck, will require more model turns and background work."

Even routine checks across connected applications consume against the five-hour session and weekly limits. These checks run every few hours rather than continuously, partly to preserve a subscriber's allowance. This approach carries a tradeoff: urgent items arriving between checks could go undetected until the next scan. More frequent checking would consume more resources, while less frequent checking would delay alerts. The company's materials do not quantify what portion of typical subscriber usage goes to background activity.

Duet's research preview is available through a waitlist to U.S. Pro and Max subscribers. Anthropic staff have used it to process task lists while away and discuss workload priorities during walks, though the company acknowledges the agent performs less reliably on consumer tasks such as ticket booking.

Anthropic intends to continue refining the product as testing progresses.