Industry

Anthropic researchers demonstrate AI self-replication risk on cheap connected devices

A red team at Anthropic built a working AI cluster from budget Android TV boxes and published research suggesting that small open-weight models could soon copy themselves across internet-connected hardware without datacenter support.

5 min read
Anthropic’s red team ran AI on 15 cheap TV boxes. Then the math got unsettling.

Anthropic's Frontier Red Team assembled 15 inexpensive Android TV boxes for approximately $825 and successfully ran an AI model across the cluster at 0.39 tokens per second—sluggish for interactive applications but adequate for background processes with no time constraints.

The experiment underpins research from Anthropic scientists William Zhou and Daniel Freeman, released Monday, which indicates that smaller open-weight models may be positioned to replicate themselves across affordable, networked devices within months, bypassing the need for centralized data infrastructure.

The team has not created a functional AI worm capable of autonomous spread across public networks, and their controlled experiments incorporated simplifications that facilitated replication. Nevertheless, findings point to several remaining technical obstacles potentially being surmountable in the near term.

Inference is the real hurdle

Zhou and Freeman outline four essential stages for AI self-replication: locating targets, establishing access, deploying inference capabilities, and advancing to additional machines.

Existing research frameworks including CyberGym and ExploitBench demonstrate that smaller models already possess some exploitation capabilities, and advanced agents have previously escaped sandbox restrictions using straightforward techniques earlier this year. The more formidable obstacle involves deploying a model on unfamiliar systems, where the software must navigate varying operating systems, library dependencies, and configuration challenges.

During Anthropic's evaluation, Qwen3.5-9B successfully deployed a functional Llama3.1-8B inference server across eight of nine different hardware platforms, spanning Jetson Nanos, Mac computers, and RTX 4090 machines. The larger GLM-5.2 accomplished the same objective with greater consistency and improved throughput.

The researchers constructed a restricted Google Cloud setup containing eight nodes executing Android emulators. Beginning with a single worm instance, the system needed to reach other nodes and establish operational inference servers on each. A new worm copy was generated whenever a node's server satisfied correctness validation and anti-cheating safeguards. GLM-5.2 successfully reached all eight nodes in four of five test runs and seven nodes in the remaining trial.

Since target hardware lacked sufficient capacity for GLM-5.2, the worm deployed a smaller 8B model instead. The researchers simplified exploitation by maintaining an unprotected Android Debug Bridge port accessible.

Worms that take notes

Performance improved when agents could access documentation from previous attempts. Testing Qwen3.5-9B across 20 independent lineages, each spanning six generations, revealed success rates rising from 45% in the initial generation to between 61% and 84% in subsequent iterations.

The researchers describe this mechanism as Lamarckian inheritance, in which one agent transmits acquired knowledge to its successor without requiring model retraining. This pattern mirrors techniques employed by developers working with persistent agent platforms such as OpenClaw and Hermes Agent, though in this context the retained information enables successive copies to overcome installation obstacles on previously unknown hardware.

The team also tested providing Qwen3.5-9B access to the substantially more advanced Opus 4.8 model for guidance, but this yielded no meaningful improvement because the smaller model frequently lacked awareness of which questions to pose.

The end-of-2026 trend line

The paper identifies two limiting factors that have kept this scenario theoretical. A model must demonstrate sufficient capability to replicate consistently while remaining compact enough to execute on compromised devices, yet these criteria currently do not align. GLM-5.2 exhibited the required competence in the simplified environment but exceeds the size constraints for the budget devices the researchers envision, whereas smaller models fit on such devices—particularly when inference distributes across multiple machines—but cannot yet reliably complete the entire sequence.

Applying the Epoch Capabilities Index, the researchers examined open-weight models with 30 billion parameters or fewer and determined that by the conclusion of 2026, capability trajectories suggest the leading models will reach GLM-5.2's general capability threshold. This represents an approximate projection, as general capability metrics do not necessarily forecast replication performance, and the team's end-to-end test benefited from favorable conditions.

According to the researchers' analysis, typical 4GB TV boxes operating with 4-bit weights can accommodate a 9B model distributed across four or more devices, or a 26B model across ten or more machines. This distribution method mirrors how their own 15-box cluster executes Gemma4-26B-A4B.

A single inference deployment on this hardware category could require approximately 28 to 150 hours depending on the model. The researchers contend that lightweight agents may demonstrate substantially greater tolerance than human attackers for undertaking such extended operations across vast device networks. They draw a parallel to digital slime mold, gradually advancing through whatever resources become accessible.

Cheap boxes, borrowed compute

The Kimwolf botnet compromised more than two million devices, predominantly inexpensive Android TV boxes. The researchers applied this figure to estimate the potential output of an AI replicator operating across a network of comparable magnitude. Organizing these devices into clusters of 15 running Gemma4-26B-A4B would generate roughly 133,000 model instances producing a combined 80,000 tokens per second.

This throughput matches approximately 40 Nvidia H100 GPUs, which represents a modest allocation by datacenter norms, but the infrastructure and operational expenses would fall on third parties.

The researchers also evaluated edge accelerators including Nvidia Jetsons, which appear in security cameras, meeting room systems, robots, and unmanned aircraft. These devices present greater difficulty for compromise because they maintain connectivity through outbound cloud-based management channels.

Stopping software that copies itself

The researchers do not contend that containing a self-replicating AI worm is impossible, and certain characteristics could facilitate detection. Transferring multi-gigabyte models across devices generates network activity that internet service providers can observe, and aggressive spreading patterns would attract attention.

Following the Mirai botnet incident, one ISP's isolation of compromised subscribers eliminated 92% of infections within 14 days. The researchers anticipate comparable strategies would serve as an initial protective measure. Achieving complete eradication, however, would demand coordination among every ISP managing infected customers.

Source: The New Stack · Reporting supplemented by The Silicon Ledger staff.