DeepSeek Recruits 150 Backend Engineers to Scale Its Agent Sandbox Infrastructure
The AI company is hiring heavily in systems engineering and infrastructure rather than research, focusing on the DSec platform that runs hundreds of thousands of concurrent agent environments on single clusters.
DeepSeek is moving to expand its backend engineering workforce by approximately 150 positions, with the hiring push centered on scaling DSec, its sandbox infrastructure designed to execute hundreds of thousands of concurrent agent environments across individual clusters.
The company's existing infrastructure already supports massive concurrent loads of AI agent sandboxes on single clusters. As these workloads continue to expand—along with training, evaluation, and other backend operations—DeepSeek is building out its team to manage the resulting demands.
Cui Tianyi, who joined DeepSeek in March as part of its Harness team responsible for agent infrastructure and evaluation environments, disclosed the hiring initiative on September 7 via an X post. The roughly 150 positions emphasize server-side engineering and Agent Elastic Compute roles rather than AI research work. The positions span operating systems, virtualization, networking, storage, scheduling, and control-plane services that manage these resources.
According to Cui, "DeepSeek's existing backend systems will need upgrades, maintenance, and rewrites as workloads grow." One critical system facing these scaling pressures is DeepSeek Elastic Compute, or DSec, the sandbox infrastructure built to execute agent workloads during post-training and evaluation phases.
Four sandboxes, one SDK
Running agent workloads requires more than GPU inference capacity; each agent also demands an isolated environment where it can execute code, invoke tools, modify files, and gather results.
DSec provides four distinct environment types through a unified Python SDK. Lightweight function calls route to pre-warmed containers, while Docker-compatible containers handle jobs requiring persistent state. Firecracker microVMs deliver stronger isolation when necessary, and QEMU virtual machines support workloads needing a complete guest OS.
This diversity allows the same infrastructure to manage everything from basic tool invocations to complex software-engineering tasks requiring full operating system access. The industry faces similar challenges as agents transition from experimental prototypes to deployed systems. OpenAI, for example, recently developed custom silicon to address the computational demands of agent workloads, while DeepSeek released its own agent harness as open source in August.
Lazy loading agent environments
Each sandbox requires its own environment, yet distributing complete container or VM images to every host would demand substantial storage and bandwidth while increasing startup latency. DeepSeek solves this by integrating DSec with 3FS, its distributed filesystem originally created for AI infrastructure, and storing container base images and filesystem commits as read-only layers backed by 3FS.
Metadata remains local while underlying data blocks load only when accessed. MicroVMs employ the same approach, sharing read-only base layers through 3FS while individual sandbox modifications stay in local copy-on-write layers.
According to DeepSeek, "DSec reduces duplicate page-cache usage across virtualized environments and reclaims memory to allow safe overcommitment, while changes to the container runtime cut the CPU overhead of each sandbox." The team also addressed spinlock contention within the container runtime. While negligible at smaller scales, this contention becomes a bottleneck at scale, constraining how densely these environments can be packed onto individual hosts.
When replay breaks training
Reinforcement learning and other post-training workloads often run large numbers of agent rollouts simultaneously, and jobs may be interrupted as compute resources shift. Restarting from scratch discards prior work, but resuming is not straightforward since replaying earlier commands might produce different outcomes.
Some commands may have modified files or changed the environment, so re-executing them could alter results or corrupt the training state. DSec addresses this through a globally ordered trajectory log capturing commands and their results.
When a rollout resumes, DSec fast-forwards through completed work using logged results rather than re-executing commands. This approach lowers the expense of interruptions across thousands of training and evaluation runs, while the same logs maintain a record of sandbox changes and enable earlier sessions to be replayed.
Engineers, not researchers, wanted
The roughly 150 positions span DeepSeek's backend operations, encompassing lower-level systems work supporting Agent Elastic Compute as well as services powering its models and agents.
In June, DeepSeek indicated plans to at least double departmental headcount, but this hiring round prioritizes systems infrastructure beneath its models over model development itself. DSec represents a key part of this effort, with hundreds of thousands of sandboxes operating concurrently and creating strain across job scheduling, interruption recovery, and related systems.
Source: The New Stack