Datadog's Propolis bet: AI agents writing tests to keep pace with AI-written code
As coding agents accelerate software delivery, testing has become the new bottleneck. Datadog's acquisition of Propolis aims to let AI agents write and execute tests from natural language, potentially unblocking the development pipeline.
The speed at which coding agents produce software now outpaces the ability of human testers to validate it. Datadog acquired Propolis this year to address this imbalance, enabling agents to generate and execute tests based on goal-oriented specifications derived from plain-language instructions.
The shift toward agentic coding has accelerated dramatically over the past year. Teams across the industry have adopted AI-driven code generation as standard practice. Anthropic's engineering staff, for instance, now ships eight times more code than previously, with some individual engineers submitting thousands of pull requests monthly.
Yet increased code velocity does not automatically compress the entire software development lifecycle. The constraint has migrated from code authorship to validation. Once agents produce new code, it requires testing—traditionally a labor-intensive process requiring humans to identify which user interactions warrant coverage, write test scripts, and maintain them as application behavior evolves.
The industry has made code generation accessible while simultaneously placing greater demands on human testers. Addressing this imbalance is essential to realizing meaningful reductions in development timelines.
Propolis brings a different approach to test generation. Rather than relying on hand-coded test scripts, the system allows agents to explore applications independently, trace user interactions, and produce tests from conversational prompts. When combined with Datadog's observability capabilities—its logging and tracing infrastructure—the resulting platform can validate code while gathering the telemetry needed to assess system behavior, avoiding the brittleness inherent in traditional scripted testing.
Datadog will host a discussion on Wednesday, November 4, at 9 a.m. Pacific / 12 p.m. Eastern with Ruxanda Lueck, a group product manager focused on goal-based testing of AI features, and Matthew White, who co-founded Propolis and served as its CTO. The conversation will explore how much test authorship teams can delegate to agents today and whether agentic testing has reached a comparable inflection point to recent breakthroughs in code generation.
Additional topics will include whether natural-language test specifications lower the barrier to entry for non-specialists to contribute to testing efforts, and the timeline for agents to read and validate agent-generated code without human intervention.
Agentic code generation has introduced friction at multiple stages of the development pipeline that no single solution can resolve independently. However, if testing acceleration can match code generation speed, teams will remove a critical barrier between conception and deployed software.
What you'll learn
- How agents can test user flows that no one thought of
- How to write plain-language testing prompts
- How goal-based testing persists even when user flows change
- How to test AI features that return probabilistic, instead of deterministic, outputs
- Where scripted tests still have their place