The 88% Problem: Why Most AI Pilots Stall Before Production

An autonomous AI system begins looping through the same function call at 3 a.m., and no one is watching. This scenario illustrates a broader crisis in enterprise AI deployment: according to research conducted jointly by IDC and Lenovo, 88% of artificial intelligence proof-of-concept initiatives fail to advance into production environments. The culprits are familiar but stubborn—vague project goals, data that hasn't been properly prepared, and insufficient internal knowledge of agentic AI systems.

The gap between concept and operation often stems from organizational fragmentation. The team responsible for building and deploying AI agents, frequently an external partner, hands off the system to an operations group that may lack deep familiarity with how these systems behave. When that operations team attempts to run the agentic system independently, problems emerge. A single agent that strays beyond acceptable parameters can trigger cascading failures and serious business damage.

Serhii Semenchenko, CTO of Dedicatted, a Toronto-based AWS Premier Tier Partner specializing in Agentic AI, describes the fundamental disconnect between demonstration environments and real-world deployment. "The demo is a clean environment where we have clean, prepared data. No edge cases, so everything is defined — input and output." In contrast, production systems encounter unpredictable data, edge cases, and real-world complexity. To bridge this gap, Semenchenko argues that every step in a production agentic AI system must include logging, tracing, monitoring, circuit breakers, and validation checks—infrastructure designed to prepare for failure rather than assume success.

Building Production-Ready Agentic Systems

Semenchenko outlines a basic production checklist for agentic AI: each agent must have a timeout mechanism, and every output generated by an agent should be logged for traceability. Beyond these fundamentals, agentic AI systems should possess the ability to self-assess during operation, determine whether to proceed after a particular event, or defer tasks for later execution.

He draws a parallel between agentic AI architecture and microservices design. Rather than granting agents permanent, broad access credentials, each agent should operate with a specific set of permissions tied to short-lived keys that grant access only to the particular tools and services it requires. This approach—treating each agent as an isolated service—prevents unintended tool invocation, reduces the risk of data leaks, and guards against prompt injection attacks.

Multi-agent orchestration introduces another layer of complexity. Semenchenko describes a typical workflow: a router component receives input from the user, system, and platform, then routes that input to specialized agents—one handling documentation, another processing video, a third managing audio streams, and so forth. Each specialist agent possesses its own set of tools, which may include third-party APIs. A final composition agent then assembles outputs from all previous agents and tools into a unified response. Critically, if the composition step fails, the agent does not fabricate missing data; instead, it either retries or explicitly marks the task as failed. This creates a clear chain of responsibility and state management.

Memory poisoning—corruption of an agent's stored information—represents another production hazard. Semenchenko introduces the concept of guardrails: all documents and data flowing through the system undergo deterministic validation to confirm data integrity and trace any inconsistencies back to their source in records, databases, or duplicate entries.

Lessons From Real Deployments

Dedicatted constructed a multi-agent system for CASSEN Testing Laboratories, a Canadian chemical and environmental testing firm struggling with multi-day report turnaround times caused by manual regulatory testing and lengthy processing workflows. Although the project began with clear input and output specifications, human-centered edge cases surfaced during operation. The chemistry team held specific preferences about how reports and chemical compound findings should be formatted—preferences that were more idiosyncratic than domain-standard. When the system entered production, these stylistic expectations became critical. Semenchenko explains the resolution: "We adjusted our flow, adjusted the agent that composed this report. So, everything in the final, after these adjustments, reports look the same as if chemists had manually prepared them."

Infrastructure migration introduces its own set of production challenges. Zivian Health, a U.S.-based compliance platform serving organizations that manage nurse practitioner and physician assistant workforces across multiple states, underwent a re-platforming from AWS to Azure while operating under regulatory constraints. AWS and Azure employ fundamentally different access control mechanisms, making least-privilege permission models essential. Semenchenko notes that "everything should be mapped, adjusted, and validated first." The team also had to contend with models unavailable on both platforms during the transition. Their solution involved cloud-agnostic monitoring and tracing tools, followed by a comprehensive validation suite covering all use cases and requirements. After migration, they verified that each agent, workflow, and user interaction maintained the same success rate, latency, and token cost.

The 3 a.m. Misbehavior Problem

The scenario that opened this discussion—an agent misbehaving in the middle of the night—occurs more often than organizations expect. Semenchenko describes a typical failure mode: "An agent could use different tools, and at some moment it could decide that the output of one tool is not enough, and will try to call it, like, 2, 3, 5 more times. Every single time it will decide that it's not enough; it should call again, maybe some new data will appear." Advanced monitoring systems detect such anomalies and flag them as errors sent to the observability platform. The response depends on the operational model: if the agentic AI partner operates the system, they receive alerts and can respond immediately through dashboards and active monitoring. If the client's internal team operates the system, they receive error notifications and must consult runbooks provided by the partner to resolve the issue themselves.

The advantage of engaging a partner to operate agentic AI systems is continuous surveillance, immediate response to unexpected events, and detailed reporting. Partners assume full recovery responsibility and can immediately halt an agent or revoke its access without waiting for business hours—reducing the burden on in-house teams and preventing further damage.

The Commercial and Organizational Dimension

Denys Yermakov, CEO of Dedicatted, observes that agent failures often go unnoticed on the business side until they accumulate. Preventing such failures requires extensive monitoring infrastructure, regression testing, multiple test scenarios, and varied agent behavior validation. How quickly a company can ship fixes matters as much as detection capability. Infrastructure outages or faulty deployments carry higher costs in agentic AI systems than in other platforms, Yermakov warns.

When evaluating an agentic AI partner, Yermakov recommends starting with a small proof-of-concept or limited demonstration to assess how the partner delivers results. The critical distinction lies between a reseller offering generic solutions and a genuine service provider. "Because this part has to be delivered on real data, with a real workflow and with real challenges you face in your business. Not based on some mock data," Yermakov explains. A partner who surfaces challenges during the demonstration phase—rather than smoothing over them with scripted scenarios—signals genuine expertise. "If the [agentic AI] partner comes with the right questions and approach, the right partner is here," he adds.

Change management and long-term flexibility matter enormously. Organizations should clarify with their partner how the solution will adapt to future requests, who owns the code and platform, and how incident response and maintenance will be handled once the partnership ends. These questions determine whether the organization can eventually operate the system independently.

Data Security and Sensitive Information

A Modern Data Company survey found that 75.9% of enterprise data leaders and practitioners across 66 countries identify data quality and trust as barriers preventing AI systems from reaching production. The concern intensifies when sensitive data must be shared with external partners. Yermakov emphasizes the business imperative: "It's so important that your data stay within your perimeter and not leave your secure perimeter, which you already have in your platform." Traditional enterprises with large-scale on-premises infrastructure face particular risk, as some AI providers may use client data to train public models.

Semenchenko describes how Dedicatted addressed this challenge in healthcare projects where personally identifiable information tied to patient health is non-negotiable. The constraint was absolute: "The problem was that this PII shouldn't be stored anywhere." Yet agents needed access to this data to extract or process outputs. The solution involved an additional processing layer that strips all personally identifiable information from agent outputs before data reaches the observability platform. This ensures that logs and monitoring systems contain no sensitive health information.

Build Versus Partner: The Staffing Question

Yermakov emphasizes how nascent agentic AI remains as an enterprise technology. Many organizations remain uncertain about foundational questions: how to architect the platform, what it should look like, what benefits it delivers, and what it costs per user. Partners bring experience from shipping systems after encountering and solving real obstacles. They understand how to construct platforms efficiently with predictable outcomes. Yermakov stresses that "It's quite important to build and architect all the platforms from the very beginning in the very right way, to not redo what you've implemented into the business processes."

The operational reality of platform maintenance demands flexibility and adaptability. Yermakov considers it essential to avoid early failures in every agentic AI initiative. "It's so important to build your team, to build your knowledge in-house," he says, pointing toward eventual platform maturity and independence.

When deciding between internal development and outsourced delivery, Yermakov recommends examining the decision from two angles: the business process side—how the company actually operates—and the technical challenges it faces. The path to success, he argues, requires staffing that combines both perspectives: "Always the team effort of the in-house resources and the outsourcing resources." He cautions against outsourcing everything, as external partners cannot develop the deep business understanding that in-house teams possess. Subject matter experts and partners must engage from the project's inception to enable effective knowledge transfer. The in-house team should own and protect the business process, while the partner designs and operates the core runtime.