When AI Agents Pass Tests but Fail Users: Why Observability Gaps Hide Silent Failures

Your machine learning agent sends back a 200 status code, clears its faithfulness validation, yet the user receives an inaccurate response. The real explanation sits in the execution trace, not in the code diff.

A code change is not proof of what happened. It is merely a description of intent.

Tests pass. Code review completes. The deployment goes live. Then a user reports slowness, incorrect answers, or both. You examine the code change. Your debugging tool identifies the modified function and suggests a plausible explanation.

It appears reasonable. It might be wrong.

This represents the observability problem that AI systems create. According to Dynatrace's 2026 State of SRE and Platform Engineering report, which surveyed 919 enterprise leaders worldwide, 77% of platform engineering teams have integrated observability into at least some services. However, only 40% have achieved full integration across all deployments. This gap was tolerable when systems behaved predictably. With AI agents, it becomes a serious risk.

A conventional service fails loudly… An AI agent fails quietly. It returns a 200. It passes faithfulness checks. And the customer still gets the wrong answer.

Traditional services announce problems visibly: HTTP 500 errors, response time spikes, unresponsive dependencies. AI agents fail silently. They deliver a 200 response. They satisfy faithfulness validation. Yet the customer receives incorrect information.

You cannot set an alert for "incorrect." You need actual data from the operating system—and for AI systems, this means more than request logs and error metrics.

Locate the request first

Imagine running a support chatbot on product documentation. A user asks about configuring export in version 2026.3. Your team used an AI coding assistant to rewrite the documentation retrieval logic. Continuous integration passed. Existing evaluations passed.

Following deployment, responses become slower. Some responses reference older product versions.

Begin with a single problematic execution. You need its software version, retrieval settings, and feature flag configuration—these must be stored as root span attributes set when the span begins, not reconstructed later from deployment records. Then place that execution alongside a comparable request from before the change.

For an AI agent, this means the sequence of operations: each model invocation and tool invocation, in sequence, including their inputs and outputs. A distributed trace captures these as spans and connects them across service boundaries through context propagation.

Here is one execution, condensed, with its evaluation attached separately.

# Illustrative pseudotelemetry, not a captured incident.

# Names, IDs, timings, and labels are invented, not a standard schema.

# Selected spans shown in execution order; other work is omitted.

trace: example-run-a | session: example-session-7 | release: 2026.9.2

requested.product_version: "2026.3"

agent.run                                  12.4s

  model.choose_tool                         1.0s

  tool.search_docs                          0.9s

    args: {query: "configure export", product_version: null}

  tool.search_docs                          0.8s

    args: {query: "configure export", product_version: null}

  tool.search_docs                          0.9s

    args: {query: "configure export", product_version: null}

    returned.doc_versions: ["2024.1", "2024.1", "2023.9"]

  model.generate_answer                     8.1s

linked_evaluation:

  trace: example-run-a

  faithfulness: pass

  requested_version_answered: fail

Two issues demand investigation. The redundant searches. The null version parameter.

Three identical searches consume 2.6 seconds. The trace reveals the symptom. It does not reveal the cause.

However, observe what lies between them. Nothing. A single model.choose_tool span at the beginning, and no model invocation between the second search and the third. The model did not request those retries. Something else did: the framework that executes tools, manages retries, and maintains state. A model.choose_tool span between each search would indicate the opposite: a model repeatedly requesting the same tool, which points to a prompt or tool-description issue. Identical symptom, different location to investigate.

This does not necessarily mean the retries are incorrect. Examine the retry logic, then examine the tool results. A 200 response from a search service can include zero results, or results below your relevance threshold, and retrying in those cases is justified.

The answer generation consumes the larger portion, at 8.1 seconds. Compare its input token count and duration against other similar executions. If the framework appended all three result sets into the context, the retries enlarged that input, and you paid for them twice: once in latency and once in tokens. Examine downstream services and traffic patterns too before attributing the slowdown to the release.

How do you bring this analysis into your code editor? Narrow the question. Provide your debugging assistant with the service name, the software version, the time range, and the trace identifiers. Have it align the modified code path with the service calls visible in the affected trace. Then distinguish what the evidence actually demonstrates from what it is inferring.

This same process debugs a payment service making three identical database queries. You do not need an AI agent to apply it.

A grounded answer can still fail

Now examine the answer itself.

In this scenario, it faithfully reproduces the retrieved documentation. Faithfulness passes, or groundedness, depending on the terminology your evaluation system uses.

The customer still receives instructions for the incorrect version.

Whether you label it faithfulness or groundedness, this metric only confirms whether the answer is backed by the sources you provided. It reveals nothing about whether those were the appropriate sources.

The natural next step is a retrieval evaluator. It still will not solve this. Such evaluators measure whether the retrieved context answers the question, and the 2024.1 export instructions do answer the question about configuring export. They simply do not apply to the version requested. Those documents answer the query. They do not satisfy the version the customer asked about. Answering is not the same as being correct.

Those documents are relevant to the query. They are not valid for the version the customer requested. Relevance is not validity.

This is not a generation problem. It is a retrieval precondition that was never verified, and the null parameter reveals it: the requested version never made it to the retrieval function. Verify this before modifying the prompt or the model.

Most of this can be tested with standard programming techniques. Supply the test fixtures with documents that include version information, then verify the retrieval directly, without involving the model:

def test_lookup_filters_to_requested_version(docs_fixture):

    hits = search_docs(query="configure export", product_version="2026.3")

    assert hits, "no hits for a version that has docs"

    assert {h.product_version for h in hits} == {"2026.3"}

This is deterministic, inexpensive, and belongs in continuous integration. Then assess the answer separately, which is the part you cannot verify with assertions: does it provide usable 2026.3 instructions, or does it acknowledge that the available documentation cannot support one? Two separate tests, because they fail for different reasons and you want to know which one broke.

The assertion will not catch every incorrect answer. It will catch this missing constraint every time, which exceeds what a human evaluator scoring helpfulness on a scale will accomplish.

This is why evaluation requires stored context. Record the prompt version, model identifier, retrieval settings, and document identifiers and versions alongside the software version. Preserve sufficient permitted information to reconstruct the answer later, with sensitive data removed before sharing.

Connect results using trace and span identifiers. If evaluation occurs after the span finishes, save a separate linked result. Do not attempt to write attributes to a completed span: the OpenTelemetry tracing specification states that implementations should disregard updates after End.

Semantic conventions for generative AI are still developing, and different instrumentation libraries describe similar concepts using different attribute labels.

Make the failure part of the next release check

Once you have confirmed the causes? Test each correction against the behavior it should change.

For the redundant searches, create a regression test that reproduces the repetition without preventing legitimate retries. Do not require one specific tool sequence when multiple orderings accomplish the task successfully; a trajectory test that mandates a single path will fail on every valid refactoring.

For the version mismatch, restore the version filter. Include these scenarios: the current version, an older supported version the customer explicitly names, documentation that does not apply, and situations where no supportable answer exists. Run the answer evaluations multiple times when output differs, because a single successful run is not conclusive.

Use code for anything you can verify with assertions. Use a model-based evaluator for answer quality, and validate that evaluator against examples that people have reviewed. An unvalidated evaluator is another model you are accepting without verification.

To run evaluation against real production traffic instead of test data, you will need a mechanism to select spans already in your system, evaluate them with a judge model, and attach each result to the original trace—the linked-result approach described above, not a write to a closed span. Whatever system you select, version the evaluation model. A scoring change that appears to be a product improvement might not be one.

After the release, measure latency and task completion on equivalent requests, and display tool-call counts and token counts on the same view. Examine them together, or they will mislead you. Tool calls decreasing from three to one can indicate the fix is working, but it can also indicate a retrieval step you accidentally removed. Fewer output tokens appear to be a cost improvement, and they also appear to be an answer that quietly omitted step four.

Bring one debugging question

If you would not know where to begin the investigation, your instrumentation is incomplete.

For a traditional service, that is the request path and service latencies. For an AI system, add what it retrieved, what it generated, and how you will determine whether that was the correct result.

Dynatrace is sponsoring WeAreDevelopers World Congress Americas, September 23-25, 2026, in San José. Bring a debugging question from an AI-assisted release or from an AI system you are building, and we will work through it.

Source: The New Stack