Debug a failed AI agent run with execution traces

An agent task fails. The final response says “something went wrong,” but the useful question is more specific: which step stopped the run, and what evidence do we have?

A good execution trace turns that question into a sequence you can inspect. It does not replace instrumentation or explain every hidden decision a model made. It records the events your runtime reports, so an operator can connect an outcome to the steps before it.

The support workflow below is a fictional example, not a customer case study.

Start with one operational question

Imagine a support agent that reads a ticket, retrieves account information, and drafts a reply. Today it retrieves the ticket but never produces the draft. Before collecting every prompt and response, decide what you need to distinguish:

• Did the account lookup fail, time out, or return incomplete data?
• Did the agent attempt the drafting step?
• Was this one failed run or a recurring problem?

These questions suggest a small set of useful events. Record the boundaries around the ticket lookup, account lookup, and draft generation. Use meaningful names, outcome statuses, and timestamps. A long payload is not automatically useful evidence; a clear error category and correlation ID often help more.

Give the run a stable identity

Robots Center accepts traces through POST /api/v1/traces. Use a credential with traces:write and give the run an external_trace_id that your runtime can keep across retries. Treat that ID as unique to this run within the workspace, not a label to reuse for every support ticket.

Here is an illustrative start payload. The identifiers are synthetic and contain no customer information:

{
  "name": "Support reply",
  "external_trace_id": "support-demo-run-42",
  "status": "running",
  "events": [
    {
      "source_event_id": "support-demo-run-42-ticket",
      "event_type": "custom",
      "name": "Ticket retrieved",
      "status": "ok"
    }
  ]
}

When the run ends, send another trace request with the same external ID and its terminal status. For example:

{
  "name": "Support reply",
  "external_trace_id": "support-demo-run-42",
  "status": "error",
  "events": [
    {
      "source_event_id": "support-demo-run-42-account",
      "event_type": "custom",
      "name": "Account lookup failed",
      "status": "error",
      "error_payload": {"category": "upstream_timeout"}
    }
  ]
}

Stable source_event_id values protect individual events from duplicate insertion when you retry a request. Do not keep resending old events without those identifiers. Starting and finalizing a trace is different from appending a new, unrelated run.

The trace API reference is the source of truth for request fields and limits. Read access is separate: traces:write does not imply traces:read, so an administrator must explicitly grant the latter to a machine credential that needs to fetch traces.

Follow the evidence, not the last message

In the operator console, locate the run using its external ID. Check the recorded status and follow the timeline from the last successful event to the failure.

In our example, “Ticket retrieved” followed by “Account lookup failed” narrows the investigation. It does not prove whether the upstream service was down or the agent used an unrealistic timeout. Compare the event timing and the integration's own logs before choosing a fix.

If drafting has no event, say “no drafting event was reported”, not “the agent definitely never tried to draft.” Missing instrumentation and missing execution can look identical. The observability console guide lists the trace, explorer, and failure-group surfaces for investigating further.

Compare related failures

Use available agent, environment, and workflow-target filters to narrow the affected runs. Failure groups can bring recurring errors together, giving the team one place to record investigation context rather than treating every timeout as a separate incident.

Keep the first follow-up small. Check whether the failures share a target or time window. Record what is known, what is a hypothesis, and who owns the next verification. Do not turn a plausible pattern into a claim that the root cause is already established.

Replay deliberately, then add a regression case

A replay can help compare a proposed change with the original run, but it is not a risk-free recording viewer. Supported replay targets can reexecute HTTP or connector actions and create real side effects. Use a controlled target, reviewed input, and the required target binding described in the replay reference.

Once the team understands the failure, turn it into a focused evaluation case: “when account lookup times out, return a bounded failure instead of inventing account details.” Review an incident-generated draft before treating it as a lasting regression test. The evaluation review guide explains the review surface.

Useful observability ends with a better next decision: a narrower investigation, a verified fix, or a regression case that catches the same failure again.

Start with one workflow and enough instrumentation to answer its most important failure question. Expand from evidence, not from the assumption that collecting everything will make the answer obvious. For the broader workflow, see AI agent observability.