OBSERVABILITY FOR AI AGENTS

Debug AI Agents
Like Software.

Replay any failed execution. Inspect state transitions. Fork traces and test fixes safely.

Get API Key Read the docs

Installnpm i tracepilot-openai

Installnpm i tracepilot-vercel

Trace Explorer

tr_8f2a3c1e·agent.support·2,445msFAILEDLIVE

OperationTimelineDurationStatus

Span DetailERROR

span_id: sp_e4b2f1a0

operation: tool.query_database

duration: 561ms

model: gpt-4o

tokens: 2,891 prompt → 412 completion

cost: $0.0289

Input

SELECT * FROM devices WHERE battery_charge < 10

Output

PostgresError: relation "battery_charge" does not exist
LINE 1: SELECT * FROM devices WHERE battery_charge < 10
                                    ^
HINT: Did you mean "battery_level"?

WHAT GOES WRONG

Agents fail in ways logs can't explain.

A prompt loop doesn't look like a bug in your logs. A hallucinated schema doesn't surface in Datadog. Token costs spiral without triggering alerts. These are AI-specific failure modes that require AI-specific tooling.

Prompt Loop

tr_a1b2c3d4

00:00.000agent.entry → calling openai.chat.completions

00:00.142openai.response → "I need to search for error logs"

00:00.143tool.call → search_web("production error logs db")

00:01.204tool.response → 4 results found

00:01.205agent.evaluate → calling openai.chat.completions

00:01.544openai.response → "I need to search for error logs"

00:01.545tool.call → search_web("production error logs db")

00:02.607agent.evaluate → calling openai.chat.completions

...↻ identical cycle repeated 154 times

00:47.201agent.exit → MAX_RETRIES_EXCEEDED · 154 iterations · $11.20

Schema Hallucination

tr_e5f6g7h8

00:00.000agent.entry → processing user request

00:00.891openai.response → "I'll query the devices table"

00:00.892tool.call → query_database("...WHERE battery_charge < 10")

00:01.453tool.response → ERROR: relation "battery_charge" does not exist

HINT: Did you mean "battery_level"?

00:01.454agent.retry → calling openai.chat.completions with error context

00:02.103openai.response → "I'll query the devices table"

00:02.665agent.exit → ToolExecutionFailed · same hallucinated field

Context Bloat

tr_i9j0k1l2

Step 1prompt=1,247 completion=412 total=1,659 cost=$0.016

Step 2prompt=2,891 completion=1,203 total=4,094 cost=$0.041

Step 3prompt=48,291 completion=12,041 total=60,332 cost=$0.603

Step 4prompt=91,442 completion=8,291 total=99,733 cost=$0.997

Step 5prompt=203,891 completion=44,102 total=247,993 cost=$2.480

Step 6prompt=412,291 completion=88,412 total=500,703 cost=$5.007

─────────────────────────────────────────────────

TOTAL 913,514 tokens consumed · $9.14 · single execution

"Traditional metrics notify you when your stack breaks. TracePilot allows you to freeze the execution frame, rewind model states, correct variables, and replay securely."

Learn about Time-Travel Debugging

TIME-TRAVEL DEBUGGING

Replay from any step. Edit state. Fork the timeline.

Open a failed trace, select the failing span, modify the input or model output, and replay execution forward. The original trace stays untouched — your fork runs in an isolated sandbox.

ORIGINAL TRACEtr_8f2a3c1e

agent.entry2,445ms

├─openai.chat.completions142ms

├─tool.search_web1,061ms

├─agent.evaluate1,042ms

├─openai.chat.completions339ms

├─tool.query_database561ms

└─agent.exit200ms

Failed input

SELECT * FROM devices WHERE battery_charge < 10

FORKED TRACEtr_8f2a3c1e~fork_1

forked

agent.entry2,445ms

├─openai.chat.completions142ms

├─tool.search_web1,061ms

├─agent.evaluate1,042ms

├─openai.chat.completions339ms

├─tool.query_database412msNEW

└─agent.exit89msNEW

Edited input

Edit this value →

SELECT * FROM devices WHERE battery_level < 10

Replayed successfully. Cost: $0.0031

HOW IT WORKS

Three Steps to Full Agent Control

1-Line

Drop-in Integration

Replace your existing OpenAI or Vercel AI call with our drop-in wrapper. Zero refactoring. Full observability instantly active in production.

01 — PIPELINE STAGE

Branching Graph

Trace variables, system messages, database schemas, and tool outcomes displayed on a single, high-fidelity tree timeline.

02 — PIPELINE STAGE

Time-Travel Replay

Open failed traces, hot-swap values in our interactive sandbox console, and replay the agent execution onwards instantly.

03 — PIPELINE STAGE

tracepilot-integration.ts

DROP-IN

// Before — no observability

const res = await openai.chat.completions.create({
  model: "gpt-4o",
  messages: [{ role: "user", content: prompt }]
});

// After — full tracing with TracePilot

import { traceOpenAI } from 'tracepilot-openai';

const res = await traceOpenAI(tp, openai, {
  model: "gpt-4o",
  messages: [{ role: "user", content: prompt }]
});

// → trace captured, tokens logged, errors surfaced

INTEGRATION

Two packages. One line each.

tracepilot-openai

v0.3.1

# Install

$ npm install tracepilot-openai

import { traceOpenAI } from 'tracepilot-openai';

const res = await traceOpenAI(tp, openai, {
  model: "gpt-4o",
  messages: [{ role: "user", content: prompt }]
});

tracepilot-vercel

v0.2.0

# Install

$ npm install tracepilot-vercel

import { traceVercel } from 'tracepilot-vercel';

const result = await traceVercel(tp, {
  model: "gpt-4o",
  messages,
  tools: { searchWeb, queryDatabase },
  maxSteps: 5,
});

Deploy with Vercel

TracePilot tracing, OpenAI agent, streaming UI, API key management

Deploy with Vercel View on GitHub

ARCHITECTURE

How it works.

Your Code

Your code calls an LLM or tool.

TracePilot SDK

TracePilot SDK wraps the call, captures input, output, tokens, latency, and errors.

TracePilot API

Data is sent to the TracePilot API over encrypted HTTP.

Dashboard

The dashboard renders the trace tree, waterfall, and span details in real time.

All trace data is encrypted in transit and at rest. SDK adds <2ms latency per call.

Trusted by AI teams in production

2M+

Traces processed monthly

Agent frameworks supported

<2ms

SDK overhead per call

GET STARTED

Start tracing in two minutes.

Generate an API key, install the SDK, and see your first trace in the dashboard.

Authenticate with GitHub or Google

OAuth 2.0 · No tokens stored

Generate your API key

Scoped per project · Revocable

Install SDK and see your first trace

npm install · 1-line integration

OAuth login required. No anonymous keys.

terminal

ready

$npm install tracepilot-openai

✓ added 1 package in 0.8s

export TRACEPILOT_API_KEY="tp_live_••••"

#Update your code: openai → traceOpenAI

$node agent.ts

→

Trace captured. Open dashboard to inspect.

Debug AI AgentsLike Software.