How to Add Observability to Your AI Agents in 3 Lines of Code

Most teams spend weeks building custom observability for their AI agents: logging LLM calls, tracking costs, building dashboards, setting up alerts. By the time they're done, the agent codebase has doubled in size and half of it is monitoring code.

There's a better way. The Canary SDK gives you session tracking, cost monitoring, tool call analytics, error detection, and daily digests with three lines of integration code. Here's how.

Step 1: Install the SDK

npm install @heycanary/sdk
# or
pnpm add @heycanary/sdk
# or
yarn add @heycanary/sdk

Step 2: Get Your API Key

Sign up at canary.dev and grab your API key from the dashboard. Add it to your environment:

CANARY_API_KEY=ck_your_api_key_here

Step 3: Initialize the SDK

At the top of your agent code, import and initialize Canary:

import { Canary } from '@heycanary/sdk';

const canary = new Canary({ 
  apiKey: process.env.CANARY_API_KEY 
});

That's it. Now let's add session tracking.

Basic Session Tracking

Wrap your agent execution in a session. This automatically captures timing, cost, and outcome:

async function runAgent(userId: string, task: string) {
  const session = canary.startSession({
    userId,
    agentId: 'customer-support-agent',
    metadata: { task }
  });

  try {
    // Your agent code here
    const result = await executeAgentTask(task);
    
    session.end({ 
      outcome: 'success',
      result 
    });
    
    return result;
  } catch (error) {
    session.end({ 
      outcome: 'error',
      error: error.message 
    });
    throw error;
  }
}

Automatic LLM Call Tracking

Canary auto-detects calls to OpenAI, Anthropic, and other major providers if you use the SDK's LLM wrappers:

import { OpenAI } from '@heycanary/sdk/openai';

const openai = new OpenAI({ 
  apiKey: process.env.OPENAI_API_KEY,
  canary // Pass the Canary instance
});

// All calls are automatically tracked
const response = await openai.chat.completions.create({
  model: 'gpt-4',
  messages: [{ role: 'user', content: 'Hello!' }]
});

// Canary captures: prompt, response, tokens, cost, latency

Manual LLM Call Logging (For Custom Providers)

If you're using a provider we don't auto-wrap yet, log calls manually:

const llmCall = session.startLLMCall({
  model: 'claude-3-opus',
  provider: 'anthropic'
});

const response = await anthropic.messages.create({
  model: 'claude-3-opus-20240229',
  messages: [{ role: 'user', content: prompt }]
});

llmCall.end({
  promptTokens: response.usage.input_tokens,
  completionTokens: response.usage.output_tokens,
  response: response.content[0].text,
  cost: calculateCost(response.usage) // Your cost calc
});

Tool Call Tracking

Track when your agent calls external tools or APIs:

const toolCall = session.startToolCall({
  toolName: 'search_database',
  parameters: { query: 'user_id:12345' }
});

try {
  const result = await searchDatabase({ query: 'user_id:12345' });
  
  toolCall.end({ 
    success: true, 
    result,
    latency: Date.now() - toolCall.startTime
  });
} catch (error) {
  toolCall.end({ 
    success: false, 
    error: error.message 
  });
}

What You Get Automatically

With just the session and LLM tracking above, Canary gives you:

• Real-time dashboard showing active sessions, success rate, and cost per hour
• Session traces with full LLM call history, tool calls, and timing breakdowns
• Cost analytics by model, agent, and user—updated in real time
• Error detection with automatic alerts for failure spikes or cost anomalies
• Daily digest emailed each morning with yesterday's agent performance

Advanced: Custom Metrics

Need to track agent-specific metrics? Add custom data points:

session.recordMetric('documents_retrieved', 15);
session.recordMetric('user_satisfaction', 4.2);
session.recordMetric('response_relevance', 0.87);

These appear in your dashboard and can be queried via the API.

Real Example: Customer Support Agent

Here's a complete customer support agent with Canary observability:

import { Canary } from '@heycanary/sdk';
import { OpenAI } from '@heycanary/sdk/openai';

const canary = new Canary({ apiKey: process.env.CANARY_API_KEY });
const openai = new OpenAI({ 
  apiKey: process.env.OPENAI_API_KEY,
  canary 
});

export async function handleSupportTicket(ticketId: string, userMessage: string) {
  const session = canary.startSession({
    userId: ticketId,
    agentId: 'support-agent-v2',
    metadata: { ticketId }
  });

  try {
    // Retrieve context
    const contextTool = session.startToolCall({
      toolName: 'get_ticket_history',
      parameters: { ticketId }
    });
    const history = await getTicketHistory(ticketId);
    contextTool.end({ success: true, result: history });

    // Generate response
    const response = await openai.chat.completions.create({
      model: 'gpt-4-turbo',
      messages: [
        { role: 'system', content: 'You are a helpful support agent.' },
        { role: 'user', content: userMessage },
        { role: 'assistant', content: JSON.stringify(history) }
      ]
    });

    session.recordMetric('response_length', response.choices[0].message.content.length);
    session.end({ outcome: 'success' });

    return response.choices[0].message.content;
  } catch (error) {
    session.end({ outcome: 'error', error: error.message });
    throw error;
  }
}

Now every ticket gets full observability: timing, cost, tool calls, and outcomes—all visible in your Canary dashboard.

Deployment Patterns

Production: Use Environment-Based Configuration

const canary = new Canary({
  apiKey: process.env.CANARY_API_KEY,
  environment: process.env.NODE_ENV, // 'production', 'staging', 'dev'
  enableDailyDigest: process.env.NODE_ENV === 'production'
});

Development: Disable Canary Locally

const canary = new Canary({
  apiKey: process.env.CANARY_API_KEY,
  disabled: process.env.NODE_ENV === 'development' // No-op in dev
});

Cost Impact

The SDK adds ~2-5ms of overhead per LLM call (for logging and serialization). Network calls to Canary's API are async and non-blocking. Total latency impact on your agent: negligible.

Cost impact: none. Canary doesn't charge per API call or trace. Pricing is flat monthly based on tier.

Next Steps

You now have production-grade observability. Here's what to explore next:

• Set up cost budgets to alert when agents exceed spending thresholds
• Enable anomaly detection to catch behavioral drift before it breaks production
• Configure custom alerts for session duration, error rates, or tool failures
• Use the Canary API to build custom dashboards or integrate with Slack/PagerDuty

Get started with Canary for free →