AI Observer

Unified local observability for AI coding assistants

AI Observer is a self-hosted, single-binary, OpenTelemetry-compatible observability backend designed specifically for monitoring local AI coding tools like Claude Code, Gemini CLI, OpenAI Codex CLI, GitHub Copilot, and OpenCode.

Track token usage, costs, API latency, error rates, and session activity across all your AI coding assistants in one unified dashboard—with real-time updates and zero external dependencies.

Why AI Observer?

AI coding assistants are becoming essential development tools, but understanding their behavior and costs remains a challenge:

• Visibility: See exactly how your AI tools are performing across sessions
• Cost tracking: Monitor token usage and API calls to understand spending
• Debugging: Trace errors and slow responses back to specific interactions
• Privacy: Keep your telemetry data local—no third-party services required

Features

• Multi-tool support — Works with Claude Code, Gemini CLI, OpenAI Codex CLI, GitHub Copilot, and OpenCode
• Real-time dashboard — Live updates via WebSocket as telemetry arrives
• Persistent dashboards — Drag-and-drop dashboard builder with saved widgets and multiple widget types
• File watcher mode — Watch Claude, Codex, and Gemini session files in real-time, no OTLP configuration needed
• Historical import — Import past Claude, Codex, and Gemini sessions from local JSONL/JSON files with cost calculation
• Cost tracking — Embedded pricing data across Claude, Codex/OpenAI, Gemini, and GitHub Copilot model aliases, plus native OpenCode plugin cost metrics
• Fast analytics — DuckDB-powered storage for instant queries on large datasets
• Single binary — One ~54MB executable with embedded frontend—no external dependencies
• Multi-arch Docker — Ready-to-run ~97MB images for linux/amd64 and linux/arm64
• OTLP-native — Standard OpenTelemetry Protocol ingestion (HTTP/JSON and HTTP/Protobuf)

Documentation

• Import Command — Import historical session data from local AI tool files
• Export Command — Export telemetry data to Parquet files for archiving and sharing
• Watch Command — Watch local session files and import incrementally in real-time
• Pricing System — Cost calculation for Claude, Codex/OpenAI, Gemini, and GitHub Copilot models; OpenCode costs are ingested from the OTEL plugin

Screenshots

Dashboard

Metrics View

Logs View

Traces View

Quick Start

Using Docker (Recommended)

docker run -d \
  -p 8080:8080 \
  -p 4318:4318 \
  -v ai-observer-data:/app/data \
  --name ai-observer \
  tobilg/ai-observer:latest

Dashboard: http://localhost:8080

Using a local directory for data persistence:

# Create a local data directory
mkdir -p ./ai-observer-data

# Run with local volume mount
docker run -d \
  -p 8080:8080 \
  -p 4318:4318 \
  -v $(pwd)/ai-observer-data:/app/data \
  -e AI_OBSERVER_DATABASE_PATH=/app/data/ai-observer.duckdb \
  --name ai-observer \
  tobilg/ai-observer:latest

This stores the DuckDB database in your local ./ai-observer-data directory, making it easy to backup or inspect.

Using Homebrew (macOS Apple Silicon)

brew tap tobilg/ai-observer
brew install ai-observer
ai-observer

Using Binary

Download the latest release for your platform from Releases, then:

./ai-observer

Building from Source

git clone https://github.com/tobilg/ai-observer.git
cd ai-observer
make setup   # Install dependencies
make all     # Build single binary with embedded frontend
./bin/ai-observer

Configuration

Environment Variables

CORS and WebSocket origins allow AI_OBSERVER_FRONTEND_URL plus http://localhost:5173 and http://localhost:8080; set AI_OBSERVER_FRONTEND_URL when serving a custom UI origin.

CLI Options

ai-observer [command] [options]

Commands:

Global Options:

Examples:

# Start the server (default, no command needed)
ai-observer

# Show version
ai-observer --version

# Show setup instructions for Claude Code
ai-observer setup claude-code

# Show setup instructions for GitHub Copilot
ai-observer setup github-copilot

# Show setup instructions for OpenCode
ai-observer setup opencode

# Import data from all file-backed AI tools
ai-observer import all

# Export data to Parquet files
ai-observer export all --output ./export

# Delete data in a date range
ai-observer delete all --from 2025-01-01 --to 2025-01-31

# Watch local session files for changes
ai-observer watch all

Import Command

Import historical session data from local AI coding tool files into AI Observer.

ai-observer import [claude-code|codex|gemini|all] [options]

File locations:

Override with environment variables: AI_OBSERVER_CLAUDE_PATH, AI_OBSERVER_CODEX_PATH, AI_OBSERVER_GEMINI_PATH

Examples:

# Import from all file-backed tools
ai-observer import all

# Import Claude data from specific date range
ai-observer import claude-code --from 2025-01-01 --to 2025-12-31

# Dry run to see what would be imported
ai-observer import all --dry-run

# Force re-import and recalculate costs
ai-observer import claude-code --force --pricing-mode calculate

See docs/import.md for detailed documentation and docs/pricing.md for pricing calculation details.

Watch Command

Watch local session files in real-time and import new data incrementally as it's written. This is an alternative to configuring OTLP exporters for Claude Code, Codex CLI, and Gemini CLI — just start the watcher and it picks up data from those tools' native log files.

ai-observer watch [claude-code|codex|gemini|all] [options]

How it works:

• First start (default): Records the current position in each file without importing historical data. Only data written after startup is imported.
• First start with --backfill: Loads all existing session data, then watches for new changes.
• Restart: Resumes from where it left off, importing any data written while the watcher was stopped.

At startup, the watcher detects which tools are installed and reports their status:

File watcher starting...
  [claude-code] Watching ~/.claude/projects/ (2 directories)
  [codex]       Watching ~/.codex/sessions/ (1 directory)
  [gemini]      ~/.gemini/tmp/ not found — will poll for directory creation

Directories that don't exist yet are polled every 30 seconds and automatically added when they appear.

File locations are the same as the import command — override with AI_OBSERVER_CLAUDE_PATH, AI_OBSERVER_CODEX_PATH, AI_OBSERVER_GEMINI_PATH.

Note: Watch mode and OTLP ingestion (serve) are mutually exclusive. Running both simultaneously would produce duplicate data. Use watch for file-based ingestion or serve for OTLP — not both. GitHub Copilot and OpenCode are OTLP-only in AI Observer; use serve and the OTLP setup settings below.

Examples:

# Watch all file-backed tools for new data
ai-observer watch all

# Watch only Claude Code sessions
ai-observer watch claude-code

# First run: load all historical data, then watch
ai-observer watch all --backfill

# Watch Gemini CLI only
ai-observer watch gemini

Export Command

Export telemetry data to portable Parquet files with an optional DuckDB views database.

ai-observer export [claude-code|codex|gemini|opencode|copilot-chat|github-copilot|all] --output <directory> [options]

Output files:

• traces.parquet — All trace/span data
• logs.parquet — All log records
• metrics.parquet — All metric data points
• ai-observer-export-{SOURCE}-{RANGE}.duckdb — Views database with relative paths

Examples:

# Export all data from database
ai-observer export all --output ./export

# Export Claude data with date filter
ai-observer export claude-code --output ./export --from 2025-01-01 --to 2025-01-15

# Export GitHub Copilot VS Code Extension telemetry
ai-observer export copilot-chat --output ./export

# Export OpenCode telemetry
ai-observer export opencode --output ./export

# Export to ZIP archive
ai-observer export all --output ./export --zip

# Export directly from raw files (without prior import)
ai-observer export claude-code --output ./export --from-files

# Dry run to preview export
ai-observer export all --output ./export --dry-run

--from-files is only available for local file sources (claude-code, codex, gemini, all). GitHub Copilot and OpenCode are OTLP-only and export from the DuckDB database.

See docs/export.md for detailed documentation.

Delete Command

Delete telemetry data from the database by time range.

ai-observer delete [logs|metrics|traces|all] --from DATE --to DATE [options]

Examples:

# Delete all data in a date range
ai-observer delete all --from 2025-01-01 --to 2025-01-31

# Delete only logs in a date range
ai-observer delete logs --from 2025-01-01 --to 2025-01-31

# Delete only Claude Code data
ai-observer delete all --from 2025-01-01 --to 2025-01-31 --service claude-code

# Skip confirmation prompt
ai-observer delete all --from 2025-01-01 --to 2025-01-31 --yes

AI Tool Setup

# Enable telemetry (required)
export CLAUDE_CODE_ENABLE_TELEMETRY=1

# Configure exporters
export OTEL_METRICS_EXPORTER=otlp
export OTEL_LOGS_EXPORTER=otlp

# Set OTLP endpoint (HTTP)
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

# Set shorter intervals
export OTEL_METRIC_EXPORT_INTERVAL=10000  # 10 seconds (default: 60000ms)
export OTEL_LOGS_EXPORT_INTERVAL=5000     # 5 seconds (default: 5000ms)

{
  "telemetry": {
    "enabled": true,
    "target": "local",
    "useCollector": true,
    "otlpEndpoint": "http://localhost:4318",
    "otlpProtocol": "http",
    "logPrompts": true
  }
}

export OTEL_METRIC_EXPORT_TIMEOUT=10000
export OTEL_LOGS_EXPORT_TIMEOUT=5000

[otel]
log_user_prompt = true  # set to false to redact prompts
exporter = { otlp-http = { endpoint = "http://localhost:4318/v1/logs", protocol = "binary" } }
trace_exporter = { otlp-http = { endpoint = "http://localhost:4318/v1/traces", protocol = "binary" } }

{
  "$schema": "https://opencode.ai/config.json",
  "plugin": ["@devtheops/opencode-plugin-otel"]
}

export OPENCODE_ENABLE_TELEMETRY=1
export OPENCODE_OTLP_ENDPOINT=http://localhost:4318
export OPENCODE_OTLP_PROTOCOL=http/protobuf

{
  "github.copilot.chat.otel.enabled": true,
  "github.copilot.chat.otel.exporterType": "otlp-http",
  "github.copilot.chat.otel.otlpEndpoint": "http://localhost:4318",
  "github.copilot.chat.otel.captureContent": true
}

export COPILOT_OTEL_ENABLED=true
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export COPILOT_OTEL_CAPTURE_CONTENT=true

Architecture

OTLP mode (ai-observer or ai-observer serve):

flowchart TB
    claude[Claude Code<br/>traces, metrics, logs]
    gemini[Gemini CLI<br/>traces, metrics, logs]
    codex[OpenAI Codex CLI<br/>logs, traces]
    copilot[GitHub Copilot<br/>VS Code + CLI<br/>traces, metrics, logs]
    opencode[OpenCode<br/>OTEL plugin<br/>traces, metrics, logs]

    subgraph observer[AI Observer]
        ingest[OTLP Ingestion<br/>HTTP/JSON + HTTP/Protobuf<br/>Port 4318]
        derived[Derived Metrics<br/>tokens, costs, deltas]
        db[(DuckDB<br/>local analytics)]
        api[REST API + WebSocket Hub<br/>Port 8080]
        ui[React Dashboard<br/>embedded frontend]
    end

    claude -->|OTLP/HTTP| ingest
    gemini -->|OTLP/HTTP| ingest
    codex -->|OTLP/HTTP| ingest
    copilot -->|OTLP/HTTP| ingest
    opencode -->|OTLP/HTTP| ingest
    ingest --> derived
    ingest --> db
    derived --> db
    db --> api
    api --> ui

File watcher mode (ai-observer watch all):

flowchart TB
    claudeFiles[Claude Code<br/>~/.claude/projects/**/*.jsonl]
    geminiFiles[Gemini CLI<br/>~/.gemini/tmp/**/session-*.json]
    codexFiles[OpenAI Codex CLI<br/>~/.codex/sessions/*.jsonl]

    subgraph observer[AI Observer]
        watcher[File Watcher<br/>fsnotify + polling<br/>incremental parsing]
        db[(DuckDB<br/>local analytics)]
        api[REST API + WebSocket Hub<br/>Port 8080]
        ui[React Dashboard<br/>embedded frontend]
    end

    claudeFiles -->|file changes| watcher
    geminiFiles -->|file changes| watcher
    codexFiles -->|file changes| watcher
    watcher --> db
    db --> api
    api --> ui

GitHub Copilot and OpenCode are not shown in watcher mode because AI Observer currently receives their telemetry through OTLP only.

Tech Stack:

• Backend: Go 1.26+, chi router, DuckDB 1.5.3, gorilla/websocket
• Frontend: React 19, TypeScript, Vite, Tailwind CSS v4, Zustand, Recharts

API Reference

AI Observer exposes two HTTP servers:

OTLP Ingestion (Port 4318)

Standard OpenTelemetry Protocol endpoints for receiving telemetry data.

• Transport is HTTP/1.1 + h2c (no gRPC listener exposed); Content-Encoding: gzip is supported for compressed payloads.

Query API (Port 8080)

REST API for querying stored telemetry data. Unless otherwise specified, from/to default to the last 24 hours.

Data Collected

AI Observer receives standard OpenTelemetry data:

All data is stored locally in DuckDB. Nothing is sent to external services.

Telemetry Reference

Each AI coding tool exports different telemetry signals. Here's what you can observe:

flowchart TB
    trace[Trace ID<br/>session-level]
    task[run_task]
    turn1[run_sampling_request / run_turn<br/>agent turn 1]
    try1[try_run_sampling_request / try_run_turn]
    stream1[receiving_stream]
    reasoning[reasoning / function_call]
    receiving[receiving]
    turn2[run_sampling_request / run_turn<br/>agent turn 2]
    more[...]

    trace --> task
    task --> turn1
    turn1 --> try1
    turn1 --> stream1
    stream1 --> reasoning
    stream1 --> receiving
    turn1 --> more
    task --> turn2
    task --> more

Metric Availability by Ingestion Mode

Not all metrics are available in every mode. OTLP receives telemetry emitted by each tool's built-in OpenTelemetry instrumentation (in-memory counters, histograms, traces), while watch and import parse local session files which only contain conversation messages and per-response token/cost data.

Claude Code

Local JSONL files store conversation messages and API usage per response. Operational metrics (lines of code, active time, git activity) are tracked in-memory by Claude Code's OTel instrumentation and only emitted over the network — they are never written to disk.

OpenAI Codex CLI

Local JSONL files store conversation events and cumulative token counts. The full trace/span hierarchy (session → task → turn → stream) is only available via OTLP.

Gemini CLI

Local JSON session files store messages with per-response token counts. All operational metrics (API latency, tool call timing, agent duration, memory/CPU usage, and ~20 other metrics) are tracked in-memory by Gemini CLI's OTel instrumentation and only emitted over the network.

GitHub Copilot

GitHub Copilot telemetry is OTLP-only in AI Observer. There is no local file watcher or historical file import parser for Copilot data.

OpenCode

OpenCode telemetry is OTLP-only in AI Observer through the @devtheops/opencode-plugin-otel plugin. There is no local file watcher or historical file import parser for OpenCode data.

Summary: OTLP mode provides the richest telemetry — all metrics, traces, and events emitted by each tool's built-in or plugin instrumentation. Watch and import modes provide token usage, cost metrics, and full session transcripts parsed from local files. Operational metrics (lines of code, active time, API latency, git activity, etc.) only exist in the OTel telemetry stream and cannot be reconstructed from local files.

Understanding Token Metrics: OTLP vs Local Files

When comparing token usage from AI Observer's OTLP ingestion with tools like ccusage that parse local session files, you may notice significant differences in reported values. This is expected behavior due to different counting semantics.

Example Comparison

Here's a real comparison from a single day of Claude Code usage:

Why This Happens

The discrepancy is most pronounced for input and output tokens:

1. Claude Code OTLP metrics appear to report tokens differently than the API response's usage object that gets written to JSONL files.

2. Local JSONL files store the exact usage.input_tokens and usage.output_tokens values from Claude's API response, which ccusage reads directly.

3. Cache tokens (creation and read) are much closer between the two sources, suggesting these are counted consistently.

Token Type Comparison

Which Data Source Should I Use?

Reconciling the Data

If you need ccusage-compatible metrics in AI Observer:

# Import from local files instead of relying on OTLP
ai-observer import claude-code --from 2025-01-01 --to 2025-12-31

Imported data uses the same token counting as ccusage and will show matching values.

Technical Details

• OTLP metrics arrive with aggregationTemporality: 1 (DELTA), meaning each data point is a per-request value
• The type attribute distinguishes token types: input, output, cacheCreation, cacheRead
• Imported metrics include an import_source: local_jsonl attribute to distinguish them from OTLP data
• Watch mode metrics include an import_source: file_watcher attribute
• OTLP metrics have no import_source attribute (or it's null)

Development

Developer quickstart

make setup          # install Go + frontend deps
make backend-dev    # terminal 1: run API/OTLP server on 8080/4318
make frontend-dev   # terminal 2: Vite dev server on http://localhost:5173
# browse http://localhost:5173 (API + /ws proxied to :8080)

Prerequisites

• Go 1.26+
• Node.js 22+
• pnpm 10+
• Make

Commands

make setup        # Install all dependencies
make dev          # Run backend + frontend in dev mode
make test         # Run all tests
make lint         # Run linters
make clean        # Clean build artifacts

Project Structure

flowchart TB
    root[ai-observer]
    backend[backend]
    cmd[cmd/server<br/>main entry point]
    internal[internal]
    api[api<br/>API types and helpers]
    deleter[deleter<br/>data deletion logic]
    exporter[exporter<br/>Parquet export and views database]
    handlers[handlers<br/>HTTP handlers]
    importer[importer<br/>historical import for Claude, Codex, Gemini]
    otlp[otlp<br/>OTLP decoders and derived telemetry]
    pricing[pricing<br/>embedded pricing and cost calculation]
    server[server<br/>server setup and routing]
    storage[storage<br/>DuckDB storage layer]
    watcher[watcher<br/>incremental file ingestion]
    websocket[websocket<br/>real-time updates]
    compression[pkg/compression<br/>GZIP decompression]
    frontend[frontend]
    src[src]
    components[components<br/>React components]
    pages[pages<br/>page components]
    stores[stores<br/>Zustand stores]
    lib[lib<br/>utilities]
    docs[docs<br/>documentation]
    makefile[Makefile]

    root --> backend
    backend --> cmd
    backend --> internal
    internal --> api
    internal --> deleter
    internal --> exporter
    internal --> handlers
    internal --> importer
    internal --> otlp
    internal --> pricing
    internal --> server
    internal --> storage
    internal --> watcher
    internal --> websocket
    backend --> compression
    root --> frontend
    frontend --> src
    src --> components
    src --> pages
    src --> stores
    src --> lib
    root --> docs
    root --> makefile

CI/CD

GitHub Actions automatically:

Creating a Release

git tag v1.0.0
git push origin v1.0.0

Troubleshooting

AI_OBSERVER_API_PORT=9090 AI_OBSERVER_OTLP_PORT=4319 ./ai-observer

AI_OBSERVER_FRONTEND_URL=http://localhost:3000 ./ai-observer

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add some amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• Built with OpenTelemetry standards
• Powered by DuckDB for fast analytics
• UI components from shadcn/ui