LLM Observability
Provider-agnostic tracing, cost, and latency for LLM apps.
The problem
LLM apps fail in ways normal software doesn't: cost balloons silently with token usage, latency drifts, models error under load. Without tracing you're flying blind.
The solution
A dependency-light library that wraps any LLM call to capture prompt, model, tokens, latency, cost and errors — stored in SQLite and rendered as HTML/Markdown reports or a live dashboard. It also ingests real Claude Code subscription usage from local logs.
What it does
- ▹Wrap any call with @observe / trace() — provider-agnostic
- ▹Ingests ~/.claude/projects/*.jsonl into spans (real subscription usage)
- ▹Cost attribution per model and per project
- ▹Static HTML + Markdown reports, or a live dashboard (llm-obs serve)
- ▹No API key, no cloud — 37 tests, 89% coverage
How it works
- 1
Capture
Each observed call becomes a Span (a Pydantic model); spans sharing a trace_id form a Trace.
- 2
Store
Spans persist to a SQLite SpanStore, with a pricing table turning tokens into cost.
- 3
Report
Aggregates (cost, p95 latency, error rate per model/project) render as a static report or a live interactive dashboard.
Architecture
Click a node to see what it does.
How it looks
Tech stack
- Python 3.11
- SQLite
- Pydantic
- HTML dashboards