Receipt Extract
Receipt / invoice → validated, queryable structured data.
The problem
Receipts are messy: faded print, multi-page PDFs, foreign VAT. Naive 'ask the LLM for JSON' pipelines emit malformed output, silently hallucinate totals, and give you no ground truth.
The solution
A production-shaped pipeline pairing a vision LLM with strict schema enforcement, a deterministic Swiss QR-bill parser that overrides the model, and an offline eval harness producing real per-field precision/recall/F1.
What it does
- ▹Tool-use schema enforcement — the model never emits free-text JSON
- ▹Deterministic Swiss QR-bill parser overrides the LLM for provable fields
- ▹Pydantic consistency validators + retry-on-error loop
- ▹SQLite store: receipts, line items, extraction runs
- ▹Golden dataset + per-field P/R/F1 harness — 67 tests, 90% coverage
How it works
- 1
Load media
Images and PDFs are normalised to PNG pages via pypdfium2.
- 2
Extract & validate
A vision LLM fills a record_receipt tool schema; the result is validated against a Pydantic contract, feeding errors back on failure (max 2 retries).
- 3
Merge deterministic truth
If a Swiss QR-bill sidecar is present, its parsed IBAN/amount/creditor win over the model, then everything lands in SQLite for evaluation.
Architecture
Click a node to see what it does.
How it looks
Tech stack
- Python 3.11
- Anthropic Vision
- Pydantic v2
- SQLite
- pypdfium2