Problem

Production AI systems need provider-agnostic LLM access with consistent error handling, structured output validation, streaming support, and operational guardrails. Existing wrappers either lock you into one provider, lack production-grade retry/circuit-breaker patterns, or impose framework-level coupling (LangChain).

Solution

One Protocol, three provider adapters (OpenAI, Anthropic, Google), and an ExecutionEngine with retries, failover, circuit breakers, idempotency keys and diagnostics. Structured output goes through a validate-and-repair loop that keeps attempt traces. Four cache backends, a typed tool and agent runtime, a USD budget ledger, a redaction policy, deterministic replay, rate limiting, and a 930-line model catalog. The test suite carries provider contract harnesses, normalization fuzzing, failure injection, and a weekly cron that detects upstream provider-catalog drift.

Impact

The LLM kernel underneath every AI surface I've shipped at CanApply — crawling, outreach, retrieval and the Dana agent runtime, which consumes it pinned to a 40-character commit SHA enforced in four independent places, two of them asserted inside the built image. Open source, but honestly: no external adopters yet.

Stack