PromptAnalyzer
A little more visibility into your LLM calls. Track prompt changes, responses, and costs with a single Python decorator.
View source Python / LLM observability / SQLite
The idea
Prompts change quickly. Keeping track of which version produced a response, and what that call cost, should be part of the development workflow.
How I built it
I built a local-first observability library with automatic prompt versioning, inference logging, and token and cost tracking. SHA-256 hashes identify prompt versions, while diffs and full-text search make changes easier to inspect.
Where it landed
An open-source Python library with provider adapters for OpenAI, Anthropic, Gemini, Ollama, and other LLM providers. The repository contains the implementation and usage instructions.