Shipped an internal-audit LLM agent that answers the audit department's standing questions in natural language.
How it was done
Built on the company's in-house agent platform with LangGraph, FastAPI, PostgreSQL. The agent reaches production data through a fixed set of read-only SQL tools rather than free-form query generation, and every figure it reports is re-verified by an independent query before it reaches a person. I also built the evaluation harness that keeps those answers honest as the tool set grows, and the measured case for moving the workload to a columnar store.
What this is not
The verification layer is the point, not a detail. An agent that answers confidently and wrongly is worse than no agent — so the design assumes the model will be wrong and checks it. I am not claiming the agent replaces an auditor; it removes the query-writing step between a question and an answer.