A Compliance-Reviewed AI Agent in Regulated Lending
Regulated consumer lending ·
Took an AI agent capability from prototype through security, legal, and compliance review into production at a regulated consumer lender.
Problem
An AI agent that touches a consumer-lending decision is not a model problem. It is an evidence problem. The organisation can be entirely satisfied that the agent works and still be unable to ship it, because nobody can produce the artifact a regulator would ask for eighteen months later.
The moving target made it worse. On May 12, 2025 the CFPB withdrew 67 guidance documents in a single Federal Register notice — 8 policy statements, 7 interpretive rules, 13 advisory opinions, and 39 others. Two of them were the circulars telling lenders how adverse-action notices had to work when a complex algorithm made the decision. Not one word of the underlying duty changed: ECOA still requires a statement of specific reasons, and Regulation B still defines what "specific" means.
Constraints
- Guidance is the most volatile layer in the stack and the easiest to mistake for the requirement — the design had to hold when guidance was withdrawn underneath it.
- Every agent decision needs a reason string traceable to the component that actually made the decision, not a plausible reconstruction generated after the fact.
- Trace retention had to survive the full adverse-action dispute window, not the 30-day default most agent logging ships with.
- Irreversible actions could not be model-discretionary.
- Review had to satisfy three separate functions — security, legal, and compliance — each with its own definition of sufficient evidence.
My role
I owned the agent capability end to end: the architecture, the tool surface the model was allowed to call, the evidence design that got it through legal and compliance review, and the approval gates on irreversible actions. Engineering built alongside me; the review strategy, the retention decision, and the control design were mine.
What shipped
- An agent capability running in production inside a regulated consumer lender.
- A tool surface designed so the model can only take actions the control design permits, with human approval gates on everything irreversible.
- A trace and retention design built to a 120-day floor derived from the statutory dispute window rather than from logging defaults.
- An eval harness that treats every instruction-file edit as a deploy requiring evidence.
- The written record that carried the capability through security, legal, and compliance review.
Outcome
- 120-day trace retention floor — Derived from the adverse-action dispute window rather than from a logging default. Most agent logging ships at 30 days, which expires before the evidence is ever requested. Read the full write-up
- 67 guidance documents withdrawn, zero design changes required — Building against the statute rather than against guidance meant the May 2025 CFPB withdrawal — which removed both circulars covering algorithmic adverse-action notices — did not invalidate the control design. Read the full write-up
Adoption and operational results are not yet cleared for publication. The design decisions and the evidence approach above are documented in full; the deployment numbers are not mine to publish yet.
Guidance is the layer that moves. Write against the layer that doesn't.
The single most useful rule from this work: design the agent against the statute and against the evidence it will have to produce. Guidance is the most volatile layer in the stack and the easiest one to mistake for the requirement.
The review is an evidence problem, not a model problem
Security, legal, and compliance were not asking whether the model was good. They were asking what artifact exists, where it lives, how long it survives, and who can produce it under request. Reframing the work around that question is what moved it.
The reason string has to come from the thing that made the decision
A reason generated after the fact by a model that did not make the decision is not a statement of specific reasons. It is a plausible narrative. The tool surface had to carry the reason out of the deciding component itself.
Counting is a compliance control
Wherever a control depends on a count, the count is where the bug will be. That is not a maxim — it is the failure I went looking for and found.
The organisation is described here as "a regulated consumer lender" throughout. The design decisions are mine to discuss; the deployment specifics are not.
Stack
Python · TypeScript · Model Context Protocol · PostgreSQL · Eval harness