Making AI Pass Audit: A Model-Risk-Management Framework for LLMs in Financial Crime and Identity

Every conversation about deploying AI in a regulated financial institution reaches the same objection, usually from the Chief Compliance Officer, and usually early: AI is a compliance risk. They're right. But they're right for a reason most AI vendors misdiagnose.

The risk isn't the model's intelligence. It's the absence of a management framework around it. A logistic-regression credit model with no validation, no monitoring, and no audit trail is also a compliance risk — a well-understood one. What makes a large language model feel different is that it's often introduced outside the model-risk process entirely: bought as a productivity tool, wired into a workflow, and pointed at consequential decisions before anyone asks who validated it or how a decision it influenced could be reconstructed for an examiner.

Financial institutions already have a playbook for this exact problem. In the United States it is SR 11-7 — the Federal Reserve and OCC's supervisory guidance on model risk management. The task in front of a compliance or financial-crime leader is not to invent new governance for LLMs. It is to extend a mature discipline onto a technology that breaks some of its old assumptions.

What SR 11-7 already gives us

SR 11-7 defines model risk as the potential for adverse consequences from decisions based on model outputs that are incorrect or misused. It is deliberately principles-based and scaled to the complexity and materiality of the model. It rests on three pillars:

• Robust development, implementation, and use — sound design, appropriate data, and use consistent with the model's intended purpose.
• Effective validation — independent challenge covering conceptual soundness, ongoing monitoring, and outcomes analysis.
• Sound governance, policies, and controls — clear ownership, documentation, and oversight, operationalized through the three lines of defense: the business that owns the model, independent validation, and internal audit.

None of that is obsolete in the age of generative AI. All of it still applies. What changes is where the hard parts are.

Where LLMs stress the playbook

Classic model risk management was built around models that are, broadly, reproducible and inspectable: given the same inputs they produce the same output, and their logic can be examined. Large language models violate several of those assumptions at once:

• Non-determinism. The same prompt can yield different outputs. Validation approaches that assume reproducibility need new tools — pinned versions, controlled sampling, and distributional testing rather than single-point checks.
• Opaque reasoning. There are no coefficients to inspect. "Explainability" has to be engineered into the surrounding system rather than read off the model.
• Confabulation. The failure mode NIST calls confabulation — a fluent, confident, wrong answer — is uniquely dangerous in compliance, where a plausible but fabricated rationale can pass a cursory human review.
• The input is an attack surface. Prompt injection and adversarial inputs mean the data flowing into the model can subvert it. That is a control problem the credit-model era never had.
• Unbounded scope. An LLM will answer almost anything, including questions well outside the use case it was validated for. Scope has to be enforced, not assumed.
• Foundation-model dependency. The core model is usually a third party you did not build, cannot fully inspect, and that changes underneath you when the provider ships a new version — a silent revalidation trigger.
• It's a system, not a model. The risk lives in the whole pipeline — prompt, retrieved context, tools, guardrails, and the human step — not in a single artifact. Governance has to attach to the system.

These are not reasons to keep AI out of regulated workflows. They are a map of exactly where to add controls.

The framework: extending SR 11-7 to LLMs

The mapping below takes each SR 11-7 pillar, names the LLM-specific risk that stresses it, and states the control that closes the gap. It is model-provider-agnostic by design — the point is the control discipline, not any particular vendor.

1. Development, implementation, and use

The highest-leverage single control is grounding. An LLM that answers from retrieved, cited source records — the customer file, the transaction history, the sanctions list — rather than from its own parametric memory is dramatically more explainable and far less prone to confabulation. Pair that with treating the prompt as model code: versioned, reviewed, and change-controlled, because a prompt edit can change model behavior as much as a coefficient change once did. Enforce scope so the system refuses questions outside its validated use, and minimize the sensitive data the model can see.

2. Validation

Independent validation starts with the most important question, which is often skipped: is an LLM even the right instrument for this decision? For decisions that require a legally reasoned, specific justification — an adverse action against a consumer, for instance — the answer is frequently no, at least not on its own. Where an LLM is appropriate, validation must measure what the old playbook never had to: hallucination rate, output variance, susceptibility to prompt injection, and behavior under adversarial input, alongside conventional bias and outcomes testing. Because the foundation model can change, validation is not a one-time gate but a monitoring commitment.

3. Governance, policies, and controls

The three lines of defense map cleanly onto AI: the business owns the AI system and its outcomes, an independent function validates and challenges it, and internal audit tests the whole arrangement. Two controls deserve special emphasis. First, change control must treat a foundation-model version update as a model change — because it is one — triggering revalidation rather than passing silently. Second, and most important, is the audit trail.

The examinability principle

Examinability is the through-line of this entire framework. For any AI-assisted decision, the institution should be able to reconstruct, after the fact, the complete chain: the input, the model and its exact version, the full prompt, the retrieved context the model was given, the raw output, the results of any guardrail or validation checks, and the human disposition — including who made it. Designed in from the start, this log is cheap. Bolted on after an examination request, it is often impossible. Design the decision log before you design the model's cleverness.

A tiered control model

Not every use case earns the same rigor, and pretending otherwise is how AI programs stall. Tier by two axes — decision impact and model autonomy — and match the control budget to the tier:

• Tier 0 — Assistive. Summarization, drafting, search. A human owns every decision; the model only accelerates. Light controls, but still logged.
• Tier 1 — Recommending. The model proposes a disposition and a rationale; a human decides. Moderate controls, a mandatory human gate, and full per-decision logging. Most high-value financial-crime use cases live here.
• Tier 2 — Autonomous. The model acts without per-case human review. Reserved for low-stakes, high-volume, reversible decisions, with the heaviest monitoring, sampling-based human oversight, and tight kill-switch controls.

The recurring failure is applying Tier-2 ambition on a Tier-0 control budget — letting a model quietly make consequential decisions that no one validated and no one can reconstruct. Match the controls to the stakes, explicitly, and write the tier down.

Applying it to financial crime and identity

AML alert triage. An LLM summarizes a transaction-monitoring alert, pulls the relevant customer and transaction context, and drafts a disposition recommendation with its reasoning. This is a Tier-1 use case: the analyst still dispositions the alert, and every step is logged for SAR defensibility. Done this way, it attacks the brutal false-positive economics of transaction monitoring — where the overwhelming majority of alerts are cleared — without surrendering the audit trail an examiner will demand.

KYC and identity. Document extraction and adverse-media summarization are strong assistive use cases, provided outputs cite their sources and a human retains the identity decision itself. The value is in compressing analyst review time, not in replacing the judgment.

Synthetic identity and first-party fraud. Here the explainability constraint becomes a legal one. Where a decision drives an adverse action against a consumer, the Fair Credit Reporting Act and ECOA require specific, accurate reasons. A black-box LLM score cannot supply a defensible adverse-action reason on its own. The disciplined answer is to keep the model assistive, or to pair it with an explainable decisioning layer that owns the reason codes. The skill is knowing where explainability is a regulatory requirement rather than a nice-to-have — and designing so the requirement is met by construction, not by hope.

AI passes audit when it's a managed model, not a magic box

The institutions that deploy AI successfully in financial crime and identity will not be the ones with the most capable model. They will be the ones that treat the model as something under management: developed deliberately, validated independently, governed continuously, and logged completely. That is not a tax on innovation. It is the precondition for using AI where the stakes — and the regulators — are highest.

"AI is a compliance risk" is true right up until you put a framework around it. After that, it's a control.