Page 28 - GS250402

Insights and Expertise
ChapterTitle
Model governance: Why LLM governance feels familiar

Despite the unique challenges LLMs introduce, their gov-
The AI risk financials ernance shares core principles with traditional model
oversight. Banks have long been required to validate and
cannot overlook document the models used in lending, risk management
and fraud detection. These models must be explainable,
auditable and free from discrimination.

Similarly, LLMs demand:
• Transparency: Just as banks document how risk
models generate credit scores, institutions must track
how LLMs make decisions.
• Bias mitigation: Financial models are tested for fair-
ness to prevent discrimination in lending. LLMs must
be evaluated for biases that could lead to regulatory
or ethical concerns.
• Change control: Traditional financial models require
By Karl Mattson versioning and strict change management. The same
discipline must apply to LLMs to ensure updates
Endor Labs don't introduce new risks.
arge language models (LLMs) are transforming The key takeaway: While LLMs operate differently, the
financial services, bringing efficiency and inno- governance mindset remains the same—rigorous valida-
vation at an unprecedented scale. For example, tion, continuous oversight and accountability are non-ne-
L JPMorgan Chase's LLM Suite is used by over gotiable.
200,000 employees globally to enhance productivity and
improve customer services. BloombergGPT is a 50 bil- Why LLM governance is different
lion parameter model that's widely effective in sentiment
analysis, among other sophisticated tasks. The financial Traditional financial models work with structured data
services industry is diving head first into the latest in and predefined rules. LLMs, on the other hand, are proba-
LLM technologies. But with great power comes great risk. bilistic and non-deterministic. The same input can yield
The challenge isn't just using LLMs—it's governing them different outputs, making verification more complex.
effectively.
LLM governance checklist
Regulators demand clear, demonstrable proof of a model's for payments providers
integrity. They ask:
• Is the model open source or proprietary? Regulators As large language models become more common in pay-
will require documentation that traces the model's ments and financial services, strong governance is essen-
origin, including licensing details, modification his- tial to ensure security, compliance and client trust.
tory and security assessments of its dependencies. • Inventory is crucial: Know which LLMs are in use
• Was it trained on biased or unverified data? Institu- across your systems and by partners.
tions must provide records of data provenance, bias • Validate data integrity: Ensure AI models are
audits and testing methodologies to prove the model trained on verifiable, bias-tested data to meet com-
does not introduce discriminatory outcomes. pliance standards.
• Can its outputs be explained and controlled? Finan- • Demand transparency: Require vendors to docu-
cial institutions will need to implement model ex- ment model origin, explainability methods, and bias
plainability techniques, such as LIME or SHAP, and mitigation practices.
demonstrate governance mechanisms that prevent • Implement control points: Establish governance
unintended behaviors or hallucinations. frameworks at model selection, deployment, and on-
going monitoring stages.
For decades, financial institutions have used models to set
policies, such as predicting loan default rates or setting • Prioritize security: Guard against adversarial at-
credit card interest rates. These models undergo rigorous tacks, prompt injections, and data leaks tied to AI
validation to ensure they don't introduce bias or hidden deployments.
risks. LLMs must now undergo the same scrutiny. Just as
banks stress-test risk models, LLMs require adversarial Strong governance will help payments providers harness
testing to ensure fairness, reliability and compliance. LLMs while protecting merchants, maintaining regula-
tory compliance and minimizing reputational risks.

28