Point of view
AI Governance & Risk
Model risk management
Inventory, risk tiering, validation and ongoing monitoring for traditional ML and generative AI models.
Overview
Models now inform credit decisions, pricing, clinical operations, hiring and customer interactions, and many organizations are adding generative AI and third-party models at speed. Model risk management provides the discipline to know which models you rely on, how much risk they carry and whether they continue to perform as intended.
We help establish or modernize model risk programs: a complete model inventory that includes vendor and embedded models, risk-tiering criteria, development and documentation standards, independent validation and ongoing performance monitoring. For financial services clients, we align practices with supervisory guidance such as SR 11-7 and its equivalents in other jurisdictions.
Generative AI and agentic systems need adapted methods. We extend validation to cover prompt and retrieval design, groundedness, robustness to adversarial inputs, bias and the reliability of tool use, and we define what ongoing monitoring looks like when outputs are open-ended. Our validators can also add independent validation capacity when your second line needs support.
What you receive
- 01Model inventory covering internal, vendor and embedded models
- 02Risk-tiering criteria and model risk policy
- 03Model development and documentation standards
- 04Independent validation reports for priority models
- 05Ongoing monitoring framework and reporting
Technologies we work with
- MLflow
- Unity Catalog
- SHAP
- Fairlearn
- Evidently
- Azure Machine Learning
- Amazon SageMaker Model Monitor
We are technology-agnostic: tools are chosen to fit your requirements and existing investments.
Questions we often hear
Does model risk management apply to generative AI?
Yes, though the methods need adapting. Generative AI models are harder to validate with traditional statistical tests, so validation also covers evaluation sets, adversarial testing, groundedness and human review processes.
Can you perform independent model validation?
Yes. Our validators can review models independently of the teams that built them, providing documented findings and recommendations to support your second-line oversight.
Insights
Related thinking
Let’s build what’s next, together.
Tell us about your goals. We’ll connect you with the right experts, usually within one business day.