AI Governance & Risk

Model risk management

Inventory, risk tiering, validation and ongoing monitoring for traditional ML and generative AI models.

Vue d’ensemble

Models now inform credit decisions, pricing, clinical operations, hiring and customer interactions, and many organizations are adding generative AI and third-party models at speed. Model risk management provides the discipline to know which models you rely on, how much risk they carry and whether they continue to perform as intended.

We help establish or modernize model risk programs: a complete model inventory that includes vendor and embedded models, risk-tiering criteria, development and documentation standards, independent validation and ongoing performance monitoring. For financial services clients, we align practices with supervisory guidance such as SR 11-7 and its equivalents in other jurisdictions.

Generative AI and agentic systems need adapted methods. We extend validation to cover prompt and retrieval design, groundedness, robustness to adversarial inputs, bias and the reliability of tool use, and we define what ongoing monitoring looks like when outputs are open-ended. Our validators can also add independent validation capacity when your second line needs support.

Ce que vous obtenez

01Model inventory covering internal, vendor and embedded models
02Risk-tiering criteria and model risk policy
03Model development and documentation standards
04Independent validation reports for priority models
05Ongoing monitoring framework and reporting

Technologies que nous maîtrisons

MLflow
Unity Catalog
SHAP
Fairlearn
Evidently
Azure Machine Learning
Amazon SageMaker Model Monitor

Nous sommes indépendants des éditeurs : les outils sont choisis en fonction de vos besoins et de vos investissements existants.

Les questions que l’on nous pose souvent

Does model risk management apply to generative AI?

Yes, though the methods need adapting. Generative AI models are harder to validate with traditional statistical tests, so validation also covers evaluation sets, adversarial testing, groundedness and human review processes.

Can you perform independent model validation?

Yes. Our validators can review models independently of the teams that built them, providing documented findings and recommendations to support your second-line oversight.