Automating Model Testing of Pricing and Market Risk Models
Client : Large US Investment Bank
Objective
- To help a large US investment bank speed model validation and monitoring time, reduce model risk and eliminate costs by automating model testing of pricing and market risk models.
CRISIL's Solution
- Built a suite of tests for different models and respective use cases on the client proprietary platform to reduce the overall model validation timelines
- Set up a nine-member team (a healthy mix of quants and programmers), led by an onsite project manager
- Leverage team's strong experience in pricing models with programming in C++, Python and R
- Sample tests implemented:
- MC convergence
- Hedge backtesting
- Assumptions
- Stress scenarios
- Model stability
- VaR grid granularity
- Other features:
- Flexible test interface that could be used for CCAR and other mandates
- Allowed definition of further tests
- Interactive results on a dashboard
Client Impact
- Helped automate tests across models to reduce model validation time by ~40% for low-risk models and ~25% for high-risk models
- Increased efficiencies in the model monitoring process, enabling faster results for high-risk models on a monthly basis, and for lower risk models on a semi-annual basis
- Achieved reductions in model risk; exceptions from the scheduled batch immediately flag any potential model risk issues
- Helped the bank reduce timelines and the number of contractor personnel required for model validation, minimizing costs
- CRISIL solution covered quantitative tests for MC convergence, hedge backtesting, stability testing, assumptions and limitation testing, stress scenarios, VaR grid granularity, etc.