Data Standardization and Measuring Data Quality Assessment for large scale migration from legacy risk management systems to a strategic system for one of the leading US Bank
Client : One of the leading US banks
Objective
- Standardize data, implement and monitor a control framework to assess data quality for large scale migration from legacy risk management systems to a strategic system
Challenges
- Presence of multiple risk systems and inconsistent data feeds
- Existence of different naming conventions and formats of the data
- Generation of multiple reports as per regulatory and internal requirements
- Need to maintain data granularity for reporting purposes
CRISIL’s solution
- CRISIL followed a 3-step end-to-end process for data standardization and quality enhancement
Establishing Control Framework
- Gap Analysis - Analyze the as is process and identify pain points
- Data Sourcing - Understand the sources and agree on the data that needs to be normalized
Data Analysis & Standardization
- Cluster Analysis – Identify and group the data on the basis of underlying issues or patterns
- Machine Learning - Approximate string matching using fuzzy logic for data which could not be clustered
- Standardization- Standardized data across all products at various levels
Resolution & Reporting
- Reporting- Data quality issue summary reported to senior management
- Reports generated using BI tools to ensure data consistency
- Resolution - Identified and remediated causes of data quality issues by coordinating with technology partners
Value addition
- Automation across 3 stages with Python
- Incorporated machine learning matching algorithms (fuzzy logic) using distance and ratio methodology to automate data quality checks
Client impact
- Created an end-to-end control framework for data quality management
- Established a scalable and standardized reporting framework for both internal use and regulatory requirements
- Overall automation resulted in throughput reduction of ~85% in terms of TAT
Questions