Representative engagementData Quality
Source-to-Target Data Validation
- Challenge
- An organization moving reporting to a new data warehouse relied on spot checks of a few records to confirm each load. Differences between source systems and reports were discovered by business users, weeks after the fact.
- Approach
- Mapping documents were translated into executable SQL validations covering counts, completeness, transformation rules and totals at each layer. Checks were scheduled to run after every load, with results summarized for the data team.
- Technologies
- SQL
- SQL Server
- Databricks
- Python
- Azure DevOps
- Business outcome
- Differences are identified at the layer where they occur, before they reach reports, and migration sign-off is based on full reconciliation rather than samples.