[FEATURE]: Regression testing harness for post-migration validation of configurations (Workload Assets) #1122
Description
Is there an existing issue for this?
- I have searched the existing issues
Problem statement
Problem statement
- [FEATURE]: Regression testing harness for post-migration validation for managed tabled (Data Assets) #927 for Data-related component
As a customer, much of the risk and work of a large migration is testing and validation. There is currently no opinionated pattern from Databricks on how to do regression testing. UC migrations provide an impetus to build one, as this type of migration has high risk.
Proposed Solution
The feature should:
- Account for any workload assets being migrated, including dependencies (cluster configurations, policies, ...) - this should be handled by assessment job already.
- Provide a useful changelog of asset changes made that can be analyzed further. Similar to a system table. Example rough schema (includes many optional fields, can be split into denormalized asset-specific tables):
For migration completeness:
asset_type | asset_name | migrated_flag
For migration workflow success:
workflow_name | execution_duration | success_flag | error_message
Additional Context
If any amount of rollback is possible in these kinds of migrations, that should also be considered in this feature's design.