Release Notes Template

Release notes

• These release notes provide information about new features, fixes, and updates that are included in the latest update release.
• We often roll out features (and sometimes even fixes) to Current over a period of time. If you don’t see something described below right away, you can expect it soon

ADB to Purview Connector Solution Accelerator - Version 2.1

Version 2.1

Features

• Support for OpenLineage version 13
• Column level lineage support in parser for Hive and Hive Delta
  • Note column lineage support for other sources will be coming as OpenLineage supports them
    • OpenLineage column support anticipated roadmap
      • ABFSS: September
      • WASBS: September
      • SQL / JDBC: Q4 CY 2022
      • Postgres / JDBC: Q4 CY 2022
      • MySQL / JDBC: Q4 CY 2022
• Service which updates custom type entities when underlying data source is scanned. Previously, updates only occurred when notebooks were re-run
• Support for new source types:
  • MySQL
  • Postgres
• ADB managed Hive table lineage support

Major Bug Fixes

• OpenLineage 13 resolves SaveAsTable driver crash issue

Other

• Improved documentation for custom source configuration
• Created custom source gallery adding ADLS Gen 1

Upgrade from 2.0 to 2.1

OpenLineage / Databricks Installation

• Remove the openlineage-spark-0.11.0.jar and install the openlineage-spark-0.13.0.jar

Changes to Function App Settings

• Update OlToPurviewMappings with the latest mappings found in the deployment/infra/OlToPurviewMappings.json file.

Auto-Generated What's Changed