-
Notifications
You must be signed in to change notification settings - Fork 96
Open
Open
Copy link
Labels
feat/workflowtriggered as a Databricks Job managed by UCXtriggered as a Databricks Job managed by UCXmigrate/managedgo/uc/upgrade Upgrade Managed Tables and Jobsgo/uc/upgrade Upgrade Managed Tables and Jobs
Description
Based on
ucx/src/databricks/labs/ucx/hive_metastore/tables.py
Lines 101 to 105 in db580a9
def sql_migrate_dbfs(self, target_table_key): | |
if not self.is_delta: | |
msg = f"{self.key} is not DELTA: {self.table_format}" | |
raise ValueError(msg) | |
return f"CREATE TABLE IF NOT EXISTS {target_table_key} DEEP CLONE {self.key};" |
and
ucx/src/databricks/labs/ucx/hive_metastore/table_migrate.py
Lines 277 to 284 in ec1801c
def _migrate_dbfs_root_table(self, src_table: Table, rule: Rule, grants: list[Grant] | None = None): | |
target_table_key = rule.as_uc_table_key | |
table_migrate_sql = src_table.sql_migrate_dbfs(target_table_key) | |
logger.debug(f"Migrating managed table {src_table.key} to using SQL query: {table_migrate_sql}") | |
self._backend.execute(table_migrate_sql) | |
self._backend.execute(src_table.sql_alter_to(rule.as_uc_table_key)) | |
self._backend.execute(src_table.sql_alter_from(rule.as_uc_table_key, self._ws.get_workspace_id())) | |
return self._migrate_acl(src_table, rule, grants) |
Tasks remaining:
- Allow skipping DBFS Root Delta Table if size exceeds preconfigured threshold
Metadata
Metadata
Assignees
Labels
feat/workflowtriggered as a Databricks Job managed by UCXtriggered as a Databricks Job managed by UCXmigrate/managedgo/uc/upgrade Upgrade Managed Tables and Jobsgo/uc/upgrade Upgrade Managed Tables and Jobs
Type
Projects
Status
No status