Added force_refresh parameter for Assessment #4183

pritishpai · 2025-06-25T14:56:52Z

Changes

Added a parameter to the assessment job allowing users to force a complete refresh of the assessment data

Resolves #4184

Functionality

added relevant user documentation

Tests

manually tested
added unit tests
added integration tests
verified on staging environment (screenshot attached)

github-actions · 2025-06-25T17:02:51Z

❌ 55/56 passed, 1 failed, 9 skipped, 2h32m1s total

❌ test_installation_when_dashboard_id_is_invalid[01ef4d7b294112968fa07ffae17dd55f]: databricks.sdk.errors.platform.PermissionDenied: User does not have read permission for dashboard [dashboardId=01ef4d7b294112968fa07ffae17dd55f]. Config: host=https://DATABRICKS_HOST, account_id=DATABRICKS_ACCOUNT_ID, CLOUD_ENV_client_id=0a330eb5-dd51-4d97-b6e4-c474356b1d5d, CLOUD_ENV_tenant_id=9f37a392-f0ae-4280-9796-f1864a10effc, auth_type=metadata-service, cluster_id=TEST_DEFAULT_CLUSTER_ID, warehouse_id=DATABRICKS_WAREHOUSE_ID, metadata_service_url=***. Env: DATABRICKS_HOST, DATABRICKS_ACCOUNT_ID, ARM_CLIENT_ID, ARM_TENANT_ID, DATABRICKS_AUTH_TYPE, DATABRICKS_CLUSTER_ID, DATABRICKS_WAREHOUSE_ID, DATABRICKS_METADATA_SERVICE_URL (47.443s)

databricks.sdk.errors.platform.PermissionDenied: User does not have read permission for dashboard [dashboardId=01ef4d7b294112968fa07ffae17dd55f]. Config: host=https://DATABRICKS_HOST, account_id=DATABRICKS_ACCOUNT_ID, CLOUD_ENV_client_id=0a330eb5-dd51-4d97-b6e4-c474356b1d5d, CLOUD_ENV_tenant_id=9f37a392-f0ae-4280-9796-f1864a10effc, auth_type=metadata-service, cluster_id=TEST_DEFAULT_CLUSTER_ID, warehouse_id=DATABRICKS_WAREHOUSE_ID, metadata_service_url=***. Env: DATABRICKS_HOST, DATABRICKS_ACCOUNT_ID, ARM_CLIENT_ID, ARM_TENANT_ID, DATABRICKS_AUTH_TYPE, DATABRICKS_CLUSTER_ID, DATABRICKS_WAREHOUSE_ID, DATABRICKS_METADATA_SERVICE_URL
[gw1] linux -- Python 3.10.18 /home/runner/work/ucx/ucx/.venv/bin/python
20:20 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QXTy/config.yml) doesn't exist.
20:20 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
20:20 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
20:20 INFO [databricks.labs.ucx.install] Fetching installations...
20:20 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
20:20 DEBUG [tests.integration.conftest] Waiting for clusters to start...
20:20 DEBUG [tests.integration.conftest] Waiting for clusters to start...
20:20 INFO [databricks.labs.ucx.install] Installing UCX v0.59.2+1920250708202030
20:20 INFO [databricks.labs.ucx.install] Creating ucx schemas...
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
20:20 INFO [databricks.labs.ucx.install] Creating dashboards...
20:20 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
20:20 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
20:20 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
20:20 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
20:20 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
20:20 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
20:20 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
20:20 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
20:20 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
20:20 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
20:20 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
20:20 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:20 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:20 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:20 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:20 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:20 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:20 ERROR [databricks.labs.blueprint.parallel] installing dashboards(PosixPath('/home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main'), parent_path='/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QXTy/dashboards') task failed: User does not have read permission for dashboard [dashboardId=01ef4d7b294112968fa07ffae17dd55f]. Config: host=https://DATABRICKS_HOST, account_id=DATABRICKS_ACCOUNT_ID, CLOUD_ENV_client_id=0a330eb5-dd51-4d97-b6e4-c474356b1d5d, CLOUD_ENV_tenant_id=9f37a392-f0ae-4280-9796-f1864a10effc, auth_type=metadata-service, cluster_id=TEST_DEFAULT_CLUSTER_ID, warehouse_id=DATABRICKS_WAREHOUSE_ID, metadata_service_url=***. Env: DATABRICKS_HOST, DATABRICKS_ACCOUNT_ID, ARM_CLIENT_ID, ARM_TENANT_ID, DATABRICKS_AUTH_TYPE, DATABRICKS_CLUSTER_ID, DATABRICKS_WAREHOUSE_ID, DATABRICKS_METADATA_SERVICE_URL
Traceback (most recent call last):
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 168, in inner
    return func(*args, **kwargs), None
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 57, in wrapper
    raise err
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 36, in wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/install.py", line 669, in _create_dashboard
    dashboard_id = self._handle_existing_dashboard(dashboard_id, metadata.display_name, parent_path)
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/install.py", line 630, in _handle_existing_dashboard
    dashboard = self._ws.lakeview.get(dashboard_id)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/dashboards.py", line 2252, in get
    res = self._api.do("GET", f"/api/2.0/lakeview/dashboards/{dashboard_id}", headers=headers)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 85, in do
    return self._api_client.do(
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/_base_client.py", line 196, in do
    response = call(
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 57, in wrapper
    raise err
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 36, in wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/_base_client.py", line 298, in _perform
    raise error from None
databricks.sdk.errors.platform.PermissionDenied: User does not have read permission for dashboard [dashboardId=01ef4d7b294112968fa07ffae17dd55f]. Config: host=https://DATABRICKS_HOST, account_id=DATABRICKS_ACCOUNT_ID, CLOUD_ENV_client_id=0a330eb5-dd51-4d97-b6e4-c474356b1d5d, CLOUD_ENV_tenant_id=9f37a392-f0ae-4280-9796-f1864a10effc, auth_type=metadata-service, cluster_id=TEST_DEFAULT_CLUSTER_ID, warehouse_id=DATABRICKS_WAREHOUSE_ID, metadata_service_url=***. Env: DATABRICKS_HOST, DATABRICKS_ACCOUNT_ID, ARM_CLIENT_ID, ARM_TENANT_ID, DATABRICKS_AUTH_TYPE, DATABRICKS_CLUSTER_ID, DATABRICKS_WAREHOUSE_ID, DATABRICKS_METADATA_SERVICE_URL
20:20 ERROR [databricks.labs.blueprint.parallel] More than half 'installing dashboards' tasks failed: 0% results available (0/7). Took 0:00:02.574683
20:20 ERROR [databricks.labs.blueprint.parallel] installing components task failed: User does not have read permission for dashboard [dashboardId=01ef4d7b294112968fa07ffae17dd55f]. Config: host=https://DATABRICKS_HOST, account_id=DATABRICKS_ACCOUNT_ID, CLOUD_ENV_client_id=0a330eb5-dd51-4d97-b6e4-c474356b1d5d, CLOUD_ENV_tenant_id=9f37a392-f0ae-4280-9796-f1864a10effc, auth_type=metadata-service, cluster_id=TEST_DEFAULT_CLUSTER_ID, warehouse_id=DATABRICKS_WAREHOUSE_ID, metadata_service_url=***. Env: DATABRICKS_HOST, DATABRICKS_ACCOUNT_ID, ARM_CLIENT_ID, ARM_TENANT_ID, DATABRICKS_AUTH_TYPE, DATABRICKS_CLUSTER_ID, DATABRICKS_WAREHOUSE_ID, DATABRICKS_METADATA_SERVICE_URL
Traceback (most recent call last):
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 168, in inner
    return func(*args, **kwargs), None
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/install.py", line 557, in _create_database_and_dashboards
    Threads.strict("installing dashboards", list(self._get_create_dashboard_tasks()))
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 72, in strict
    raise errs[0]
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 168, in inner
    return func(*args, **kwargs), None
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 57, in wrapper
    raise err
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 36, in wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/install.py", line 669, in _create_dashboard
    dashboard_id = self._handle_existing_dashboard(dashboard_id, metadata.display_name, parent_path)
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/install.py", line 630, in _handle_existing_dashboard
    dashboard = self._ws.lakeview.get(dashboard_id)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/dashboards.py", line 2252, in get
    res = self._api.do("GET", f"/api/2.0/lakeview/dashboards/{dashboard_id}", headers=headers)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 85, in do
    return self._api_client.do(
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/_base_client.py", line 196, in do
    response = call(
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 57, in wrapper
    raise err
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 36, in wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/_base_client.py", line 298, in _perform
    raise error from None
databricks.sdk.errors.platform.PermissionDenied: User does not have read permission for dashboard [dashboardId=01ef4d7b294112968fa07ffae17dd55f]. Config: host=https://DATABRICKS_HOST, account_id=DATABRICKS_ACCOUNT_ID, CLOUD_ENV_client_id=0a330eb5-dd51-4d97-b6e4-c474356b1d5d, CLOUD_ENV_tenant_id=9f37a392-f0ae-4280-9796-f1864a10effc, auth_type=metadata-service, cluster_id=TEST_DEFAULT_CLUSTER_ID, warehouse_id=DATABRICKS_WAREHOUSE_ID, metadata_service_url=***. Env: DATABRICKS_HOST, DATABRICKS_ACCOUNT_ID, ARM_CLIENT_ID, ARM_TENANT_ID, DATABRICKS_AUTH_TYPE, DATABRICKS_CLUSTER_ID, DATABRICKS_WAREHOUSE_ID, DATABRICKS_METADATA_SERVICE_URL
20:20 ERROR [databricks.labs.blueprint.parallel] More than half 'installing components' tasks failed: 0% results available (0/2). Took 0:00:16.820173
20:20 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QXTy/config.yml) doesn't exist.
20:20 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
20:20 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
20:20 INFO [databricks.labs.ucx.install] Fetching installations...
20:20 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
20:20 DEBUG [tests.integration.conftest] Waiting for clusters to start...
20:20 DEBUG [tests.integration.conftest] Waiting for clusters to start...
20:20 INFO [databricks.labs.ucx.install] Installing UCX v0.59.2+1920250708202030
20:20 INFO [databricks.labs.ucx.install] Creating ucx schemas...
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
20:20 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
20:20 INFO [databricks.labs.ucx.install] Creating dashboards...
20:20 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
20:20 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
20:20 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
20:20 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
20:20 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
20:20 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
20:20 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
20:20 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
20:20 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
20:20 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
20:20 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
20:20 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:20 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:20 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:20 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:20 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:20 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:20 ERROR [databricks.labs.blueprint.parallel] installing dashboards(PosixPath('/home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main'), parent_path='/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.QXTy/dashboards') task failed: User does not have read permission for dashboard [dashboardId=01ef4d7b294112968fa07ffae17dd55f]. Config: host=https://DATABRICKS_HOST, account_id=DATABRICKS_ACCOUNT_ID, CLOUD_ENV_client_id=0a330eb5-dd51-4d97-b6e4-c474356b1d5d, CLOUD_ENV_tenant_id=9f37a392-f0ae-4280-9796-f1864a10effc, auth_type=metadata-service, cluster_id=TEST_DEFAULT_CLUSTER_ID, warehouse_id=DATABRICKS_WAREHOUSE_ID, metadata_service_url=***. Env: DATABRICKS_HOST, DATABRICKS_ACCOUNT_ID, ARM_CLIENT_ID, ARM_TENANT_ID, DATABRICKS_AUTH_TYPE, DATABRICKS_CLUSTER_ID, DATABRICKS_WAREHOUSE_ID, DATABRICKS_METADATA_SERVICE_URL
Traceback (most recent call last):
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 168, in inner
    return func(*args, **kwargs), None
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 57, in wrapper
    raise err
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 36, in wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/install.py", line 669, in _create_dashboard
    dashboard_id = self._handle_existing_dashboard(dashboard_id, metadata.display_name, parent_path)
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/install.py", line 630, in _handle_existing_dashboard
    dashboard = self._ws.lakeview.get(dashboard_id)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/dashboards.py", line 2252, in get
    res = self._api.do("GET", f"/api/2.0/lakeview/dashboards/{dashboard_id}", headers=headers)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 85, in do
    return self._api_client.do(
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/_base_client.py", line 196, in do
    response = call(
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 57, in wrapper
    raise err
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 36, in wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/_base_client.py", line 298, in _perform
    raise error from None
databricks.sdk.errors.platform.PermissionDenied: User does not have read permission for dashboard [dashboardId=01ef4d7b294112968fa07ffae17dd55f]. Config: host=https://DATABRICKS_HOST, account_id=DATABRICKS_ACCOUNT_ID, CLOUD_ENV_client_id=0a330eb5-dd51-4d97-b6e4-c474356b1d5d, CLOUD_ENV_tenant_id=9f37a392-f0ae-4280-9796-f1864a10effc, auth_type=metadata-service, cluster_id=TEST_DEFAULT_CLUSTER_ID, warehouse_id=DATABRICKS_WAREHOUSE_ID, metadata_service_url=***. Env: DATABRICKS_HOST, DATABRICKS_ACCOUNT_ID, ARM_CLIENT_ID, ARM_TENANT_ID, DATABRICKS_AUTH_TYPE, DATABRICKS_CLUSTER_ID, DATABRICKS_WAREHOUSE_ID, DATABRICKS_METADATA_SERVICE_URL
20:20 ERROR [databricks.labs.blueprint.parallel] More than half 'installing dashboards' tasks failed: 0% results available (0/7). Took 0:00:02.574683
20:20 ERROR [databricks.labs.blueprint.parallel] installing components task failed: User does not have read permission for dashboard [dashboardId=01ef4d7b294112968fa07ffae17dd55f]. Config: host=https://DATABRICKS_HOST, account_id=DATABRICKS_ACCOUNT_ID, CLOUD_ENV_client_id=0a330eb5-dd51-4d97-b6e4-c474356b1d5d, CLOUD_ENV_tenant_id=9f37a392-f0ae-4280-9796-f1864a10effc, auth_type=metadata-service, cluster_id=TEST_DEFAULT_CLUSTER_ID, warehouse_id=DATABRICKS_WAREHOUSE_ID, metadata_service_url=***. Env: DATABRICKS_HOST, DATABRICKS_ACCOUNT_ID, ARM_CLIENT_ID, ARM_TENANT_ID, DATABRICKS_AUTH_TYPE, DATABRICKS_CLUSTER_ID, DATABRICKS_WAREHOUSE_ID, DATABRICKS_METADATA_SERVICE_URL
Traceback (most recent call last):
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 168, in inner
    return func(*args, **kwargs), None
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/install.py", line 557, in _create_database_and_dashboards
    Threads.strict("installing dashboards", list(self._get_create_dashboard_tasks()))
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 72, in strict
    raise errs[0]
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 168, in inner
    return func(*args, **kwargs), None
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 57, in wrapper
    raise err
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 36, in wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/install.py", line 669, in _create_dashboard
    dashboard_id = self._handle_existing_dashboard(dashboard_id, metadata.display_name, parent_path)
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/install.py", line 630, in _handle_existing_dashboard
    dashboard = self._ws.lakeview.get(dashboard_id)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/service/dashboards.py", line 2252, in get
    res = self._api.do("GET", f"/api/2.0/lakeview/dashboards/{dashboard_id}", headers=headers)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 85, in do
    return self._api_client.do(
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/_base_client.py", line 196, in do
    response = call(
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 57, in wrapper
    raise err
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 36, in wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/sdk/_base_client.py", line 298, in _perform
    raise error from None
databricks.sdk.errors.platform.PermissionDenied: User does not have read permission for dashboard [dashboardId=01ef4d7b294112968fa07ffae17dd55f]. Config: host=https://DATABRICKS_HOST, account_id=DATABRICKS_ACCOUNT_ID, CLOUD_ENV_client_id=0a330eb5-dd51-4d97-b6e4-c474356b1d5d, CLOUD_ENV_tenant_id=9f37a392-f0ae-4280-9796-f1864a10effc, auth_type=metadata-service, cluster_id=TEST_DEFAULT_CLUSTER_ID, warehouse_id=DATABRICKS_WAREHOUSE_ID, metadata_service_url=***. Env: DATABRICKS_HOST, DATABRICKS_ACCOUNT_ID, ARM_CLIENT_ID, ARM_TENANT_ID, DATABRICKS_AUTH_TYPE, DATABRICKS_CLUSTER_ID, DATABRICKS_WAREHOUSE_ID, DATABRICKS_METADATA_SERVICE_URL
20:20 ERROR [databricks.labs.blueprint.parallel] More than half 'installing components' tasks failed: 0% results available (0/2). Took 0:00:16.820173
20:20 INFO [databricks.labs.ucx.install] Deleting UCX v0.59.2+1920250708202030 from https://DATABRICKS_HOST
20:20 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_sytlr
20:20 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=759981816281113, as it is no longer needed
20:20 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=400204229533961, as it is no longer needed
20:20 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=165409936841599, as it is no longer needed
20:20 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=100320917559427, as it is no longer needed
20:20 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=583685394218928, as it is no longer needed
20:20 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=149387546305032, as it is no longer needed
20:20 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=79445655000531, as it is no longer needed
20:20 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=43730117345877, as it is no longer needed
20:21 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=421488638683850, as it is no longer needed
20:21 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=417406123365064, as it is no longer needed
20:21 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=948691188613669, as it is no longer needed
20:21 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=872657204299735, as it is no longer needed
20:21 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=751085823139224, as it is no longer needed
20:21 INFO [databricks.labs.ucx.install] Deleting cluster policy
20:21 INFO [databricks.labs.ucx.install] Deleting secret scope
20:21 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw1] linux -- Python 3.10.18 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from acceptance #8680}

FastLee

LGTM

Add a CLI command to rerun assessment
Add the named parameter to the workflow fixture and include in a test.

…ssessment

…rkflow

pritishpai requested a review from a team as a code owner June 25, 2025 14:56

pritishpai had a problem deploying to account-admin June 25, 2025 14:56 — with GitHub Actions Error

pritishpai had a problem deploying to account-admin June 25, 2025 15:02 — with GitHub Actions Failure

pritishpai had a problem deploying to account-admin June 26, 2025 21:32 — with GitHub Actions Error

pritishpai had a problem deploying to account-admin June 26, 2025 21:36 — with GitHub Actions Failure

pritishpai changed the title ~~Full refresh all crawlers for each assessment run~~ Added force_refresh parameter for Assessment Jun 26, 2025

gueniai added this to UCX Jun 27, 2025

gueniai moved this to Ready for Review in UCX Jun 27, 2025

pritishpai force-pushed the feat/rerun_assessment branch from 1b33d5c to 336eec1 Compare June 30, 2025 18:02

pritishpai had a problem deploying to account-admin June 30, 2025 18:02 — with GitHub Actions Failure

pritishpai temporarily deployed to account-admin June 30, 2025 19:38 — with GitHub Actions Inactive

pritishpai requested a review from FastLee July 1, 2025 20:47

pritishpai temporarily deployed to account-admin July 1, 2025 20:51 — with GitHub Actions Inactive

FastLee requested changes Jul 2, 2025

View reviewed changes

pritishpai had a problem deploying to account-admin July 3, 2025 20:21 — with GitHub Actions Failure

pritishpai had a problem deploying to account-admin July 4, 2025 14:32 — with GitHub Actions Failure

pritishpai had a problem deploying to account-admin July 4, 2025 16:11 — with GitHub Actions Error

pritishpai added 12 commits July 4, 2025 12:14

Full refresh all crawlers for each assessment run

e66ec2d

Add documentation

0ca3981

Make force_refresh of assessment parameter driven

5d567e1

Add a parameter to the assessment job for force_refresh of data

56329bf

Use set for membership test

b68b6d4

Fetching permissions returns groups and users that have access.

03dacfb

Update readme to reflect parameter change

f2a5e9e

Add force_refresh flag to ensure-assessment-run for full refresh of a…

be17d85

…ssessment

Fetch flag value of force_refresh for assessment rerun

b53a116

Add integration test, fetch parameter from current job run

118b2f0

Update cli command

cd9b811

Fix fmt

36b2255

Add job_parameters to the test run_now to imitate adding it to run_wo…

a0e0325

…rkflow

pritishpai force-pushed the feat/rerun_assessment branch from e4bfabc to a0e0325 Compare July 4, 2025 16:15

pritishpai had a problem deploying to account-admin July 4, 2025 16:15 — with GitHub Actions Failure

Indicate job_parameters argument is intentionally unused

a14165d

pritishpai had a problem deploying to account-admin July 5, 2025 15:14 — with GitHub Actions Failure

Fix error due to list to set comparison

4d6f377

pritishpai had a problem deploying to account-admin July 7, 2025 14:53 — with GitHub Actions Error

Use set comprehension directly

2f1db1e

pritishpai had a problem deploying to account-admin July 7, 2025 15:02 — with GitHub Actions Error

Fix unit tests and cli command

6c5d996

pritishpai had a problem deploying to account-admin July 7, 2025 15:26 — with GitHub Actions Failure

Update cli and docs

986e0da

pritishpai had a problem deploying to account-admin July 7, 2025 16:08 — with GitHub Actions Failure

pritishpai had a problem deploying to account-admin July 7, 2025 16:58 — with GitHub Actions Failure

pritishpai had a problem deploying to account-admin July 7, 2025 19:18 — with GitHub Actions Failure

pritishpai had a problem deploying to account-admin July 8, 2025 13:57 — with GitHub Actions Failure

pritishpai had a problem deploying to account-admin July 8, 2025 20:08 — with GitHub Actions Failure

pritishpai requested a review from asnare July 9, 2025 14:37

pritishpai self-assigned this Jul 11, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Added force_refresh parameter for Assessment #4183

Added force_refresh parameter for Assessment #4183

Uh oh!

pritishpai commented Jun 25, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Jun 25, 2025 •

edited

Loading

Uh oh!

FastLee left a comment

Uh oh!

Uh oh!

Added force_refresh parameter for Assessment #4183

Are you sure you want to change the base?

Added force_refresh parameter for Assessment #4183

Uh oh!

Conversation

pritishpai commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Functionality

Tests

Uh oh!

github-actions bot commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FastLee left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pritishpai commented Jun 25, 2025 •

edited

Loading

github-actions bot commented Jun 25, 2025 •

edited

Loading