Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test failure: test_experimental_permissions_migration_for_group_with_same_name #1254

Closed
github-actions bot opened this issue Apr 4, 2024 · 1 comment
Closed

Test failure: test_experimental_permissions_migration_for_group_with_same_name #1254

github-actions bot opened this issue Apr 4, 2024 · 1 comment

Comments

@github-actions
Copy link

github-actions bot commented Apr 4, 2024

❌ test_experimental_permissions_migration_for_group_with_same_name: AttributeError: 'WorkflowsDeployment' object has no attribute 'run_workflow' (1m53.751s) 
AttributeError: 'WorkflowsDeployment' object has no attribute 'run_workflow'
[gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
07:09 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='[email protected]', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='[email protected]')], entitlements=[], external_id=None, groups=[], id='4335939779773899', name=Name(family_name=None, given_name='[email protected]'), roles=[], schemas=[<UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_USER: 'urn:ietf:params:scim:schemas:core:2.0:User'>, <UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_EXTENSION_WORKSPACE_2_0_USER: 'urn:ietf:params:scim:schemas:extension:workspace:2.0:User'>], user_name='[email protected]')
07:09 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group ucx_EsEe: https://DATABRICKS_HOST#setting/accounts/groups/435139586061819
07:09 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='ucx_EsEe', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='435139586061819', members=[ComplexValue(display='[email protected]', primary=None, ref='Users/4335939779773899', type=None, value='4335939779773899')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[<GroupSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_GROUP: 'urn:ietf:params:scim:schemas:core:2.0:Group'>])
07:09 INFO [databricks.labs.ucx.mixins.fixtures] Account group ucx_EsEe: https://accounts.CLOUD_ENVdatabricks.net/users/groups/1025693803732766/members
07:09 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='ucx_EsEe', entitlements=[], external_id=None, groups=[], id='1025693803732766', members=[ComplexValue(display='[email protected]', primary=None, ref='Users/4335939779773899', type=None, value='4335939779773899')], meta=None, roles=[], schemas=[<GroupSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_GROUP: 'urn:ietf:params:scim:schemas:core:2.0:Group'>])
07:09 INFO [databricks.labs.ucx.mixins.fixtures] Cluster policy: https://DATABRICKS_HOST#setting/clusters/cluster-policies/view/00172391F37C28AF
07:09 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster policy fixture: CreatePolicyResponse(policy_id='00172391F37C28AF')
07:09 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster_policy permissions fixture: 00172391F37C28AF [group_name admins CAN_USE] -> [group_name ucx_EsEe CAN_USE]
07:09 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_srcef: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_srcef
07:09 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_srcef', metastore_id=None, name='ucx_srcef', owner=None, properties=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:09 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_srcef.ucx_tidwe: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_srcef/ucx_tidwe
07:09 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_srcef.ucx_tidwe', metastore_id=None, name='ucx_tidwe', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='ucx_srcef', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/ucx_srcef/ucx_tidwe', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
07:09 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.pDDp/config.yml) doesn't exist.
07:09 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
07:09 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
07:09 INFO [databricks.labs.ucx.install] Fetching installations...
07:09 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
07:09 INFO [databricks.labs.ucx.install] Installing UCX v0.21.1+2720240404070910
07:09 INFO [databricks.labs.ucx.install] Creating dashboards...
07:09 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
07:09 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Estimates)...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 01_0_group_migration.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 01_0_group_migration.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 00_0_metastore_assignment.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 00_0_metastore_assignment.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 02_0_data_modeling.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 02_0_data_modeling.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 03_0_data_migration.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 03_0_data_migration.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
07:09 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Main)...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 10___data_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 10___data_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 15___storage_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 15___storage_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 40___last_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 40___last_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 20___compute_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 20___compute_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 00___assessment_overview.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 00___assessment_overview.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 05___findings_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 05___findings_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 30_0_job_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 30_0_job_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
07:09 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Azure)...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
07:09 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Interactive)...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 00_0_interactive.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 00_0_interactive.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 02_0_cluster_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 02_0_cluster_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.installer.workflows] Creating jobs from tasks in main
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
07:09 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
07:09 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=099-destroy-schema
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
07:09 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.pDDp/README for the next steps.
07:09 INFO [databricks.labs.ucx.install] Installing UCX v0.21.1+2720240404070957
07:09 INFO [databricks.labs.ucx.install] Creating dashboards...
07:09 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
07:09 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Estimates)...
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 01_0_group_migration.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 01_0_group_migration.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 00_0_metastore_assignment.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 00_0_metastore_assignment.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 02_0_data_modeling.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 02_0_data_modeling.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 03_0_data_migration.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 03_0_data_migration.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
07:10 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Main)...
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 10___data_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 10___data_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 15___storage_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 15___storage_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 40___last_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 40___last_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 20___compute_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 20___compute_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 00___assessment_overview.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 00___assessment_overview.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 05___findings_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 05___findings_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 30_0_job_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 30_0_job_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
07:10 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Azure)...
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
07:10 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Interactive)...
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 00_0_interactive.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 00_0_interactive.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 02_0_cluster_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 02_0_cluster_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.installer.workflows] Creating jobs from tasks in main
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=migrate-groups-experimental job_id=815564432967504
07:10 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
07:10 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=assessment job_id=674589678445898
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=validate-groups-permissions job_id=841242590580262
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=migrate-tables job_id=269683596832287
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=099-destroy-schema job_id=515653996835866
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=migrate-tables-in-mounts-experimental job_id=416470814836785
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=migrate-groups job_id=670548058821920
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=remove-workspace-local-backup-groups job_id=162756226100937
07:10 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.pDDp/README for the next steps.
07:10 DEBUG [databricks.labs.ucx.workspace_access.manager] Crawling permissions
07:10 INFO [databricks.labs.ucx.workspace_access.generic] Listed cluster-policies in 0:00:00.000007
07:10 INFO [databricks.labs.ucx.workspace_access.manager] Starting to crawl permissions. Total tasks: 2
07:10 INFO [databricks.labs.ucx.workspace_access.manager] Total crawled permissions: 2
07:10 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_Siqd3.permissions] found 2 new records for permissions
07:10 INFO [databricks.labs.ucx.workspace_access.manager] Successfully saved the items to inventory table
07:10 INFO [databricks.labs.ucx.workspace_access.manager] Saved 2 to hive_metastore.ucx_Siqd3.permissions
07:09 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='[email protected]', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='[email protected]')], entitlements=[], external_id=None, groups=[], id='4335939779773899', name=Name(family_name=None, given_name='[email protected]'), roles=[], schemas=[<UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_USER: 'urn:ietf:params:scim:schemas:core:2.0:User'>, <UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_EXTENSION_WORKSPACE_2_0_USER: 'urn:ietf:params:scim:schemas:extension:workspace:2.0:User'>], user_name='[email protected]')
07:09 INFO [databricks.labs.ucx.mixins.fixtures] Workspace group ucx_EsEe: https://DATABRICKS_HOST#setting/accounts/groups/435139586061819
07:09 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace group fixture: Group(display_name='ucx_EsEe', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='435139586061819', members=[ComplexValue(display='[email protected]', primary=None, ref='Users/4335939779773899', type=None, value='4335939779773899')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[<GroupSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_GROUP: 'urn:ietf:params:scim:schemas:core:2.0:Group'>])
07:09 INFO [databricks.labs.ucx.mixins.fixtures] Account group ucx_EsEe: https://accounts.CLOUD_ENVdatabricks.net/users/groups/1025693803732766/members
07:09 DEBUG [databricks.labs.ucx.mixins.fixtures] added account group fixture: Group(display_name='ucx_EsEe', entitlements=[], external_id=None, groups=[], id='1025693803732766', members=[ComplexValue(display='[email protected]', primary=None, ref='Users/4335939779773899', type=None, value='4335939779773899')], meta=None, roles=[], schemas=[<GroupSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_GROUP: 'urn:ietf:params:scim:schemas:core:2.0:Group'>])
07:09 INFO [databricks.labs.ucx.mixins.fixtures] Cluster policy: https://DATABRICKS_HOST#setting/clusters/cluster-policies/view/00172391F37C28AF
07:09 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster policy fixture: CreatePolicyResponse(policy_id='00172391F37C28AF')
07:09 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster_policy permissions fixture: 00172391F37C28AF [group_name admins CAN_USE] -> [group_name ucx_EsEe CAN_USE]
07:09 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_srcef: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_srcef
07:09 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_srcef', metastore_id=None, name='ucx_srcef', owner=None, properties=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:09 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_srcef.ucx_tidwe: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_srcef/ucx_tidwe
07:09 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_srcef.ucx_tidwe', metastore_id=None, name='ucx_tidwe', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='ucx_srcef', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/ucx_srcef/ucx_tidwe', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
07:09 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.pDDp/config.yml) doesn't exist.
07:09 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
07:09 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
07:09 INFO [databricks.labs.ucx.install] Fetching installations...
07:09 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
07:09 INFO [databricks.labs.ucx.install] Installing UCX v0.21.1+2720240404070910
07:09 INFO [databricks.labs.ucx.install] Creating dashboards...
07:09 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
07:09 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Estimates)...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 01_0_group_migration.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 01_0_group_migration.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 00_0_metastore_assignment.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 00_0_metastore_assignment.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 02_0_data_modeling.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 02_0_data_modeling.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 03_0_data_migration.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 03_0_data_migration.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
07:09 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Main)...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 10___data_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 10___data_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 15___storage_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 15___storage_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 40___last_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 40___last_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 20___compute_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 20___compute_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 00___assessment_overview.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 00___assessment_overview.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 05___findings_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 05___findings_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 30_0_job_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 30_0_job_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
07:09 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Azure)...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
07:09 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Interactive)...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 00_0_interactive.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 00_0_interactive.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 02_0_cluster_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 02_0_cluster_summary.md because it's a text widget
07:09 DEBUG [databricks.labs.ucx.installer.workflows] Creating jobs from tasks in main
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
07:09 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
07:09 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=099-destroy-schema
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
07:09 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
07:09 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.pDDp/README for the next steps.
07:09 INFO [databricks.labs.ucx.install] Installing UCX v0.21.1+2720240404070957
07:09 INFO [databricks.labs.ucx.install] Creating dashboards...
07:09 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
07:09 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
07:09 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Estimates)...
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 01_0_group_migration.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 01_0_group_migration.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 00_0_metastore_assignment.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 00_0_metastore_assignment.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 02_0_data_modeling.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 02_0_data_modeling.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 03_0_data_migration.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 03_0_data_migration.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
07:10 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Main)...
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 10___data_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 10___data_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 15___storage_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 15___storage_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 40___last_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 40___last_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 20___compute_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 20___compute_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 00___assessment_overview.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 00___assessment_overview.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 05___findings_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 05___findings_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 30_0_job_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 30_0_job_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
07:10 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Azure)...
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Reading dashboard folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
07:10 INFO [databricks.labs.ucx.framework.dashboards] Creating dashboard [PDDP] UCX  Assessment (Interactive)...
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 00_0_interactive.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 00_0_interactive.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping query 02_0_cluster_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.framework.dashboards] Skipping viz 02_0_cluster_summary.md because it's a text widget
07:10 DEBUG [databricks.labs.ucx.installer.workflows] Creating jobs from tasks in main
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=migrate-groups-experimental job_id=815564432967504
07:10 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
07:10 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=assessment job_id=674589678445898
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=validate-groups-permissions job_id=841242590580262
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=migrate-tables job_id=269683596832287
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=099-destroy-schema job_id=515653996835866
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=migrate-tables-in-mounts-experimental job_id=416470814836785
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=migrate-groups job_id=670548058821920
07:10 INFO [databricks.labs.ucx.installer.workflows] Updating configuration for step=remove-workspace-local-backup-groups job_id=162756226100937
07:10 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.pDDp/README for the next steps.
07:10 DEBUG [databricks.labs.ucx.workspace_access.manager] Crawling permissions
07:10 INFO [databricks.labs.ucx.workspace_access.generic] Listed cluster-policies in 0:00:00.000007
07:10 INFO [databricks.labs.ucx.workspace_access.manager] Starting to crawl permissions. Total tasks: 2
07:10 INFO [databricks.labs.ucx.workspace_access.manager] Total crawled permissions: 2
07:10 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_Siqd3.permissions] found 2 new records for permissions
07:10 INFO [databricks.labs.ucx.workspace_access.manager] Successfully saved the items to inventory table
07:10 INFO [databricks.labs.ucx.workspace_access.manager] Saved 2 to hive_metastore.ucx_Siqd3.permissions
07:10 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 cluster_policy permissions fixtures
07:10 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster_policy permissions fixture: 00172391F37C28AF [group_name admins CAN_USE] -> [group_name ucx_EsEe CAN_USE]
07:10 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 cluster policy fixtures
07:10 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster policy fixture: CreatePolicyResponse(policy_id='00172391F37C28AF')
07:10 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 table fixtures
07:10 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_srcef.ucx_tidwe', metastore_id=None, name='ucx_tidwe', owner=None, pipeline_id=None, properties=None, row_filter=None, schema_name='ucx_srcef', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/ucx_srcef/ucx_tidwe', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
07:10 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace user fixtures
07:10 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='[email protected]', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='[email protected]')], entitlements=[], external_id=None, groups=[], id='4335939779773899', name=Name(family_name=None, given_name='[email protected]'), roles=[], schemas=[<UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_USER: 'urn:ietf:params:scim:schemas:core:2.0:User'>, <UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_EXTENSION_WORKSPACE_2_0_USER: 'urn:ietf:params:scim:schemas:extension:workspace:2.0:User'>], user_name='[email protected]')
07:10 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 account group fixtures
07:10 DEBUG [databricks.labs.ucx.mixins.fixtures] removing account group fixture: Group(display_name='ucx_EsEe', entitlements=[], external_id=None, groups=[], id='1025693803732766', members=[ComplexValue(display='[email protected]', primary=None, ref='Users/4335939779773899', type=None, value='4335939779773899')], meta=None, roles=[], schemas=[<GroupSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_GROUP: 'urn:ietf:params:scim:schemas:core:2.0:Group'>])
07:10 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 workspace group fixtures
07:10 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace group fixture: Group(display_name='ucx_EsEe', entitlements=[ComplexValue(display=None, primary=None, ref=None, type=None, value='allow-cluster-create')], external_id=None, groups=[], id='435139586061819', members=[ComplexValue(display='[email protected]', primary=None, ref='Users/4335939779773899', type=None, value='4335939779773899')], meta=ResourceMeta(resource_type='WorkspaceGroup'), roles=[], schemas=[<GroupSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_GROUP: 'urn:ietf:params:scim:schemas:core:2.0:Group'>])
07:10 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures
07:10 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_srcef', metastore_id=None, name='ucx_srcef', owner=None, properties=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
07:10 INFO [databricks.labs.ucx.install] Deleting UCX v0.21.1+2720240404071044 from https://DATABRICKS_HOST
07:10 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_Siqd3
07:10 INFO [databricks.labs.ucx.install] Deleting jobs
07:10 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=815564432967504.
07:10 INFO [databricks.labs.ucx.install] Deleting assessment job_id=674589678445898.
07:10 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=841242590580262.
07:10 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=269683596832287.
07:10 INFO [databricks.labs.ucx.install] Deleting 099-destroy-schema job_id=515653996835866.
07:10 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=416470814836785.
07:10 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=670548058821920.
07:10 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=162756226100937.
07:10 INFO [databricks.labs.ucx.install] Deleting cluster policy
07:10 INFO [databricks.labs.ucx.install] Deleting secret scope
07:10 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python

Running from nightly #14
@github-actions github-actions bot added the bug label Apr 4, 2024
@nfx
Copy link
Collaborator

nfx commented Apr 4, 2024

Fixed in #1250

@nfx nfx closed this as completed Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@nfx