Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow-listing language update #2101

Merged
merged 4 commits into from
Jul 9, 2024
Merged

Allow-listing language update #2101

merged 4 commits into from
Jul 9, 2024

Conversation

asnare
Copy link
Contributor

@asnare asnare commented Jul 8, 2024

Changes

This PR updates the language we use for allow-listing functionality. Some public APIs are affected, but I think the compatibility issues here are acceptable.

@asnare asnare added the internal this pull request won't appear in release notes label Jul 8, 2024
@asnare asnare self-assigned this Jul 8, 2024
@asnare asnare requested review from a team and qziyuan July 8, 2024 15:50
nfx
nfx requested changes Jul 8, 2024
View reviewed changes
@asnare asnare requested a review from nfx July 9, 2024 08:22
@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 9, 2024

❌ 120/124 passed, 4 failed, 23 skipped, 6h9m6s total

❌ test_create_catalog_schema_with_principal_acl_CLOUD_ENV: AssertionError: assert PrivilegeAssignment(principal='[email protected]', privileges=[]) in [PrivilegeAssignment(principal='[email protected]', privileges=[]), Pri...SCHEMA: 'USE_SCHEMA'>]), PrivilegeAssignment(principal='ucx_GRYog', privileges=[])] (5m6.356s) 
AssertionError: assert PrivilegeAssignment(principal='[email protected]', privileges=[<Privilege.USE_SCHEMA: 'USE_SCHEMA'>]) in [PrivilegeAssignment(principal='[email protected]', privileges=[<Privilege.USE_SCHEMA: 'USE_SCHEMA'>]), Pri...SCHEMA: 'USE_SCHEMA'>]), PrivilegeAssignment(principal='ucx_GRYog', privileges=[<Privilege.USE_SCHEMA: 'USE_SCHEMA'>])]
 +  where [PrivilegeAssignment(principal='[email protected]', privileges=[<Privilege.USE_SCHEMA: 'USE_SCHEMA'>]), Pri...SCHEMA: 'USE_SCHEMA'>]), PrivilegeAssignment(principal='ucx_GRYog', privileges=[<Privilege.USE_SCHEMA: 'USE_SCHEMA'>])] = PermissionsList(privilege_assignments=[PrivilegeAssignment(principal='[email protected]', privileges=[<Priv...CHEMA: 'USE_SCHEMA'>]), PrivilegeAssignment(principal='ucx_GRYog', privileges=[<Privilege.USE_SCHEMA: 'USE_SCHEMA'>])]).privilege_assignments
08:24 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/fQ3q
08:24 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sesab: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sesab
08:24 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sesab', metastore_id=None, name='ucx_sesab', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:25 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_sesab.ucx_tass6: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sesab/ucx_tass6
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_sesab.ucx_tass6', metastore_id=None, name='ucx_tass6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_sesab', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/fQ3q', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1720513520816, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ch2et', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ch2et', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720513520816, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:25 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_ch2et.ucx_sesab: https://DATABRICKS_HOST/explore/data/ucx_ch2et/ucx_sesab
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ch2et', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ch2et.ucx_sesab', metastore_id=None, name='ucx_sesab', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
[gw8] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
08:24 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/fQ3q
08:24 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sesab: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sesab
08:24 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sesab', metastore_id=None, name='ucx_sesab', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:25 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_sesab.ucx_tass6: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sesab/ucx_tass6
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_sesab.ucx_tass6', metastore_id=None, name='ucx_tass6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_sesab', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/fQ3q', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1720513520816, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ch2et', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ch2et', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720513520816, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:25 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_ch2et.ucx_sesab: https://DATABRICKS_HOST/explore/data/ucx_ch2et/ucx_sesab
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ch2et', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ch2et.ucx_sesab', metastore_id=None, name='ucx_sesab', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: <databricks.sdk.service._internal.Wait object at 0x7f832b100a00>
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='[email protected]', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='[email protected]')], entitlements=[], external_id=None, groups=[], id='6338907467934090', name=Name(family_name=None, given_name='[email protected]'), roles=[], schemas=[<UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_USER: 'urn:ietf:params:scim:schemas:core:2.0:User'>, <UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_EXTENSION_WORKSPACE_2_0_USER: 'urn:ietf:params:scim:schemas:extension:workspace:2.0:User'>], user_name='[email protected]')
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0709-082531-tohb1e19 [group_name admins CAN_MANAGE] -> [user_name [email protected] CAN_ATTACH_TO]
08:25 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_saqrx: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_saqrx
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_saqrx', metastore_id=None, name='ucx_saqrx', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:25 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext.
08:25 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext.
08:25 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext.
08:25 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: <databricks.sdk.service._internal.Wait object at 0x7f832b0b9fc0>
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='[email protected]', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='[email protected]')], entitlements=[], external_id=None, groups=[], id='8276582481394398', name=Name(family_name=None, given_name='[email protected]'), roles=[], schemas=[<UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_USER: 'urn:ietf:params:scim:schemas:core:2.0:User'>, <UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_EXTENSION_WORKSPACE_2_0_USER: 'urn:ietf:params:scim:schemas:extension:workspace:2.0:User'>], user_name='[email protected]')
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0709-082544-s6i2lkh5 [group_name admins CAN_MANAGE] -> [user_name [email protected] CAN_ATTACH_TO]
08:25 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas
08:25 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Preparing a list of UC catalogs and schema to be created
08:25 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_saqrx.tables] fetching tables inventory
08:25 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_saqrx.tables] crawling new batch for tables
08:25 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sesab] listing tables
08:26 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sesab.ucx_tass6] fetching table metadata
08:26 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_saqrx.tables] found 1 new records for tables
08:26 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_ch2et.ucx_sesab') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_ch2et.ucx_sesab TO `ucx_GRYog`
08:26 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_ch2et.ucx_sesab') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_ch2et.ucx_sesab TO `[email protected]`
08:27 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_ch2et.ucx_sesab') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_ch2et.ucx_sesab TO `5a11359f-ba1f-483f-8e00-0fe55ec003ed`
08:27 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_ch2et.ucx_sesab') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_ch2et.ucx_sesab TO `0a330eb5-dd51-4d97-b6e4-c474356b1d5d`
08:27 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_ch2et.ucx_sesab') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_ch2et.ucx_sesab TO `[email protected]`
08:27 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_ch2et.ucx_sesab') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_ch2et.ucx_sesab TO `[email protected]`
08:28 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_ch2et') using SQL query: GRANT USE CATALOG ON CATALOG ucx_ch2et TO `[email protected]`
08:28 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_ch2et') using SQL query: GRANT USE CATALOG ON CATALOG ucx_ch2et TO `ucx_GRYog`
08:28 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_ch2et') using SQL query: GRANT USE CATALOG ON CATALOG ucx_ch2et TO `5a11359f-ba1f-483f-8e00-0fe55ec003ed`
08:28 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_ch2et') using SQL query: GRANT USE CATALOG ON CATALOG ucx_ch2et TO `[email protected]`
08:28 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_ch2et') using SQL query: GRANT USE CATALOG ON CATALOG ucx_ch2et TO `[email protected]`
08:28 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_ch2et') using SQL query: GRANT USE CATALOG ON CATALOG ucx_ch2et TO `0a330eb5-dd51-4d97-b6e4-c474356b1d5d`
08:24 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/fQ3q
08:24 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sesab: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sesab
08:24 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sesab', metastore_id=None, name='ucx_sesab', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:25 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_sesab.ucx_tass6: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sesab/ucx_tass6
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_sesab.ucx_tass6', metastore_id=None, name='ucx_tass6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_sesab', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/fQ3q', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1720513520816, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ch2et', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ch2et', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720513520816, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:25 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_ch2et.ucx_sesab: https://DATABRICKS_HOST/explore/data/ucx_ch2et/ucx_sesab
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ch2et', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ch2et.ucx_sesab', metastore_id=None, name='ucx_sesab', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: <databricks.sdk.service._internal.Wait object at 0x7f832b100a00>
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='[email protected]', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='[email protected]')], entitlements=[], external_id=None, groups=[], id='6338907467934090', name=Name(family_name=None, given_name='[email protected]'), roles=[], schemas=[<UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_USER: 'urn:ietf:params:scim:schemas:core:2.0:User'>, <UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_EXTENSION_WORKSPACE_2_0_USER: 'urn:ietf:params:scim:schemas:extension:workspace:2.0:User'>], user_name='[email protected]')
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0709-082531-tohb1e19 [group_name admins CAN_MANAGE] -> [user_name [email protected] CAN_ATTACH_TO]
08:25 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_saqrx: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_saqrx
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_saqrx', metastore_id=None, name='ucx_saqrx', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:25 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext.
08:25 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext.
08:25 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext.
08:25 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster fixture: <databricks.sdk.service._internal.Wait object at 0x7f832b0b9fc0>
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added workspace user fixture: User(active=True, display_name='[email protected]', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='[email protected]')], entitlements=[], external_id=None, groups=[], id='8276582481394398', name=Name(family_name=None, given_name='[email protected]'), roles=[], schemas=[<UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_USER: 'urn:ietf:params:scim:schemas:core:2.0:User'>, <UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_EXTENSION_WORKSPACE_2_0_USER: 'urn:ietf:params:scim:schemas:extension:workspace:2.0:User'>], user_name='[email protected]')
08:25 DEBUG [databricks.labs.ucx.mixins.fixtures] added cluster permissions fixture: 0709-082544-s6i2lkh5 [group_name admins CAN_MANAGE] -> [user_name [email protected] CAN_ATTACH_TO]
08:25 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Listing existing UC catalogs and schemas
08:25 INFO [databricks.labs.ucx.hive_metastore.catalog_schema] Preparing a list of UC catalogs and schema to be created
08:25 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_saqrx.tables] fetching tables inventory
08:25 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_saqrx.tables] crawling new batch for tables
08:25 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sesab] listing tables
08:26 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_sesab.ucx_tass6] fetching table metadata
08:26 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_saqrx.tables] found 1 new records for tables
08:26 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_ch2et.ucx_sesab') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_ch2et.ucx_sesab TO `ucx_GRYog`
08:26 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_ch2et.ucx_sesab') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_ch2et.ucx_sesab TO `[email protected]`
08:27 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_ch2et.ucx_sesab') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_ch2et.ucx_sesab TO `5a11359f-ba1f-483f-8e00-0fe55ec003ed`
08:27 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_ch2et.ucx_sesab') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_ch2et.ucx_sesab TO `0a330eb5-dd51-4d97-b6e4-c474356b1d5d`
08:27 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_ch2et.ucx_sesab') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_ch2et.ucx_sesab TO `[email protected]`
08:27 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('DATABASE', 'ucx_ch2et.ucx_sesab') using SQL query: GRANT USE SCHEMA ON DATABASE ucx_ch2et.ucx_sesab TO `[email protected]`
08:28 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_ch2et') using SQL query: GRANT USE CATALOG ON CATALOG ucx_ch2et TO `[email protected]`
08:28 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_ch2et') using SQL query: GRANT USE CATALOG ON CATALOG ucx_ch2et TO `ucx_GRYog`
08:28 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_ch2et') using SQL query: GRANT USE CATALOG ON CATALOG ucx_ch2et TO `5a11359f-ba1f-483f-8e00-0fe55ec003ed`
08:28 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_ch2et') using SQL query: GRANT USE CATALOG ON CATALOG ucx_ch2et TO `[email protected]`
08:28 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_ch2et') using SQL query: GRANT USE CATALOG ON CATALOG ucx_ch2et TO `[email protected]`
08:28 DEBUG [databricks.labs.ucx.hive_metastore.catalog_schema] Migrating acls on ('CATALOG', 'ucx_ch2et') using SQL query: GRANT USE CATALOG ON CATALOG ucx_ch2et TO `0a330eb5-dd51-4d97-b6e4-c474356b1d5d`
08:28 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 2 cluster permissions fixtures
08:28 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster permissions fixture: 0709-082531-tohb1e19 [group_name admins CAN_MANAGE] -> [user_name [email protected] CAN_ATTACH_TO]
08:28 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster permissions fixture: 0709-082544-s6i2lkh5 [group_name admins CAN_MANAGE] -> [user_name [email protected] CAN_ATTACH_TO]
08:28 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures
08:28 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1720513520816, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ch2et', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_ch2et', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720513520816, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:28 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures
08:28 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/fQ3q
08:28 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 2 cluster fixtures
08:28 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster fixture: <databricks.sdk.service._internal.Wait object at 0x7f832b100a00>
08:28 DEBUG [databricks.labs.ucx.mixins.fixtures] removing cluster fixture: <databricks.sdk.service._internal.Wait object at 0x7f832b0b9fc0>
08:28 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures
08:28 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures
08:28 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 table fixtures
08:28 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_sesab.ucx_tass6', metastore_id=None, name='ucx_tass6', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_sesab', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/fQ3q', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:29 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures
08:29 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sesab', metastore_id=None, name='ucx_sesab', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:29 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_ch2et', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_ch2et.ucx_sesab', metastore_id=None, name='ucx_sesab', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:29 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_saqrx', metastore_id=None, name='ucx_saqrx', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:29 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 2 workspace user fixtures
08:29 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='[email protected]', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='[email protected]')], entitlements=[], external_id=None, groups=[], id='6338907467934090', name=Name(family_name=None, given_name='[email protected]'), roles=[], schemas=[<UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_USER: 'urn:ietf:params:scim:schemas:core:2.0:User'>, <UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_EXTENSION_WORKSPACE_2_0_USER: 'urn:ietf:params:scim:schemas:extension:workspace:2.0:User'>], user_name='[email protected]')
08:29 DEBUG [databricks.labs.ucx.mixins.fixtures] removing workspace user fixture: User(active=True, display_name='[email protected]', emails=[ComplexValue(display=None, primary=True, ref=None, type='work', value='[email protected]')], entitlements=[], external_id=None, groups=[], id='8276582481394398', name=Name(family_name=None, given_name='[email protected]'), roles=[], schemas=[<UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_CORE_2_0_USER: 'urn:ietf:params:scim:schemas:core:2.0:User'>, <UserSchema.URN_IETF_PARAMS_SCIM_SCHEMAS_EXTENSION_WORKSPACE_2_0_USER: 'urn:ietf:params:scim:schemas:extension:workspace:2.0:User'>], user_name='[email protected]')
[gw8] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
❌ test_migrate_view: databricks.sdk.errors.platform.BadRequest: This transaction attempted to read one or more files that were deleted (for example part-00000-ccb9d9a6-2551-4ac5-ae77-e3dfc93fdaf9.c000.snappy.parquet in the root of the table) by a concurrent update. Please try the operation again. (10m34.389s) 
databricks.sdk.errors.platform.BadRequest: This transaction attempted to read one or more files that were deleted (for example part-00000-ccb9d9a6-2551-4ac5-ae77-e3dfc93fdaf9.c000.snappy.parquet in the root of the table) by a concurrent update. Please try the operation again.
Conflicting commit: {"timestamp":1720514823689,"userId":"481119220561874","userName":"0a330eb5-dd51-4d97-b6e4-c474356b1d5d","operation":"DELETE","operationParameters":{"predicate":["true"]},"readVersion":3,"isolationLevel":"WriteSerializable","isBlindAppend":false,"operationMetrics":{"numRemovedFiles":"1","numRemovedBytes":"1723","numCopiedRows":"0","numDeletionVectorsAdded":"0","numDeletionVectorsRemoved":"0","numAddedChangeFiles":"0","executionTimeMs":"5","numDeletionVectorsUpdated":"0","numDeletedRows":"4","scanTimeMs":"4","numAddedFiles":"0","numAddedBytes":"0","rewriteTimeMs":"0"},"tags":{"noRowsCopied":"true","delta.rowTracking.preserved":"false","restoresDeletedRows":"false"},"engineInfo":"Databricks-Runtime/15.1.x-photon-scala2.12","txnId":"088abb6f-1e0f-4ca0-a6d5-31dcc88e9e64"}
Refer to https://docs.microsoft.com/CLOUD_ENV/databricks/delta/concurrency-control for more details.
[gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
08:39 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_so1qr: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_so1qr
08:39 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_so1qr', metastore_id=None, name='ucx_so1qr', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:40 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_so1qr.ucx_t81bz: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_so1qr/ucx_t81bz
08:40 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_so1qr.ucx_t81bz', metastore_id=None, name='ucx_t81bz', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_so1qr', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/ucx_so1qr/ucx_t81bz', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:40 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_so1qr.ucx_tx12n: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_so1qr/ucx_tx12n
08:40 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_so1qr.ucx_tx12n', metastore_id=None, name='ucx_tx12n', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_so1qr', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.ucx_so1qr.ucx_t81bz', view_dependencies=None)
08:40 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_so1qr.ucx_tnmox: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_so1qr/ucx_tnmox
08:40 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_so1qr.ucx_tnmox', metastore_id=None, name='ucx_tnmox', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_so1qr', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.ucx_so1qr.ucx_tx12n', view_dependencies=None)
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1720514466620, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_clzoi', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_clzoi', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720514466620, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_clzoi.ucx_so1qr: https://DATABRICKS_HOST/explore/data/ucx_clzoi/ucx_so1qr
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_clzoi', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_clzoi.ucx_so1qr', metastore_id=None, name='ucx_so1qr', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:41 INFO [tests.integration.hive_metastore.test_migrate] dst_catalog=ucx_clzoi, managed_table=hive_metastore.ucx_so1qr.ucx_t81bz
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_scixj: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_scixj
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_scixj', metastore_id=None, name='ucx_scixj', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:41 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext.
08:41 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext.
08:41 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext.
08:41 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] fetching migration_status inventory
08:41 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] crawling new batch for migration_status
08:41 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.tables] fetching tables inventory
08:41 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.tables] crawling new batch for tables
08:41 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_so1qr] listing tables
08:41 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_so1qr.ucx_t81bz] fetching table metadata
08:41 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_so1qr.ucx_tnmox] fetching table metadata
08:41 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_so1qr.ucx_tx12n] fetching table metadata
08:41 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_so1qr.view3] fetching table metadata
08:42 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.tables] found 4 new records for tables
08:42 WARNING [databricks.labs.ucx.hive_metastore.migration_status] Catalog ucx_cdpqm no longer exists. Skipping checking its migration status.
08:42 WARNING [databricks.labs.ucx.hive_metastore.migration_status] Catalog ucx_ciwiz no longer exists. Skipping checking its migration status.
08:42 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] found 4 new records for migration_status
08:43 WARNING [databricks.labs.ucx.hive_metastore.migration_status] Catalog ucx_cwcse no longer exists. Skipping checking its migration status.
08:43 WARNING [databricks.labs.ucx.hive_metastore.migration_status] Catalog ucx_cym2g no longer exists. Skipping checking its migration status.
08:43 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.tables] fetching tables inventory
08:43 DEBUG [databricks.labs.ucx.hive_metastore.table_migrate] Migrating managed table hive_metastore.ucx_so1qr.ucx_t81bz to ucx_clzoi.ucx_so1qr.ucx_t81bz using SQL query: CREATE TABLE IF NOT EXISTS ucx_clzoi.ucx_so1qr.ucx_t81bz DEEP CLONE hive_metastore.ucx_so1qr.ucx_t81bz;
08:44 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] fetching migration_status inventory
08:44 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.tables] fetching tables inventory
08:45 INFO [databricks.labs.ucx.hive_metastore.mapping] The intended target for hive_metastore.ucx_so1qr.ucx_t81bz, ucx_clzoi.ucx_so1qr.ucx_t81bz, already exists.
08:45 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] fetching migration_status inventory
08:45 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] crawling new batch for migration_status
08:45 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.tables] fetching tables inventory
08:45 WARNING [databricks.labs.ucx.hive_metastore.migration_status] Catalog ucx_cenyn no longer exists. Skipping checking its migration status.
08:46 WARNING [databricks.labs.ucx.hive_metastore.migration_status] Catalog ucx_czc4s no longer exists. Skipping checking its migration status.
08:46 INFO [databricks.labs.ucx.hive_metastore.migration_status] ucx_so1qr.ucx_t81bz is set as migrated
08:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] found 4 new records for migration_status
08:47 ERROR [databricks.labs.blueprint.parallel] migrate views(ViewToMigrate(src=Table(catalog='hive_metastore', database='ucx_so1qr', name='ucx_tx12n', object_type='VIEW', table_format='UNKNOWN', location=None, view_text='SELECT * FROM hive_metastore.ucx_so1qr.ucx_t81bz', upgraded_to=None, storage_properties='{serialization.format: 1}', is_partitioned=False), rule=Rule(workspace_name='workspace', catalog_name='ucx_clzoi', src_schema='ucx_so1qr', dst_schema='ucx_so1qr', src_table='ucx_tx12n', dst_table='ucx_tx12n', recon_tolerance_percent=0, compare_rows=False)), []) task failed: This transaction attempted to read one or more files that were deleted (for example part-00000-ccb9d9a6-2551-4ac5-ae77-e3dfc93fdaf9.c000.snappy.parquet in the root of the table) by a concurrent update. Please try the operation again.
Conflicting commit: {"timestamp":1720514823689,"userId":"481119220561874","userName":"0a330eb5-dd51-4d97-b6e4-c474356b1d5d","operation":"DELETE","operationParameters":{"predicate":["true"]},"readVersion":3,"isolationLevel":"WriteSerializable","isBlindAppend":false,"operationMetrics":{"numRemovedFiles":"1","numRemovedBytes":"1723","numCopiedRows":"0","numDeletionVectorsAdded":"0","numDeletionVectorsRemoved":"0","numAddedChangeFiles":"0","executionTimeMs":"5","numDeletionVectorsUpdated":"0","numDeletedRows":"4","scanTimeMs":"4","numAddedFiles":"0","numAddedBytes":"0","rewriteTimeMs":"0"},"tags":{"noRowsCopied":"true","delta.rowTracking.preserved":"false","restoresDeletedRows":"false"},"engineInfo":"Databricks-Runtime/15.1.x-photon-scala2.12","txnId":"088abb6f-1e0f-4ca0-a6d5-31dcc88e9e64"}
Refer to https://docs.microsoft.com/CLOUD_ENV/databricks/delta/concurrency-control for more details.
Traceback (most recent call last):
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 158, in inner
    return func(*args, **kwargs), None
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/hive_metastore/table_migrate.py", line 200, in _migrate_view
    if self._view_can_be_migrated(src_view):
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/hive_metastore/table_migrate.py", line 208, in _view_can_be_migrated
    if not self.index_full_refresh().get(table.schema, table.name):
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/hive_metastore/table_migrate.py", line 65, in index_full_refresh
    self._migration_status_refresher.reset()
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/framework/crawlers.py", line 55, in reset
    self._exec(f"DELETE FROM {self.full_name}")
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/lsql/backends.py", line 197, in execute
    self._sql.execute(sql, catalog=catalog, schema=schema)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/lsql/core.py", line 266, in execute
    self._raise_if_needed(result_status)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/lsql/core.py", line 480, in _raise_if_needed
    raise error_class(error_message)
databricks.sdk.errors.platform.BadRequest: This transaction attempted to read one or more files that were deleted (for example part-00000-ccb9d9a6-2551-4ac5-ae77-e3dfc93fdaf9.c000.snappy.parquet in the root of the table) by a concurrent update. Please try the operation again.
Conflicting commit: {"timestamp":1720514823689,"userId":"481119220561874","userName":"0a330eb5-dd51-4d97-b6e4-c474356b1d5d","operation":"DELETE","operationParameters":{"predicate":["true"]},"readVersion":3,"isolationLevel":"WriteSerializable","isBlindAppend":false,"operationMetrics":{"numRemovedFiles":"1","numRemovedBytes":"1723","numCopiedRows":"0","numDeletionVectorsAdded":"0","numDeletionVectorsRemoved":"0","numAddedChangeFiles":"0","executionTimeMs":"5","numDeletionVectorsUpdated":"0","numDeletedRows":"4","scanTimeMs":"4","numAddedFiles":"0","numAddedBytes":"0","rewriteTimeMs":"0"},"tags":{"noRowsCopied":"true","delta.rowTracking.preserved":"false","restoresDeletedRows":"false"},"engineInfo":"Databricks-Runtime/15.1.x-photon-scala2.12","txnId":"088abb6f-1e0f-4ca0-a6d5-31dcc88e9e64"}
Refer to https://docs.microsoft.com/CLOUD_ENV/databricks/delta/concurrency-control for more details.
08:47 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] fetching migration_status inventory
08:47 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] crawling new batch for migration_status
08:47 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.tables] fetching tables inventory
08:48 INFO [databricks.labs.ucx.hive_metastore.migration_status] ucx_so1qr.ucx_t81bz is set as migrated
08:48 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] found 4 new records for migration_status
08:48 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] fetching migration_status inventory
08:48 DEBUG [databricks.labs.ucx.hive_metastore.table_migrate] Migrating view hive_metastore.ucx_so1qr.view3 to using SQL query: CREATE VIEW IF NOT EXISTS ucx_clzoi.ucx_so1qr.view3 (col1, col2) TBLPROPERTIES ('transient_lastDdlTime'='1720514461') AS SELECT * FROM ucx_clzoi.ucx_so1qr.ucx_t81bz
08:49 ERROR [databricks.labs.blueprint.parallel] More than half 'migrate views' tasks failed: 50% results available (1/2). Took 0:02:21.324846
08:39 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_so1qr: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_so1qr
08:39 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_so1qr', metastore_id=None, name='ucx_so1qr', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:40 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_so1qr.ucx_t81bz: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_so1qr/ucx_t81bz
08:40 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_so1qr.ucx_t81bz', metastore_id=None, name='ucx_t81bz', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_so1qr', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/ucx_so1qr/ucx_t81bz', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:40 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_so1qr.ucx_tx12n: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_so1qr/ucx_tx12n
08:40 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_so1qr.ucx_tx12n', metastore_id=None, name='ucx_tx12n', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_so1qr', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.ucx_so1qr.ucx_t81bz', view_dependencies=None)
08:40 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.ucx_so1qr.ucx_tnmox: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_so1qr/ucx_tnmox
08:40 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_so1qr.ucx_tnmox', metastore_id=None, name='ucx_tnmox', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_so1qr', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.ucx_so1qr.ucx_tx12n', view_dependencies=None)
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1720514466620, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_clzoi', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_clzoi', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720514466620, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_clzoi.ucx_so1qr: https://DATABRICKS_HOST/explore/data/ucx_clzoi/ucx_so1qr
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_clzoi', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_clzoi.ucx_so1qr', metastore_id=None, name='ucx_so1qr', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:41 INFO [tests.integration.hive_metastore.test_migrate] dst_catalog=ucx_clzoi, managed_table=hive_metastore.ucx_so1qr.ucx_t81bz
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_scixj: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_scixj
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_scixj', metastore_id=None, name='ucx_scixj', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:41 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext.
08:41 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext.
08:41 WARNING [databricks.labs.ucx.assessment.secrets] Secret in config stored in plaintext.
08:41 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] fetching migration_status inventory
08:41 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] crawling new batch for migration_status
08:41 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.tables] fetching tables inventory
08:41 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.tables] crawling new batch for tables
08:41 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_so1qr] listing tables
08:41 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_so1qr.ucx_t81bz] fetching table metadata
08:41 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_so1qr.ucx_tnmox] fetching table metadata
08:41 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_so1qr.ucx_tx12n] fetching table metadata
08:41 DEBUG [databricks.labs.ucx.hive_metastore.tables] [hive_metastore.ucx_so1qr.view3] fetching table metadata
08:42 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.tables] found 4 new records for tables
08:42 WARNING [databricks.labs.ucx.hive_metastore.migration_status] Catalog ucx_cdpqm no longer exists. Skipping checking its migration status.
08:42 WARNING [databricks.labs.ucx.hive_metastore.migration_status] Catalog ucx_ciwiz no longer exists. Skipping checking its migration status.
08:42 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] found 4 new records for migration_status
08:43 WARNING [databricks.labs.ucx.hive_metastore.migration_status] Catalog ucx_cwcse no longer exists. Skipping checking its migration status.
08:43 WARNING [databricks.labs.ucx.hive_metastore.migration_status] Catalog ucx_cym2g no longer exists. Skipping checking its migration status.
08:43 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.tables] fetching tables inventory
08:43 DEBUG [databricks.labs.ucx.hive_metastore.table_migrate] Migrating managed table hive_metastore.ucx_so1qr.ucx_t81bz to ucx_clzoi.ucx_so1qr.ucx_t81bz using SQL query: CREATE TABLE IF NOT EXISTS ucx_clzoi.ucx_so1qr.ucx_t81bz DEEP CLONE hive_metastore.ucx_so1qr.ucx_t81bz;
08:44 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] fetching migration_status inventory
08:44 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.tables] fetching tables inventory
08:45 INFO [databricks.labs.ucx.hive_metastore.mapping] The intended target for hive_metastore.ucx_so1qr.ucx_t81bz, ucx_clzoi.ucx_so1qr.ucx_t81bz, already exists.
08:45 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] fetching migration_status inventory
08:45 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] crawling new batch for migration_status
08:45 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.tables] fetching tables inventory
08:45 WARNING [databricks.labs.ucx.hive_metastore.migration_status] Catalog ucx_cenyn no longer exists. Skipping checking its migration status.
08:46 WARNING [databricks.labs.ucx.hive_metastore.migration_status] Catalog ucx_czc4s no longer exists. Skipping checking its migration status.
08:46 INFO [databricks.labs.ucx.hive_metastore.migration_status] ucx_so1qr.ucx_t81bz is set as migrated
08:46 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] found 4 new records for migration_status
08:47 ERROR [databricks.labs.blueprint.parallel] migrate views(ViewToMigrate(src=Table(catalog='hive_metastore', database='ucx_so1qr', name='ucx_tx12n', object_type='VIEW', table_format='UNKNOWN', location=None, view_text='SELECT * FROM hive_metastore.ucx_so1qr.ucx_t81bz', upgraded_to=None, storage_properties='{serialization.format: 1}', is_partitioned=False), rule=Rule(workspace_name='workspace', catalog_name='ucx_clzoi', src_schema='ucx_so1qr', dst_schema='ucx_so1qr', src_table='ucx_tx12n', dst_table='ucx_tx12n', recon_tolerance_percent=0, compare_rows=False)), []) task failed: This transaction attempted to read one or more files that were deleted (for example part-00000-ccb9d9a6-2551-4ac5-ae77-e3dfc93fdaf9.c000.snappy.parquet in the root of the table) by a concurrent update. Please try the operation again.
Conflicting commit: {"timestamp":1720514823689,"userId":"481119220561874","userName":"0a330eb5-dd51-4d97-b6e4-c474356b1d5d","operation":"DELETE","operationParameters":{"predicate":["true"]},"readVersion":3,"isolationLevel":"WriteSerializable","isBlindAppend":false,"operationMetrics":{"numRemovedFiles":"1","numRemovedBytes":"1723","numCopiedRows":"0","numDeletionVectorsAdded":"0","numDeletionVectorsRemoved":"0","numAddedChangeFiles":"0","executionTimeMs":"5","numDeletionVectorsUpdated":"0","numDeletedRows":"4","scanTimeMs":"4","numAddedFiles":"0","numAddedBytes":"0","rewriteTimeMs":"0"},"tags":{"noRowsCopied":"true","delta.rowTracking.preserved":"false","restoresDeletedRows":"false"},"engineInfo":"Databricks-Runtime/15.1.x-photon-scala2.12","txnId":"088abb6f-1e0f-4ca0-a6d5-31dcc88e9e64"}
Refer to https://docs.microsoft.com/CLOUD_ENV/databricks/delta/concurrency-control for more details.
Traceback (most recent call last):
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/blueprint/parallel.py", line 158, in inner
    return func(*args, **kwargs), None
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/hive_metastore/table_migrate.py", line 200, in _migrate_view
    if self._view_can_be_migrated(src_view):
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/hive_metastore/table_migrate.py", line 208, in _view_can_be_migrated
    if not self.index_full_refresh().get(table.schema, table.name):
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/hive_metastore/table_migrate.py", line 65, in index_full_refresh
    self._migration_status_refresher.reset()
  File "/home/runner/work/ucx/ucx/src/databricks/labs/ucx/framework/crawlers.py", line 55, in reset
    self._exec(f"DELETE FROM {self.full_name}")
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/lsql/backends.py", line 197, in execute
    self._sql.execute(sql, catalog=catalog, schema=schema)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/lsql/core.py", line 266, in execute
    self._raise_if_needed(result_status)
  File "/home/runner/work/ucx/ucx/.venv/lib/python3.10/site-packages/databricks/labs/lsql/core.py", line 480, in _raise_if_needed
    raise error_class(error_message)
databricks.sdk.errors.platform.BadRequest: This transaction attempted to read one or more files that were deleted (for example part-00000-ccb9d9a6-2551-4ac5-ae77-e3dfc93fdaf9.c000.snappy.parquet in the root of the table) by a concurrent update. Please try the operation again.
Conflicting commit: {"timestamp":1720514823689,"userId":"481119220561874","userName":"0a330eb5-dd51-4d97-b6e4-c474356b1d5d","operation":"DELETE","operationParameters":{"predicate":["true"]},"readVersion":3,"isolationLevel":"WriteSerializable","isBlindAppend":false,"operationMetrics":{"numRemovedFiles":"1","numRemovedBytes":"1723","numCopiedRows":"0","numDeletionVectorsAdded":"0","numDeletionVectorsRemoved":"0","numAddedChangeFiles":"0","executionTimeMs":"5","numDeletionVectorsUpdated":"0","numDeletedRows":"4","scanTimeMs":"4","numAddedFiles":"0","numAddedBytes":"0","rewriteTimeMs":"0"},"tags":{"noRowsCopied":"true","delta.rowTracking.preserved":"false","restoresDeletedRows":"false"},"engineInfo":"Databricks-Runtime/15.1.x-photon-scala2.12","txnId":"088abb6f-1e0f-4ca0-a6d5-31dcc88e9e64"}
Refer to https://docs.microsoft.com/CLOUD_ENV/databricks/delta/concurrency-control for more details.
08:47 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] fetching migration_status inventory
08:47 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] crawling new batch for migration_status
08:47 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.tables] fetching tables inventory
08:48 INFO [databricks.labs.ucx.hive_metastore.migration_status] ucx_so1qr.ucx_t81bz is set as migrated
08:48 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] found 4 new records for migration_status
08:48 DEBUG [databricks.labs.ucx.framework.crawlers] [hive_metastore.ucx_scixj.migration_status] fetching migration_status inventory
08:48 DEBUG [databricks.labs.ucx.hive_metastore.table_migrate] Migrating view hive_metastore.ucx_so1qr.view3 to using SQL query: CREATE VIEW IF NOT EXISTS ucx_clzoi.ucx_so1qr.view3 (col1, col2) TBLPROPERTIES ('transient_lastDdlTime'='1720514461') AS SELECT * FROM ucx_clzoi.ucx_so1qr.ucx_t81bz
08:49 ERROR [databricks.labs.blueprint.parallel] More than half 'migrate views' tasks failed: 50% results available (1/2). Took 0:02:21.324846
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1720514466620, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_clzoi', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_clzoi', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720514466620, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 table fixtures
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_so1qr.ucx_t81bz', metastore_id=None, name='ucx_t81bz', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_so1qr', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/ucx_so1qr/ucx_t81bz', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_so1qr.ucx_tx12n', metastore_id=None, name='ucx_tx12n', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_so1qr', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.ucx_so1qr.ucx_t81bz', view_dependencies=None)
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_so1qr.ucx_tx12n', metastore_id=None, name='ucx_tx12n', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_so1qr', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.ucx_so1qr.ucx_t81bz', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.ucx_so1qr.ucx_tx12n is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_so1qr.ucx_tnmox', metastore_id=None, name='ucx_tnmox', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_so1qr', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.ucx_so1qr.ucx_tx12n', view_dependencies=None)
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.ucx_so1qr.ucx_tnmox', metastore_id=None, name='ucx_tnmox', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='ucx_so1qr', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.ucx_so1qr.ucx_tx12n', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.ucx_so1qr.ucx_tnmox is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_so1qr', metastore_id=None, name='ucx_so1qr', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:50 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_clzoi', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_clzoi.ucx_so1qr', metastore_id=None, name='ucx_so1qr', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:50 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_scixj', metastore_id=None, name='ucx_scixj', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
[gw1] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
❌ test_repair_run_workflow_job: TimeoutError: Timed out after 0:05:00 (6m0.701s) 
TimeoutError: Timed out after 0:05:00
[gw8] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
08:47 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_szwxw: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_szwxw
08:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_szwxw', metastore_id=None, name='ucx_szwxw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:47 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_szwxw: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_szwxw
08:47 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_szwxw', metastore_id=None, name='ucx_szwxw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:52 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.yCvh/config.yml) doesn't exist.
08:52 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
08:52 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
08:52 INFO [databricks.labs.ucx.install] Fetching installations...
08:52 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
08:52 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:52 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:52 INFO [databricks.labs.ucx.install] Deleting UCX v0.28.1+3120240709085257 from https://DATABRICKS_HOST
08:52 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_szwxw
08:53 INFO [databricks.labs.ucx.install] Deleting jobs
08:53 ERROR [databricks.labs.ucx.install] No jobs present or jobs already deleted
08:53 INFO [databricks.labs.ucx.install] Deleting cluster policy
08:53 INFO [databricks.labs.ucx.install] Deleting secret scope
08:53 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
08:53 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures
08:53 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures
08:53 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures
08:53 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures
08:53 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures
08:53 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures
08:53 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_szwxw', metastore_id=None, name='ucx_szwxw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
[gw8] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
❌ test_table_migration_job_refreshes_migration_status[hiveserde-migrate-external-hiveserde-tables-in-place-experimental]: TimeoutError: timed out after 0:20:00: (26m24.522s) 
TimeoutError: timed out after 0:20:00:
08:30 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/0gta
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_w9ms0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_w9ms0', metastore_id=None, name='hiveserde_in_place_w9ms0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_w9ms0.parquet_serde_w9ms0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_w9ms0/parquet_serde_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_w9ms0.parquet_serde_w9ms0', metastore_id=None, name='parquet_serde_w9ms0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='hiveserde_in_place_w9ms0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0/parquet_serde_w9ms0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_w9ms0.orc_serde_w9ms0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_w9ms0/orc_serde_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_w9ms0.orc_serde_w9ms0', metastore_id=None, name='orc_serde_w9ms0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='hiveserde_in_place_w9ms0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0/orc_serde_w9ms0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_w9ms0.avro_serde_w9ms0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_w9ms0/avro_serde_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_w9ms0.avro_serde_w9ms0', metastore_id=None, name='avro_serde_w9ms0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='hiveserde_in_place_w9ms0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0/avro_serde_w9ms0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1720513892809, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cehjg', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cehjg', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720513892809, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cehjg.hiveserde_in_place_w9ms0: https://DATABRICKS_HOST/explore/data/ucx_cehjg/hiveserde_in_place_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cehjg', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cehjg.hiveserde_in_place_w9ms0', metastore_id=None, name='hiveserde_in_place_w9ms0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_st6eg: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_st6eg
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_st6eg', metastore_id=None, name='ucx_st6eg', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
[gw2] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
08:30 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/0gta
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_w9ms0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_w9ms0', metastore_id=None, name='hiveserde_in_place_w9ms0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_w9ms0.parquet_serde_w9ms0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_w9ms0/parquet_serde_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_w9ms0.parquet_serde_w9ms0', metastore_id=None, name='parquet_serde_w9ms0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='hiveserde_in_place_w9ms0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0/parquet_serde_w9ms0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_w9ms0.orc_serde_w9ms0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_w9ms0/orc_serde_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_w9ms0.orc_serde_w9ms0', metastore_id=None, name='orc_serde_w9ms0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='hiveserde_in_place_w9ms0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0/orc_serde_w9ms0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_w9ms0.avro_serde_w9ms0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_w9ms0/avro_serde_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_w9ms0.avro_serde_w9ms0', metastore_id=None, name='avro_serde_w9ms0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='hiveserde_in_place_w9ms0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0/avro_serde_w9ms0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1720513892809, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cehjg', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cehjg', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720513892809, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cehjg.hiveserde_in_place_w9ms0: https://DATABRICKS_HOST/explore/data/ucx_cehjg/hiveserde_in_place_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cehjg', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cehjg.hiveserde_in_place_w9ms0', metastore_id=None, name='hiveserde_in_place_w9ms0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_st6eg: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_st6eg
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_st6eg', metastore_id=None, name='ucx_st6eg', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:33 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.edDQ/config.yml) doesn't exist.
08:33 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
08:33 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
08:33 INFO [databricks.labs.ucx.install] Fetching installations...
08:33 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.edDQ is corrupted. Skipping...
08:33 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
08:33 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:33 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:33 INFO [databricks.labs.ucx.install] Installing UCX v0.28.1+3120240709083331
08:33 INFO [databricks.labs.ucx.install] Creating ucx schemas...
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
08:35 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.edDQ/README for the next steps.
08:35 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/649681000880133
08:30 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/0gta
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_w9ms0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_w9ms0', metastore_id=None, name='hiveserde_in_place_w9ms0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_w9ms0.parquet_serde_w9ms0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_w9ms0/parquet_serde_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_w9ms0.parquet_serde_w9ms0', metastore_id=None, name='parquet_serde_w9ms0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='hiveserde_in_place_w9ms0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0/parquet_serde_w9ms0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_w9ms0.orc_serde_w9ms0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_w9ms0/orc_serde_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_w9ms0.orc_serde_w9ms0', metastore_id=None, name='orc_serde_w9ms0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='hiveserde_in_place_w9ms0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0/orc_serde_w9ms0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_w9ms0.avro_serde_w9ms0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_w9ms0/avro_serde_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_w9ms0.avro_serde_w9ms0', metastore_id=None, name='avro_serde_w9ms0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='hiveserde_in_place_w9ms0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0/avro_serde_w9ms0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1720513892809, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cehjg', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cehjg', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720513892809, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cehjg.hiveserde_in_place_w9ms0: https://DATABRICKS_HOST/explore/data/ucx_cehjg/hiveserde_in_place_w9ms0
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cehjg', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cehjg.hiveserde_in_place_w9ms0', metastore_id=None, name='hiveserde_in_place_w9ms0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:31 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_st6eg: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_st6eg
08:31 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_st6eg', metastore_id=None, name='ucx_st6eg', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:33 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.edDQ/config.yml) doesn't exist.
08:33 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
08:33 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
08:33 INFO [databricks.labs.ucx.install] Fetching installations...
08:33 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.edDQ is corrupted. Skipping...
08:33 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
08:33 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:33 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:33 INFO [databricks.labs.ucx.install] Installing UCX v0.28.1+3120240709083331
08:33 INFO [databricks.labs.ucx.install] Creating ucx schemas...
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
08:35 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
08:35 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.edDQ/README for the next steps.
08:35 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/649681000880133
08:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_storage_dir fixtures
08:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0
08:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures
08:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/0gta
08:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures
08:55 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures
08:55 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1720513892809, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cehjg', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cehjg', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1720513892809, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:55 INFO [databricks.labs.ucx.install] Deleting UCX v0.28.1+3120240709083331 from https://DATABRICKS_HOST
08:55 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_st6eg
08:56 INFO [databricks.labs.ucx.install] Deleting jobs
08:56 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=494857944970594.
08:56 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=318591178992309.
08:56 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=649681000880133.
08:56 INFO [databricks.labs.ucx.install] Deleting failing job_id=87983122521186.
08:56 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=885317281979808.
08:56 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=165283317789451.
08:56 INFO [databricks.labs.ucx.install] Deleting assessment job_id=511880102109108.
08:56 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=298671658568460.
08:56 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=812475151142106.
08:56 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=987033500849551.
08:56 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=257319054316884.
08:56 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=737014025037899.
08:56 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=472410102596599.
08:56 INFO [databricks.labs.ucx.install] Deleting cluster policy
08:56 INFO [databricks.labs.ucx.install] Deleting secret scope
08:56 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
08:56 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures
08:56 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures
08:56 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures
08:56 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures
08:56 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 table fixtures
08:56 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_w9ms0.parquet_serde_w9ms0', metastore_id=None, name='parquet_serde_w9ms0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='hiveserde_in_place_w9ms0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0/parquet_serde_w9ms0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:56 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_w9ms0.orc_serde_w9ms0', metastore_id=None, name='orc_serde_w9ms0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='hiveserde_in_place_w9ms0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0/orc_serde_w9ms0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:56 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_w9ms0.avro_serde_w9ms0', metastore_id=None, name='avro_serde_w9ms0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070909'}, row_filter=None, schema_name='hiveserde_in_place_w9ms0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_w9ms0/avro_serde_w9ms0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:56 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures
08:56 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_w9ms0', metastore_id=None, name='hiveserde_in_place_w9ms0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:56 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cehjg', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cehjg.hiveserde_in_place_w9ms0', metastore_id=None, name='hiveserde_in_place_w9ms0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:56 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_st6eg', metastore_id=None, name='ucx_st6eg', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
[gw2] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python

Running from acceptance #4431

nfx
nfx approved these changes Jul 9, 2024
View reviewed changes
Copy link
Collaborator

@nfx nfx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@nfx nfx merged commit 2f02467 into main Jul 9, 2024
2 of 4 checks passed
@nfx nfx deleted the update-language branch July 9, 2024 09:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nfx nfx nfx approved these changes

@qziyuan qziyuan Awaiting requested review from qziyuan

Assignees

@asnare asnare

Labels
internal this pull request won't appear in release notes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@asnare @nfx