Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adjust type hints in credentials handling for Azure API client #1964

Merged
merged 1 commit into from
Jul 2, 2024
Merged

Adjust type hints in credentials handling for Azure API client #1964

merged 1 commit into from
Jul 2, 2024

Conversation

asnare
Copy link
Contributor

@asnare asnare commented Jul 2, 2024

Changes

This PR is stacked on #1943; it is stacked to verify the type hinting changes via the CI environment.

The situation is:

  • The Databricks SDK has partially implemented type hints for the credentials handlers.

  • The CredentialsProvider is the fundamental type for a callable factory that can be used to obtain credentials.

  • The CredentialsStrategy is the type that can be used for a function that can be used to obtain a provider, given the Config.

  • The @credentials_strategy decorator is used to fuse the two:

    1. It decorates a function that, given a Config, returns a CredentialsProvider.
    2. The decorated function that it returns is a CredentialsStrategy.

  • The type upstream type hinting for the @credentials_strategy decorator is incomplete: the return type is omitted. This confuses IntelliJ/PyCharm a little but not Mypy.

@asnare asnare added the bug label Jul 2, 2024
@asnare asnare self-assigned this Jul 2, 2024
@JCZuurmond JCZuurmond marked this pull request as ready for review July 2, 2024 08:19
@JCZuurmond JCZuurmond requested review from a team and mwojtyczka and removed request for a team July 2, 2024 08:19
JCZuurmond
JCZuurmond approved these changes Jul 2, 2024
View reviewed changes
Copy link
Member

@JCZuurmond JCZuurmond left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

@asnare asnare merged commit 7b9eece into dependabot/pip/databricks-sdk-gte-0.27-and-lt-0.30 Jul 2, 2024
4 of 5 checks passed
@asnare asnare deleted the fix/databricks-upgrade-type-hinting branch July 2, 2024 08:20
@github-actions GitHub Actions
Copy link

github-actions bot commented Jul 2, 2024

❌ 168/172 passed, 4 failed, 24 skipped, 5h40m32s total

❌ test_running_real_workflow_linter_job: TimeoutError: timed out after 0:20:00: (21m49.008s) 
TimeoutError: timed out after 0:20:00:
[gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt/notebook.ipynb
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/211480981591453
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=211480981591453)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_se0cb: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_se0cb
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_se0cb', metastore_id=None, name='ucx_se0cb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:26 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.WEM3/config.yml) doesn't exist.
08:26 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
08:26 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
08:26 INFO [databricks.labs.ucx.install] Fetching installations...
08:26 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
08:26 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:26 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:27 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702082700
08:27 INFO [databricks.labs.ucx.install] Creating ucx schemas...
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
08:28 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.WEM3/README for the next steps.
08:28 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/1108562563424488
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt/notebook.ipynb
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Job: https://DATABRICKS_HOST#job/211480981591453
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added job fixture: CreateResponse(job_id=211480981591453)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_se0cb: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_se0cb
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_se0cb', metastore_id=None, name='ucx_se0cb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:26 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.WEM3/config.yml) doesn't exist.
08:26 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
08:26 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
08:26 INFO [databricks.labs.ucx.install] Fetching installations...
08:26 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
08:26 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:26 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:27 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702082700
08:27 INFO [databricks.labs.ucx.install] Creating ucx schemas...
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
08:27 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
08:28 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.WEM3/README for the next steps.
08:28 DEBUG [databricks.labs.ucx.installer.workflows] starting experimental-workflow-linter job: https://DATABRICKS_HOST#job/1108562563424488
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 job fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing job fixture: CreateResponse(job_id=211480981591453)
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 directory fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing directory fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 notebook fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing notebook fixture: /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt/notebook.ipynb
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while notebook /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt/notebook.ipynb teardown: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/sdk-gITt/notebook.ipynb) doesn't exist.
08:48 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+1920240702082700 from https://DATABRICKS_HOST
08:48 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_se0cb
08:48 INFO [databricks.labs.ucx.install] Deleting jobs
08:48 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=333390159726378.
08:48 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=1108562563424488.
08:48 INFO [databricks.labs.ucx.install] Deleting failing job_id=586867081988849.
08:48 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=286142826937246.
08:48 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=359306207860365.
08:48 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=293368177752281.
08:48 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=722000416273587.
08:48 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=643512032851768.
08:48 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=320282409385093.
08:48 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=308622283731664.
08:48 INFO [databricks.labs.ucx.install] Deleting assessment job_id=122774938980681.
08:48 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=111270810186782.
08:48 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=486066255317487.
08:48 INFO [databricks.labs.ucx.install] Deleting cluster policy
08:48 INFO [databricks.labs.ucx.install] Deleting secret scope
08:48 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 schema fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_se0cb', metastore_id=None, name='ucx_se0cb', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
[gw0] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
❌ test_table_migration_job_cluster_override[regular]: AssertionError: assert False (12m15.845s) 
AssertionError: assert False
 +  where False = <bound method DeployedWorkflows.validate_step of <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f7eacb1e950>>('migrate-tables')
 +    where <bound method DeployedWorkflows.validate_step of <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f7eacb1e950>> = <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f7eacb1e950>.validate_step
 +      where <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f7eacb1e950> = <tests.integration.conftest.TestInstallationContext object at 0x7f7ec48bbc10>.deployed_workflows
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_ch2yw: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_t8hno: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_t8hno
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_t8hno', metastore_id=None, name='ucx_t8hno', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_ch2yw/ucx_t8hno', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tsvtu: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tsvtu
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.JSON: 'JSON'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tsvtu', metastore_id=None, name='ucx_tsvtu', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hbz4', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_ttrpa: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_ttrpa
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_ttrpa', metastore_id=None, name='ucx_ttrpa', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tl5sh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tl5sh
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tl5sh', metastore_id=None, name='ucx_tl5sh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_t8hno', view_dependencies=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tybtq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tybtq
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tybtq', metastore_id=None, name='ucx_tybtq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_tl5sh', view_dependencies=None)
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1719909467678, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cny0o', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909467678, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cny0o.migrate_ch2yw: https://DATABRICKS_HOST/explore/data/ucx_cny0o/migrate_ch2yw
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cny0o', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sswar: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sswar
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sswar', metastore_id=None, name='ucx_sswar', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
[gw9] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_ch2yw: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_t8hno: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_t8hno
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_t8hno', metastore_id=None, name='ucx_t8hno', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_ch2yw/ucx_t8hno', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tsvtu: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tsvtu
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.JSON: 'JSON'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tsvtu', metastore_id=None, name='ucx_tsvtu', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hbz4', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_ttrpa: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_ttrpa
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_ttrpa', metastore_id=None, name='ucx_ttrpa', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tl5sh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tl5sh
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tl5sh', metastore_id=None, name='ucx_tl5sh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_t8hno', view_dependencies=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tybtq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tybtq
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tybtq', metastore_id=None, name='ucx_tybtq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_tl5sh', view_dependencies=None)
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1719909467678, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cny0o', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909467678, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cny0o.migrate_ch2yw: https://DATABRICKS_HOST/explore/data/ucx_cny0o/migrate_ch2yw
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cny0o', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sswar: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sswar
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sswar', metastore_id=None, name='ucx_sswar', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:38 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.MMbu/config.yml) doesn't exist.
08:38 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
08:38 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
08:38 INFO [databricks.labs.ucx.install] Fetching installations...
08:38 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.MMbu is corrupted. Skipping...
08:38 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
08:38 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:38 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:38 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702083859
08:38 INFO [databricks.labs.ucx.install] Creating ucx schemas...
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter
08:39 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.MMbu/README for the next steps.
08:39 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/767010569747575
08:48 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/767010569747575
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_ch2yw: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_t8hno: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_t8hno
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_t8hno', metastore_id=None, name='ucx_t8hno', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_ch2yw/ucx_t8hno', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tsvtu: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tsvtu
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.JSON: 'JSON'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tsvtu', metastore_id=None, name='ucx_tsvtu', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hbz4', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_ttrpa: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_ttrpa
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_ttrpa', metastore_id=None, name='ucx_ttrpa', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tl5sh: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tl5sh
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tl5sh', metastore_id=None, name='ucx_tl5sh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_t8hno', view_dependencies=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_ch2yw.ucx_tybtq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_ch2yw/ucx_tybtq
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tybtq', metastore_id=None, name='ucx_tybtq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_tl5sh', view_dependencies=None)
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1719909467678, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cny0o', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909467678, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cny0o.migrate_ch2yw: https://DATABRICKS_HOST/explore/data/ucx_cny0o/migrate_ch2yw
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cny0o', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:37 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_sswar: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_sswar
08:37 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sswar', metastore_id=None, name='ucx_sswar', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:38 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.MMbu/config.yml) doesn't exist.
08:38 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
08:38 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
08:38 INFO [databricks.labs.ucx.install] Fetching installations...
08:38 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.MMbu is corrupted. Skipping...
08:38 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
08:38 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:38 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:38 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702083859
08:38 INFO [databricks.labs.ucx.install] Creating ucx schemas...
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
08:39 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter
08:39 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.MMbu/README for the next steps.
08:39 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/767010569747575
08:48 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-tables workflow: https://DATABRICKS_HOST#job/767010569747575
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1719909467678, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cny0o', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909467678, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:48 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+1920240702083859 from https://DATABRICKS_HOST
08:48 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_sswar
08:48 INFO [databricks.labs.ucx.install] Deleting jobs
08:48 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=648590615116669.
08:48 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=695545576074879.
08:48 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=1118156651358186.
08:48 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=258219036433500.
08:48 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=767010569747575.
08:48 INFO [databricks.labs.ucx.install] Deleting assessment job_id=314030895358644.
08:48 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=571417885396629.
08:48 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=52443650767869.
08:48 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=372933925240268.
08:48 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=332607072596468.
08:48 INFO [databricks.labs.ucx.install] Deleting failing job_id=148472965339827.
08:48 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=118845183898113.
08:48 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=768054714376730.
08:48 INFO [databricks.labs.ucx.install] Deleting cluster policy
08:48 INFO [databricks.labs.ucx.install] Deleting secret scope
08:48 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 5 table fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_t8hno', metastore_id=None, name='ucx_t8hno', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_ch2yw/ucx_t8hno', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.JSON: 'JSON'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tsvtu', metastore_id=None, name='ucx_tsvtu', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hbz4', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_ttrpa', metastore_id=None, name='ucx_ttrpa', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/3R3u', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tl5sh', metastore_id=None, name='ucx_tl5sh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_t8hno', view_dependencies=None)
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tl5sh', metastore_id=None, name='ucx_tl5sh', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_t8hno', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_ch2yw.ucx_tl5sh is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tybtq', metastore_id=None, name='ucx_tybtq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_tl5sh', view_dependencies=None)
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_ch2yw.ucx_tybtq', metastore_id=None, name='ucx_tybtq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_ch2yw', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_ch2yw.ucx_tl5sh', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_ch2yw.ucx_tybtq is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cny0o', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cny0o.migrate_ch2yw', metastore_id=None, name='migrate_ch2yw', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_sswar', metastore_id=None, name='ucx_sswar', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
[gw9] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
❌ test_table_migration_job_refreshes_migration_status[regular-migrate-tables]: AssertionError: No destination schema found for TableType.VIEW hive_metastore.migrate_e1qqa.ucx_tfnce given migration statuses Row(src_schema='migrate_e1qqa', src_table='ucx_tksdz', dst_catalog='ucx_c1zcx', dst_schema='migrate_e1qqa', dst_table='ucx_tksdz', update_ts='1719909569.860524') (24m2.578s) 
AssertionError: No destination schema found for TableType.VIEW hive_metastore.migrate_e1qqa.ucx_tfnce given migration statuses Row(src_schema='migrate_e1qqa', src_table='ucx_tksdz', dst_catalog='ucx_c1zcx', dst_schema='migrate_e1qqa', dst_table='ucx_tksdz', update_ts='1719909569.860524')
  Row(src_schema='migrate_e1qqa', src_table='ucx_toulq', dst_catalog='ucx_c1zcx', dst_schema='migrate_e1qqa', dst_table='ucx_toulq', update_ts='1719909569.860524')
  Row(src_schema='migrate_e1qqa', src_table='ucx_tv3gb', dst_catalog='ucx_c1zcx', dst_schema='migrate_e1qqa', dst_table='ucx_tv3gb', update_ts='1719909569.860524')
  Row(src_schema='migrate_e1qqa', src_table='ucx_ttjfx', dst_catalog='ucx_c1zcx', dst_schema='migrate_e1qqa', dst_table='ucx_ttjfx', update_ts='1719909569.860524')
  Row(src_schema='migrate_e1qqa', src_table='ucx_tfnce', dst_catalog=None, dst_schema=None, dst_table=None, update_ts='1719909569.860524')
assert 1 == 0
 +  where 1 = len(['No destination schema found for TableType.VIEW hive_metastore.migrate_e1qqa.ucx_tfnce'])
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_e1qqa: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tksdz: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tksdz
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tksdz', metastore_id=None, name='ucx_tksdz', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_e1qqa/ucx_tksdz', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_toulq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_toulq
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.JSON: 'JSON'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_toulq', metastore_id=None, name='ucx_toulq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hqF7', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tv3gb: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tv3gb
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tv3gb', metastore_id=None, name='ucx_tv3gb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_ttjfx: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_ttjfx
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_ttjfx', metastore_id=None, name='ucx_ttjfx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_tksdz', view_dependencies=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tfnce: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tfnce
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tfnce', metastore_id=None, name='ucx_tfnce', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_ttjfx', view_dependencies=None)
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1719908806757, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c1zcx', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719908806757, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c1zcx.migrate_e1qqa: https://DATABRICKS_HOST/explore/data/ucx_c1zcx/migrate_e1qqa
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c1zcx', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_shpgq: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_shpgq
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_shpgq', metastore_id=None, name='ucx_shpgq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
[gw5] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_e1qqa: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tksdz: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tksdz
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tksdz', metastore_id=None, name='ucx_tksdz', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_e1qqa/ucx_tksdz', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_toulq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_toulq
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.JSON: 'JSON'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_toulq', metastore_id=None, name='ucx_toulq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hqF7', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tv3gb: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tv3gb
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tv3gb', metastore_id=None, name='ucx_tv3gb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_ttjfx: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_ttjfx
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_ttjfx', metastore_id=None, name='ucx_ttjfx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_tksdz', view_dependencies=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tfnce: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tfnce
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tfnce', metastore_id=None, name='ucx_tfnce', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_ttjfx', view_dependencies=None)
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1719908806757, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c1zcx', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719908806757, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c1zcx.migrate_e1qqa: https://DATABRICKS_HOST/explore/data/ucx_c1zcx/migrate_e1qqa
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c1zcx', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_shpgq: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_shpgq
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_shpgq', metastore_id=None, name='ucx_shpgq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:27 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lmaO/config.yml) doesn't exist.
08:27 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
08:27 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
08:27 INFO [databricks.labs.ucx.install] Fetching installations...
08:27 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lmaO is corrupted. Skipping...
08:27 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
08:27 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:27 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:27 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702082755
08:27 INFO [databricks.labs.ucx.install] Creating ucx schemas...
08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
08:29 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lmaO/README for the next steps.
08:29 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/3607200715084
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.migrate_e1qqa: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tksdz: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tksdz
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tksdz', metastore_id=None, name='ucx_tksdz', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_e1qqa/ucx_tksdz', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_toulq: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_toulq
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.JSON: 'JSON'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_toulq', metastore_id=None, name='ucx_toulq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hqF7', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tv3gb: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tv3gb
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tv3gb', metastore_id=None, name='ucx_tv3gb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_ttjfx: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_ttjfx
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_ttjfx', metastore_id=None, name='ucx_ttjfx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_tksdz', view_dependencies=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.migrate_e1qqa.ucx_tfnce: https://DATABRICKS_HOST/explore/data/hive_metastore/migrate_e1qqa/ucx_tfnce
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tfnce', metastore_id=None, name='ucx_tfnce', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_ttjfx', view_dependencies=None)
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1719908806757, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c1zcx', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719908806757, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_c1zcx.migrate_e1qqa: https://DATABRICKS_HOST/explore/data/ucx_c1zcx/migrate_e1qqa
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c1zcx', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:26 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_shpgq: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_shpgq
08:26 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_shpgq', metastore_id=None, name='ucx_shpgq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:27 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lmaO/config.yml) doesn't exist.
08:27 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
08:27 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
08:27 INFO [databricks.labs.ucx.install] Fetching installations...
08:27 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lmaO is corrupted. Skipping...
08:27 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
08:27 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:27 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:27 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702082755
08:27 INFO [databricks.labs.ucx.install] Creating ucx schemas...
08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
08:28 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
08:29 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
08:29 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lmaO/README for the next steps.
08:29 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-tables job: https://DATABRICKS_HOST#job/3607200715084
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 make_storage_dir fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures
08:48 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1719908806757, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_c1zcx', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719908806757, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:48 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+1920240702082755 from https://DATABRICKS_HOST
08:48 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_shpgq
08:49 INFO [databricks.labs.ucx.install] Deleting jobs
08:49 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=3607200715084.
08:49 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=603586545779881.
08:49 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=87353882239896.
08:49 INFO [databricks.labs.ucx.install] Deleting assessment job_id=8284000797270.
08:49 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=713932708131772.
08:49 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=1099134119234777.
08:49 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=916981644685126.
08:49 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=212740737613385.
08:49 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=955849797870851.
08:49 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=93904157616609.
08:49 INFO [databricks.labs.ucx.install] Deleting failing job_id=155855087385096.
08:49 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=700614556745144.
08:49 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=144394863092255.
08:49 INFO [databricks.labs.ucx.install] Deleting cluster policy
08:49 INFO [databricks.labs.ucx.install] Deleting secret scope
08:49 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 5 table fixtures
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.DELTA: 'DELTA'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tksdz', metastore_id=None, name='ucx_tksdz', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/user/hive/warehouse/migrate_e1qqa/ucx_tksdz', table_constraints=None, table_id=None, table_type=<TableType.MANAGED: 'MANAGED'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.JSON: 'JSON'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_toulq', metastore_id=None, name='ucx_toulq', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/tmp/ucx_test_hqF7', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=<DataSourceFormat.CSV: 'CSV'>, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tv3gb', metastore_id=None, name='ucx_tv3gb', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/b/VAKn', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_ttjfx', metastore_id=None, name='ucx_ttjfx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_tksdz', view_dependencies=None)
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_ttjfx', metastore_id=None, name='ucx_ttjfx', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_tksdz', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_e1qqa.ucx_ttjfx is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tfnce', metastore_id=None, name='ucx_tfnce', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_ttjfx', view_dependencies=None)
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] ignoring error while table TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.migrate_e1qqa.ucx_tfnce', metastore_id=None, name='ucx_tfnce', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='migrate_e1qqa', sql_path=None, storage_credential_name=None, storage_location=None, table_constraints=None, table_id=None, table_type=<TableType.VIEW: 'VIEW'>, updated_at=None, updated_by=None, view_definition='SELECT * FROM hive_metastore.migrate_e1qqa.ucx_ttjfx', view_dependencies=None) teardown: [WRONG_COMMAND_FOR_OBJECT_TYPE] The operation DROP TABLE requires a EXTERNAL or MANAGED. But hive_metastore.migrate_e1qqa.ucx_tfnce is a VIEW. Use DROP VIEW instead. SQLSTATE: 42809
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_c1zcx', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_c1zcx.migrate_e1qqa', metastore_id=None, name='migrate_e1qqa', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:49 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_shpgq', metastore_id=None, name='ucx_shpgq', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
[gw5] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
❌ test_hiveserde_table_ctas_migration_job[hiveserde]: AssertionError: assert False (11m38.384s) 
AssertionError: assert False
 +  where False = <bound method DeployedWorkflows.validate_step of <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f0b63baebc0>>('migrate-external-tables-ctas')
 +    where <bound method DeployedWorkflows.validate_step of <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f0b63baebc0>> = <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f0b63baebc0>.validate_step
 +      where <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f0b63baebc0> = <tests.integration.conftest.TestInstallationContext object at 0x7f0b62b97be0>.deployed_workflows
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/2BT1
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.parquet_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/parquet_serde_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.parquet_serde_cygj0', metastore_id=None, name='parquet_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/parquet_serde_cygj0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.orc_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/orc_serde_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.orc_serde_cygj0', metastore_id=None, name='orc_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/orc_serde_cygj0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.avro_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/avro_serde_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.avro_serde_cygj0', metastore_id=None, name='avro_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/avro_serde_cygj0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1719909710449, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cipce', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909710449, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cipce.hiveserde_in_place_cygj0: https://DATABRICKS_HOST/explore/data/ucx_cipce/hiveserde_in_place_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cipce', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:42 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_saauj: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_saauj
08:42 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_saauj', metastore_id=None, name='ucx_saauj', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
[gw3] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/2BT1
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.parquet_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/parquet_serde_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.parquet_serde_cygj0', metastore_id=None, name='parquet_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/parquet_serde_cygj0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.orc_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/orc_serde_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.orc_serde_cygj0', metastore_id=None, name='orc_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/orc_serde_cygj0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.avro_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/avro_serde_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.avro_serde_cygj0', metastore_id=None, name='avro_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/avro_serde_cygj0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1719909710449, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cipce', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909710449, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cipce.hiveserde_in_place_cygj0: https://DATABRICKS_HOST/explore/data/ucx_cipce/hiveserde_in_place_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cipce', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:42 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_saauj: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_saauj
08:42 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_saauj', metastore_id=None, name='ucx_saauj', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:43 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Rb5s/config.yml) doesn't exist.
08:43 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
08:43 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
08:43 INFO [databricks.labs.ucx.install] Fetching installations...
08:43 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Rb5s is corrupted. Skipping...
08:43 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
08:43 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:43 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:43 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702084322
08:43 INFO [databricks.labs.ucx.install] Creating ucx schemas...
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
08:44 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Rb5s/README for the next steps.
08:44 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/5663333963439
08:52 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-tables-ctas workflow: https://DATABRICKS_HOST#job/5663333963439
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/2BT1
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.hiveserde_in_place_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.parquet_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/parquet_serde_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.parquet_serde_cygj0', metastore_id=None, name='parquet_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/parquet_serde_cygj0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.orc_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/orc_serde_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.orc_serde_cygj0', metastore_id=None, name='orc_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/orc_serde_cygj0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Table hive_metastore.hiveserde_in_place_cygj0.avro_serde_cygj0: https://DATABRICKS_HOST/explore/data/hive_metastore/hiveserde_in_place_cygj0/avro_serde_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.avro_serde_cygj0', metastore_id=None, name='avro_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/avro_serde_cygj0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1719909710449, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cipce', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909710449, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:41 INFO [databricks.labs.ucx.mixins.fixtures] Schema ucx_cipce.hiveserde_in_place_cygj0: https://DATABRICKS_HOST/explore/data/ucx_cipce/hiveserde_in_place_cygj0
08:41 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cipce', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:42 INFO [databricks.labs.ucx.mixins.fixtures] Schema hive_metastore.ucx_saauj: https://DATABRICKS_HOST/explore/data/hive_metastore/ucx_saauj
08:42 DEBUG [databricks.labs.ucx.mixins.fixtures] added schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_saauj', metastore_id=None, name='ucx_saauj', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:43 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Rb5s/config.yml) doesn't exist.
08:43 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
08:43 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
08:43 INFO [databricks.labs.ucx.install] Fetching installations...
08:43 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Rb5s is corrupted. Skipping...
08:43 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
08:43 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:43 DEBUG [tests.integration.conftest] Waiting for clusters to start...
08:43 INFO [databricks.labs.ucx.install] Installing UCX v0.27.2+1920240702084322
08:43 INFO [databricks.labs.ucx.install] Creating ucx schemas...
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-experimental
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=experimental-workflow-linter
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
08:44 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
08:44 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Rb5s/README for the next steps.
08:44 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/5663333963439
08:52 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-tables-ctas workflow: https://DATABRICKS_HOST#job/5663333963439
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_storage_dir fixtures
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_storage_dir fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 make_dbfs_data_copy fixtures
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing make_dbfs_data_copy fixture: dbfs:/mnt/TEST_MOUNT_NAME/a/b/2BT1
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 cluster fixtures
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 1 catalog fixtures
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing catalog fixture: CatalogInfo(browse_only=False, catalog_type=<CatalogType.MANAGED_CATALOG: 'MANAGED_CATALOG'>, comment='', connection_name=None, created_at=1719909710449, created_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce', isolation_mode=<CatalogIsolationMode.OPEN: 'OPEN'>, metastore_id='8952c1e3-b265-4adf-98c3-6f755e2e1453', name='ucx_cipce', options=None, owner='0a330eb5-dd51-4d97-b6e4-c474356b1d5d', properties=None, provider_name=None, provisioning_info=None, securable_kind=<CatalogInfoSecurableKind.CATALOG_STANDARD: 'CATALOG_STANDARD'>, securable_type='CATALOG', share_name=None, storage_location=None, storage_root=None, updated_at=1719909710449, updated_by='0a330eb5-dd51-4d97-b6e4-c474356b1d5d')
08:52 INFO [databricks.labs.ucx.install] Deleting UCX v0.27.2+1920240702084322 from https://DATABRICKS_HOST
08:52 INFO [databricks.labs.ucx.install] Deleting inventory database ucx_saauj
08:52 INFO [databricks.labs.ucx.install] Deleting jobs
08:52 INFO [databricks.labs.ucx.install] Deleting migrate-tables-in-mounts-experimental job_id=824210279577082.
08:52 INFO [databricks.labs.ucx.install] Deleting failing job_id=710878812724859.
08:52 INFO [databricks.labs.ucx.install] Deleting migrate-data-reconciliation job_id=230362270253763.
08:52 INFO [databricks.labs.ucx.install] Deleting migrate-external-tables-ctas job_id=5663333963439.
08:52 INFO [databricks.labs.ucx.install] Deleting migrate-external-hiveserde-tables-in-place-experimental job_id=895922783195419.
08:52 INFO [databricks.labs.ucx.install] Deleting validate-groups-permissions job_id=133524984557082.
08:52 INFO [databricks.labs.ucx.install] Deleting migrate-groups-experimental job_id=274251280803703.
08:52 INFO [databricks.labs.ucx.install] Deleting experimental-workflow-linter job_id=250963531627626.
08:52 INFO [databricks.labs.ucx.install] Deleting remove-workspace-local-backup-groups job_id=741312385250175.
08:52 INFO [databricks.labs.ucx.install] Deleting assessment job_id=979753242188797.
08:52 INFO [databricks.labs.ucx.install] Deleting migrate-tables job_id=819901583135974.
08:52 INFO [databricks.labs.ucx.install] Deleting scan-tables-in-mounts-experimental job_id=102462560190794.
08:52 INFO [databricks.labs.ucx.install] Deleting migrate-groups job_id=826178644458442.
08:52 INFO [databricks.labs.ucx.install] Deleting cluster policy
08:52 INFO [databricks.labs.ucx.install] Deleting secret scope
08:52 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace user fixtures
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 account group fixtures
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 workspace group fixtures
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 0 table fixtures
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 table fixtures
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.parquet_serde_cygj0', metastore_id=None, name='parquet_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/parquet_serde_cygj0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.orc_serde_cygj0', metastore_id=None, name='orc_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/orc_serde_cygj0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing table fixture: TableInfo(access_point=None, browse_only=None, catalog_name='hive_metastore', columns=None, comment=None, created_at=None, created_by=None, data_access_configuration_id=None, data_source_format=None, deleted_at=None, delta_runtime_properties_kvpairs=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, encryption_details=None, full_name='hive_metastore.hiveserde_in_place_cygj0.avro_serde_cygj0', metastore_id=None, name='avro_serde_cygj0', owner=None, pipeline_id=None, properties={'RemoveAfter': '2024070209'}, row_filter=None, schema_name='hiveserde_in_place_cygj0', sql_path=None, storage_credential_name=None, storage_location='dbfs:/mnt/TEST_MOUNT_NAME/a/hiveserde_in_place_cygj0/avro_serde_cygj0', table_constraints=None, table_id=None, table_type=<TableType.EXTERNAL: 'EXTERNAL'>, updated_at=None, updated_by=None, view_definition=None, view_dependencies=None)
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] clearing 3 schema fixtures
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:52 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='ucx_cipce', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='ucx_cipce.hiveserde_in_place_cygj0', metastore_id=None, name='hiveserde_in_place_cygj0', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
08:53 DEBUG [databricks.labs.ucx.mixins.fixtures] removing schema fixture: SchemaInfo(browse_only=None, catalog_name='hive_metastore', catalog_type=None, comment=None, created_at=None, created_by=None, effective_predictive_optimization_flag=None, enable_predictive_optimization=None, full_name='hive_metastore.ucx_saauj', metastore_id=None, name='ucx_saauj', owner=None, properties=None, schema_id=None, storage_location=None, storage_root=None, updated_at=None, updated_by=None)
[gw3] linux -- Python 3.10.14 /home/runner/work/ucx/ucx/.venv/bin/python

Running from acceptance #4094

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JCZuurmond JCZuurmond JCZuurmond approved these changes

@mwojtyczka mwojtyczka Awaiting requested review from mwojtyczka

Assignees

@asnare asnare

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@asnare @JCZuurmond