[EPIC] Multi-workspace installations

[nkvuong]

Make UCX work across multiple workspaces

Relevant issues:

• [FEATURE] Aggregation of UCX output of multiple workspaces as CLI command #749
• [FEATURE]: External Metastores - Generate a separate UCX Schema by workspace #1469
• [FEATURE]: Create Account group equivalent for the workspace admin group #1194
• [FEATURE]: Enable download and upload of ucx files for a collection of workspaces #1783
• [FEATURE]: Let CLI commands run on a collection of workspaces #1782
• Hive Metastore on multiple workspaces may point to the same assets. We need to dedupe upgrades. #335
• Add databricks labs ucx validate-table-locations command #673
• [FEATURE]: Joining an existing collection when installing ucx #918
• [FEATURE]: Joining an existing collection when ucx is already installed #919
• [FEATURE]: Multiple workspaces may refer to the same prefix with different principals. We need to dedupe upgrades. #917
• Migrate Groups - Edge Case - Handle Duplicate Group Name in Account Console #344
• Migrate Groups - Workspace Local Group Does not Exist in Account Console #343
• [FEATURE]: Add ability to query inventory database data for a collection of workspaces at once #1784
• [FEATURE] Enforce the same UCX version is installed accross workspaces #3043

Problem statement

Customers with many workspaces will need abilities to run assessment & migration for multiple workspaces at once.

Capabilities:

• join existing workspace collection (& re-sync the relevant info from it). There are 3 entry points

  • Account-level installation [FEATURE]: Account level installer #920
  • New installation [FEATURE]: Joining an existing collection when installing ucx #918
  • Existing installation [FEATURE]: Joining an existing collection when ucx is already installed #919

• dedupe conflict

  • check ws info is in sync, prompt if not and forcefully sync
  • Hive Metastore on multiple workspaces may point to the same assets. We need to dedupe upgrades. #335
  • [FEATURE]: Multiple workspaces may refer to the same prefix with different principals. We need to dedupe upgrades. #917

Related issues:

• Add databricks labs ucx validate-table-locations command #673
• Hive Metastore on multiple workspaces may point to the same assets. We need to dedupe upgrades. #335
• Add ability to create account groups with members from the workspace group with members #17
• [FEATURE]: Multiple workspaces may refer to the same prefix with different principals. We need to dedupe upgrades. #917
• [FEATURE]: Allow only single UCX installation per workspace or dynamically determine it #803
• [FEATURE]: Single UCX Installation per workspace #890

Proposed Solution

ucx should discover all workspaces that users have access to, and offer configuration to group those into "collections". This can either be done via prefix, tags or manually, e.g. yaml config

Collection configuration would then be stored in all workspaces installation

Also need to handle duplications in mapping tables

Additional Context

No response