Automatically assign a UC Metastore to a workspace

[nfx]

Background

Assign metastore to a workspace automatically.

Operational Considerations

If more than one metastore exists in a region, which is uncommon but exists for certain preview customers or for customers who were granted exceptions due to legal entity differences, specific legislative or regulation oversight, etc, we need to provide a way for the user to default their metastore choice.

Add default_catalog to configuration

TODO:

• Attach UC Metastore to Workspace
• Create CATALOG for Workspace with workspace ID in name
• Bind Workspace to new CATALOG
• Create Workspace Admin group at Account Level
• Add Workspace Admins as members to the group
• Set Workspace Admin group as the CATALOG owner

Process flow

The user opts to assign a workspace by making a selective action in the CLI.
Before they continue, there is an optional step to list the metastores available in their region.

e.g.
databricks labs ucx show_all_metastores <workspace_id>

Resulting action
Metastore information for the given region, specifically name and ID.

e.g
databricks labs ucx assign_metastore <workspace_id>

Resulting action
Metastore is assigned to this workspace, default catalog is the workspace catalog

e.g
databricks labs ucx assign_metastore <workspace_id> --default_catalog=main

Resulting action
Metastore is assigned to this workspace and the default catalog is set to main

For customers that have multiple metastores in a region:

e.g.
databricks labs ucx assign_metastore <workspace_id> --default_catalog=main --use_metastore_id=<metastore_id

Resulting action
The specific metastore is assigned to this workspace.

[renardeinside]

Please note that quite often there might be several metastores in one region. It's not a recommended setup but it exists in some particular cases.

[nfx]

@renardeinside can you elaborate on the reasoning behind that?

[nfx]

@zpappa the CLI naming convention will be databricks labs ucx assign-metastore, as we'll have it as an installable extension for Databricks CLI.