[FEATURE]: Create Catalog Schema should validate external location #1101
Labels
cloud/aws
Issues related to AWS
cloud/azure
issues related to Azure
enhancement
New feature or request
migrate/external
go/uc/upgrade SYNC EXTERNAL TABLES step
migrate/managed
go/uc/upgrade Upgrade Managed Tables and Jobs
Comments
FastLee
added
enhancement
nfx
pushed a commit
that referenced
this issue
Mar 26, 2024
…e-catalogs-schemas` command (#1110) In this release, we have updated the "create\_catalogs\_schemas" function in "cli.py" to create UC catalogs and schemas based on the destinations from "create\_table\_mapping". This function previously used a "CatalogSchema" object created using the workspace client, installation, and prompts, but has been modified to directly use the installation object and call the "create\_all\_catalogs\_schemas" method. We have also added a new feature for catalog validation with commit message 'Feature/create catalog validation 1101', which includes the addition of the "create\_all\_catalogs\_schemas" method in "catalog\_schema.py". The new method generates a list of missing catalogs and schemas and creates them using the "WorkspaceClient". Additionally, we have added new imports, modified the constructor of the "CatalogSchema" class, and added new methods for creating catalogs and schemas, and updated the unit tests for "hive\_metastore/catalog\_schema.py". Overall, these changes improve the robustness and reliability of the tool for working with catalogs and external locations. Closes #1101
nkvuong
pushed a commit
that referenced
this issue
Mar 27, 2024
…e-catalogs-schemas` command (#1110) In this release, we have updated the "create\_catalogs\_schemas" function in "cli.py" to create UC catalogs and schemas based on the destinations from "create\_table\_mapping". This function previously used a "CatalogSchema" object created using the workspace client, installation, and prompts, but has been modified to directly use the installation object and call the "create\_all\_catalogs\_schemas" method. We have also added a new feature for catalog validation with commit message 'Feature/create catalog validation 1101', which includes the addition of the "create\_all\_catalogs\_schemas" method in "catalog\_schema.py". The new method generates a list of missing catalogs and schemas and creates them using the "WorkspaceClient". Additionally, we have added new imports, modified the constructor of the "CatalogSchema" class, and added new methods for creating catalogs and schemas, and updated the unit tests for "hive\_metastore/catalog\_schema.py". Overall, these changes improve the robustness and reliability of the tool for working with catalogs and external locations. Closes #1101
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is there an existing issue for this?
Problem statement
Currently
create-catalog-schemadoesn't validate the external location keyed in by the user and will fail if an erroneous location is keyed in.Related:
create-catalogs-schemasshould work for external paths #1039Proposed Solution
Validate external locations before creating the catalog. Alert the user if the external location selected is not working.
Additional Context
No response
The text was updated successfully, but these errors were encountered: