Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mapping Flow from Old [HMS] to new [UC] For External Tables #354

Closed
pohlposition opened this issue Oct 2, 2023 · 6 comments
Closed

Mapping Flow from Old [HMS] to new [UC] For External Tables #354

pohlposition opened this issue Oct 2, 2023 · 6 comments
Labels
enhancement New feature or request feat/migration-index mapping of databases to catalog or potentially other databases migrate/external go/uc/upgrade SYNC EXTERNAL TABLES step

Comments

@pohlposition
Copy link
Contributor

Database / Schema > UC Catalog

  • May want to prefix / suffix a database name

Database may map to 1 or more Catalogs

Defined Table Migration Process defined in this PR: #307

Probably a file that someone downloads, modifies, and uploads (offline mapping)

@pohlposition pohlposition added the migrate/external go/uc/upgrade SYNC EXTERNAL TABLES step label Oct 2, 2023
@pohlposition
Copy link
Contributor Author

Dependencies:

  • Is there a mechanism for handling downloads / uploads

@william-conti
Copy link
Contributor

Probably we would need to have the workspace id/name in the mapping as it can happen to have the same database name between workspaces. thus creating a conflict

@nfx nfx added the feat/migration-index mapping of databases to catalog or potentially other databases label Oct 2, 2023
@pohlposition
Copy link
Contributor Author

pohlposition commented Oct 2, 2023

Need to detect conflicts and dedupe. #335

@nfx
Copy link
Collaborator

nfx commented Oct 2, 2023

@pohlposition let's merge newer issues into older issues :)

@nfx nfx added this to UCX Oct 3, 2023
@nfx nfx moved this to Design in UCX Oct 3, 2023
@nfx
Copy link
Collaborator

nfx commented Oct 16, 2023

@pohlposition :thinks realistically, we need to have an out-of-band job that can "move" objects to new catalogs. Because you're going to register a table in CATALOG.SCHEMA (A), and then realize later that it doesn't belong there for other workspaces that were referencing it.

@nfx nfx added the enhancement New feature or request label Nov 7, 2023
@nfx
Copy link
Collaborator

nfx commented Dec 6, 2023

closing in favor of #669

@nfx nfx closed this as completed Dec 6, 2023
@github-project-automation github-project-automation bot moved this from Design to Archive in UCX Dec 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request feat/migration-index mapping of databases to catalog or potentially other databases migrate/external go/uc/upgrade SYNC EXTERNAL TABLES step
Projects
Archived in project
Development

No branches or pull requests

3 participants