[Feat](Storage)Refactor Storage Integration: Unified Storage Parameter Handling for Export, TVF, and Backup #49163
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…r Handling for Export, TVF, and Backup ## PR Description This PR introduces a unified storage parameter handling mechanism for storage-related functionalities, including Export, Table-Valued Functions (TVF), and Backup. The key objective is to decouple business logic from the underlying storage implementation, ensuring that the business layer only focuses on its domain logic while storage parameters manage the interaction with the storage system. ##Key Changes ### Centralized Storage Parameter Management: Instead of handling storage configurations in multiple places, storage parameters now serve as the sole interface between the business logic and the storage layer. This improves maintainability, avoids scattered storage-related logic, and simplifies future storage extensions. ### Separation of Concerns: The business layer is storage-agnostic, meaning it does not need to handle storage-specific details. Storage parameters are responsible for translating high-level storage configurations into system-specific settings. ### Phase 1 Implementation: This PR focuses on integrating Export, TVF, and Backup with the new storage parameter mechanism. Subsequent PRs will address Broker and additional components, ensuring a smooth and incremental migration. ##Next Steps Gradually extend this unified approach to Broker and other storage-dependent functionalities. Continue refining the abstraction to support future storage systems with minimal modifications. ## Why This Change? Enhances maintainability by consolidating storage-related logic. Improves extensibility for future storage backends. Simplifies business logic, ensuring a cleaner separation of concerns.
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
morningman
reviewed
Mar 18, 2025
| @@ -643,42 +633,15 @@ private void analyzeBrokerDesc(Set<String> processedPropKeys) throws UserExcepti | |||
| } else if (filePath.toUpperCase().startsWith(HDFS_FILE_PREFIX.toUpperCase())) { | |||
| brokerName = StorageBackend.StorageType.HDFS.name(); | |||
| storageType = StorageBackend.StorageType.HDFS; | |||
| } else if (filePath.toUpperCase().startsWith("oss".toUpperCase())) { | |||
There was a problem hiding this comment.
Why only handle oss seperately?
morningman
merged commit faeb098
into
apache:branch-refactor_property
11 of 12 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Description
This PR introduces a unified storage parameter handling mechanism for storage-related functionalities, including Export, Table-Valued Functions (TVF), and Backup. The key objective is to decouple business logic from the underlying storage implementation, ensuring that the business layer only focuses on its domain logic while storage parameters manage the interaction with the storage system.
Key Changes
Centralized Storage Parameter Management:
Instead of handling storage configurations in multiple places, storage parameters now serve as the sole interface between the business logic and the storage layer. This improves maintainability, avoids scattered storage-related logic, and simplifies future storage extensions. ### Separation of Concerns:
The business layer is storage-agnostic, meaning it does not need to handle storage-specific details. Storage parameters are responsible for translating high-level storage configurations into system-specific settings.
Phase 1 Implementation:
This PR focuses on integrating Export, TVF, and Backup with the new storage parameter mechanism. Subsequent PRs will address Broker and additional components, ensuring a smooth and incremental migration.
Next Steps
Gradually extend this unified approach to Broker and other storage-dependent functionalities. Continue refining the abstraction to support future storage systems with minimal modifications.
Why This Change?
Enhances maintainability by consolidating storage-related logic. Improves extensibility for future storage backends. Simplifies business logic, ensuring a cleaner separation of concerns.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)