Skip to content

Add support for WASB to native Azure file system implementation#23535

Merged
dain merged 1 commit intotrinodb:masterfrom
dain:wasb
Sep 23, 2024
Merged

Add support for WASB to native Azure file system implementation#23535
dain merged 1 commit intotrinodb:masterfrom
dain:wasb

Conversation

@dain
Copy link
Copy Markdown
Member

@dain dain commented Sep 23, 2024

Description

Add support for WASB to native Azure file system implementation. WASB urls can be supported by simply modifying the Azure location parsing code as the underlying Azure storage APIs are set by the storage account type. The main difference to a ABFS url is the scheme is wasb and the blob endpoint is referenced instead of dfs.

Fixes #23511

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

# Section
* Add support for WASB to native Azure file system implementation. ({issue}`23511`)

@dain dain requested a review from electrum September 23, 2024 18:58
@cla-bot cla-bot bot added the cla-signed label Sep 23, 2024
Copy link
Copy Markdown
Member

@electrum electrum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should support wasbs as well, to be fully compatible with existing Hadoop locations.

<scope>runtime</scope>
</dependency>

<dependency>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it needed?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the tests calls slice.toUtf8String so it has to be declared

@dain dain merged commit f553882 into trinodb:master Sep 23, 2024
@dain dain deleted the wasb branch September 23, 2024 21:41
@github-actions github-actions bot added this to the 459 milestone Sep 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

Support wasb scheme alias in native Azure filesystem

3 participants