Skip to content

[#6049] fix(bundles): Fix scheme gs not found problem #6050

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 3, 2025

Conversation

yuqi1129
Copy link
Contributor

@yuqi1129 yuqi1129 commented Dec 31, 2024

What changes were proposed in this pull request?

Add mergeServiceFiles() when build the bundle jars

Why are the changes needed?

After add HDFS client to gravitino bundle jar, we should add make sure both Gravitino and HDFS Filesystem are merged into resource file

Fix: #6049

Does this PR introduce any user-facing change?

N/A

How was this patch tested?

Existing test.

@FANNG1
Copy link
Contributor

FANNG1 commented Dec 31, 2024

Could you explain the reason?

@yuqi1129
Copy link
Contributor Author

yuqi1129 commented Jan 2, 2025

Could you explain the reason?

without mergeServiceFiles(), the content in the file services/META-INF/org.apache.hadoop.fs.FileSystem will not contains com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem, we need to set fs.gs.impl to it explicitly, however, we do not have this operation in the file system providers, so the error will occur.

@FANNG1
Copy link
Contributor

FANNG1 commented Jan 3, 2025

LGTM, @yuqi1129 , please add more context to the description.

FANNG1
FANNG1 previously approved these changes Jan 3, 2025
@FANNG1 FANNG1 merged commit 6f63430 into apache:main Jan 3, 2025
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug report] Scheme 'gs' is not found when running GravitinoVirtualFileSystemGCSIT
3 participants