Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GCS Connector runs into StackOverflow while creating hadoop credential #219

Closed
daichi-m opened this issue Jul 19, 2019 · 2 comments · Fixed by #882
Closed

GCS Connector runs into StackOverflow while creating hadoop credential #219

daichi-m opened this issue Jul 19, 2019 · 2 comments · Fixed by #882

Comments

@daichi-m
Copy link

daichi-m commented Jul 19, 2019 •
edited by medb
Loading

I'm trying to create a hadoop credential jceks file and trying to store it in a gs bucket. The command runs into a StackOverflowError.

Command:

hadoop credential create javax.jdo.option.ConnectionPassword \
    -provider jceks://gs/${bucketName}/path/to/certs/hive.jceks

Stacktrace:

Exception in thread "main" java.lang.StackOverflowError
	at java.net.URI$Parser.scanIPv4Address(URI.java:3292)
	at java.net.URI$Parser.parseIPv4Address(URI.java:3332)
	at java.net.URI$Parser.parseServer(URI.java:3234)
	at java.net.URI$Parser.parseAuthority(URI.java:3155)
	at java.net.URI$Parser.parseHierarchical(URI.java:3097)
	at java.net.URI$Parser.parse(URI.java:3053)
	at java.net.URI.<init>(URI.java:588)
	at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:67)
	at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2143)
	at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2081)
	at com.google.cloud.hadoop.repackaged.gcs.com.google.cloud.hadoop.util.ConfigurationEntriesAdapterSpecific.getPassword(ConfigurationEntriesAdapterSpecific.java:29)
	at com.google.cloud.hadoop.repackaged.gcs.com.google.cloud.hadoop.util.HadoopCredentialConfiguration$ConfigurationEntriesAdapter.getPassword(HadoopCredentialConfiguration.java:39)
	at com.google.cloud.hadoop.repackaged.gcs.com.google.cloud.hadoop.util.EntriesCredentialConfiguration.setConfiguration(EntriesCredentialConfiguration.java:295)
	at com.google.cloud.hadoop.repackaged.gcs.com.google.cloud.hadoop.util.EntriesCredentialConfiguration$Builder.build(EntriesCredentialConfiguration.java:171)
	at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.getCredential(GoogleHadoopFileSystemBase.java:1612)
	at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.createGcsFs(GoogleHadoopFileSystemBase.java:1699)
	at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.configure(GoogleHadoopFileSystemBase.java:1658)
	at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.initialize(GoogleHadoopFileSystemBase.java:683)
	at com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystemBase.initialize(GoogleHadoopFileSystemBase.java:646)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3242)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:121)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3291)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3259)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:470)
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:356)
	at org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
	at org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:85)
	at org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:49)
	at org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:41)
	at org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
	at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73)
	at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2143)
	at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2081)</pre></code>

I'm using GCS Connector version hadoop2-1.9.17.jar in Cloud Dataproc version 1-4-10-debian9
@daichi-m daichi-m changed the title GCS Connector runs into StackOverflow while create hadoop credential GCS Connector runs into StackOverflow while creating hadoop credential Jul 19, 2019
@medb medb self-assigned this Jul 22, 2019
@medb
Copy link
Contributor

medb commented Jul 22, 2019

Thanks for report, I will take a look into this.

@medb
Copy link
Contributor

medb commented Dec 3, 2021

Is this reproducible with GCS connector 2.2.4?

@medb medb removed their assignment Dec 3, 2021
cnauroth added a commit to cnauroth/hadoop-connectors that referenced this issue Oct 4, 2022
@cnauroth
Support GCS Connector usage as a Hadoop Credential Provider.
fb50769 
Closes GoogleCloudDataproc#219.
cnauroth added a commit to cnauroth/hadoop-connectors that referenced this issue Oct 4, 2022
@cnauroth
Support GCS Connector usage as a Hadoop Credential Provider.
b4b148e 
For additional context, see
https://issues.apache.org/jira/browse/HADOOP-12846.

Closes GoogleCloudDataproc#219.
@cnauroth cnauroth mentioned this issue Oct 4, 2022
Merged
cnauroth added a commit to cnauroth/hadoop-connectors that referenced this issue Oct 4, 2022
@cnauroth
Support GCS Connector usage as a Hadoop Credential Provider.
52cf802 
For additional context, see
https://issues.apache.org/jira/browse/HADOOP-12846.

Closes GoogleCloudDataproc#219.
cnauroth added a commit that referenced this issue Oct 4, 2022
@cnauroth
Support GCS Connector usage as a Hadoop Credential Provider. (#882)
9ba13a6 
For additional context, see
https://issues.apache.org/jira/browse/HADOOP-12846.

Closes #219.
cnauroth added a commit to cnauroth/hadoop-connectors that referenced this issue Oct 4, 2022
@cnauroth
Support GCS Connector usage as a Hadoop Credential Provider. (GoogleC…
310cd6e 
…loudDataproc#882)

For additional context, see
https://issues.apache.org/jira/browse/HADOOP-12846.

Closes GoogleCloudDataproc#219.

(cherry picked from commit 9ba13a6)
@cnauroth cnauroth mentioned this issue Oct 4, 2022
Merged
cnauroth added a commit that referenced this issue Oct 5, 2022
@cnauroth
Support GCS Connector usage as a Hadoop Credential Provider. (#882) (#…
793cfa8 
…883)

For additional context, see
https://issues.apache.org/jira/browse/HADOOP-12846.

Closes #219.

(cherry picked from commit 9ba13a6)
cnauroth added a commit to cnauroth/hadoop-connectors that referenced this issue Oct 5, 2022
@cnauroth
Support GCS Connector usage as a Hadoop Credential Provider. (GoogleC…
3de60e0 
…loudDataproc#882) (GoogleCloudDataproc#883)

For additional context, see
https://issues.apache.org/jira/browse/HADOOP-12846.

Closes GoogleCloudDataproc#219.

(cherry picked from commit 9ba13a6)
(cherry picked from commit 793cfa8)
@cnauroth cnauroth mentioned this issue Oct 5, 2022
Merged
cnauroth added a commit that referenced this issue Oct 6, 2022
@cnauroth
Support GCS Connector usage as a Hadoop Credential Provider. (#882) (#…
95ddd82 
…883) (#884)

For additional context, see
https://issues.apache.org/jira/browse/HADOOP-12846.

Closes #219.

(cherry picked from commit 9ba13a6)
(cherry picked from commit 793cfa8)
@daichi-m daichi-m mentioned this issue Aug 7, 2023
Closed
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support GCS Connector usage as a Hadoop Credential Provider. cnauroth/hadoop-connectors
2 participants
@medb @daichi-m