Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

user id 'xxx' is too big ( > 2097151 ) when using commons-compress 1.21 with large user ID #4384

Closed
schmidt-galen-heb opened this issue Aug 19, 2021 · 3 comments · Fixed by #4388

Comments

@schmidt-galen-heb
Copy link
Contributor

I upgraded org.apache.commons/commons-compress to 1.21 in our project to address CVE-2021-36090, and TestContainers started failing to start with the following error:

org.testcontainers.containers.ContainerLaunchException: Container startup failed
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:334)
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:315)
	at org.testcontainers.containers.GenericContainer.starting(GenericContainer.java:1060)
	at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:29)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:190)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
	at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:43)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
	at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:82)
	at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:73)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:188)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:154)
	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:128)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548)
Caused by: org.rnorth.ducttape.RetryCountExceededException: Retry limit hit with exception
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:88)
	at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:327)
	... 36 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:523)
	at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:329)
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
	... 37 more
Caused by: java.lang.IllegalArgumentException: user id '<redacted>' is too big ( > 2097151 ).
	at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumber(TarArchiveOutputStream.java:651)
	at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumber(TarArchiveOutputStream.java:639)
	at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.failForBigNumbers(TarArchiveOutputStream.java:630)
	at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.putArchiveEntry(TarArchiveOutputStream.java:377)
	at org.testcontainers.utility.MountableFile.recursiveTar(MountableFile.java:327)
	at org.testcontainers.utility.MountableFile.transferTo(MountableFile.java:304)
	at org.testcontainers.containers.ContainerState.copyFileToContainer(ContainerState.java:276)
	at org.testcontainers.containers.ContainerState.copyFileToContainer(ContainerState.java:253)
	at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:684)
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:414)
	... 39 more

My account is provisioned via a large enterprise system, so my User ID is indeed larger than 2097151.

I was using TestContainers 1.51.1, but I tried upgrading to 1.16.0 and received the same failure.

I'm unclear why the version bump caused the issue since the relavent code block appears to be the same in commons-compress 1.20 and 1.21, but I believe the solution is just to set the TarArchiveOutputStream.bigNumberMode to TarArchiveOutputStream.BIGNUMBER_POSIX in addition to TarArchiveOutputStream.longFileMode

@batt842
Copy link

batt842 commented Aug 20, 2021

I'm experiencing the same issue. It'd be resolved by setting bigNumberMode, I guess.

@kiview
Copy link
Member

kiview commented Aug 20, 2021

Hey @schmidt-galen-heb, @batt842

Thanks for bringing this to our attention and already providing a possible explanation and fix.
Since you have a system that allows reproducing the error, could I kindly ask you to try out the proposed fix and submit a PR?

@schmidt-galen-heb
Copy link
Contributor Author

Opened #4388.

I built it locally using ./gradlew publishMavenJavaPublicationToMavenLocal -Pversion=1.16.1-4384-SNAPSHOT, and then verified that:

  1. My project failed to build when using version 1.16.0 with the above error
  2. My project built successfully when using version 1.16.1-4384-SNAPSHOT (the custom version I built with the changes from the PR)

rnorth pushed a commit that referenced this issue Sep 8, 2021
* Use POSIX mode for large numbers in tar archives (#4384)

* PR feedback
@kiview kiview added this to the next milestone Oct 13, 2021
kaklakariada added a commit to exasol/tableau-connector that referenced this issue Oct 20, 2021
user id 'xxx' is too big ( > 2097151 ) when using commons-compress 1.21
with large user ID

testcontainers/testcontainers-java#4384
kaklakariada added a commit to exasol/tableau-connector that referenced this issue Oct 27, 2021
* Update name of ODBC driver

* Update ODBC manifest

* Add JDBC connector

* Add JDBC capabilities

* Describe installation of jdbc driver

* Adapt capabilities for jdbc driver

* Add fingerprint

* Configure tdvt

* Add data source tests

* Add results workbook

* Update data source

* Update test result workbook

* Configure string trim for jdbc

* Add test with bad password

* Skip string.split test: function not supported by Exasol

* Rename column zzz2 to zzz

* Remove trailing spaces when extracting parts of a date

* Remove duplicate space

* Add rationale for skipping the 'split' function tests

* Change test user name to be equal to odbc tests

* Skip test for time data type

TIME not supported by Exasol

* Rename tdvt folder to odbc

* Adapt hostname for jdbc tests

* Adapt odbc tests to run

* Unify test configuration for jdbc & odbc

* Fix odbc issues with spaces

Remove trimming capabilities that hide issue with trailing spaces in TO_CHAR()

* Configure max number of threads to 1

* Remvove argument for number of threads

This is now configured in the ini file

* Add comment about failing tests

* Ignore Eclipse config files

* Update instructions for building the dockerfile

* Update dependencies

* Improve formatting of readme

* Add HtmlUnit driver

* Use Exasol 7.1.1, improve error handling

* Improve error handling in shell script

* Cleanup

* Adapt configuration to Eclipse 2021-09

* Validate & package jdbc connector

* Improve pom setup

* Cleanup tests: remove unused ports

* Read properties file in Java code to avoid having to set environment
variables when running tests from the IDE.

* Upgrade testcontainers to fix uploading files to the container

user id 'xxx' is too big ( > 2097151 ) when using commons-compress 1.21
with large user ID

testcontainers/testcontainers-java#4384

* Adapt UI-Tests to new Connector name / Tableau version

* Run tests in headless mode

* Document ui-test setup

* Build JDBC url with fingerprint and certificate validation arguments

* Update fingerprint in connection files

* Add developer guide

* Move test instructions to developer guide

* Add connection dialog screenshot

* Configure Eclipse formatter to never join already wrapped lines

* Extract base class for JDBC & ODBC connector tests

* Add UI tests for JDBC connector

* Fix failing StaplesDataTest

See tableau/connector-plugin-sdk#871 for details.

* Add changelog, improve documentation

* #22 Implement review findings by @AnastasiiaSergienko

* Apply suggestions from code review

Co-authored-by: Anastasiia Sergienko <[email protected]>

* Implement review findings by @AnastasiiaSergienko

Co-authored-by: Anastasiia Sergienko <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants