Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: testcontainers inside k8s Jenkins container. #654

Closed
sjvatgit opened this issue Apr 23, 2018 · 5 comments
Closed

Question: testcontainers inside k8s Jenkins container. #654

sjvatgit opened this issue Apr 23, 2018 · 5 comments

Comments

@sjvatgit
Copy link

Hello! I'm trying to setup the testing environment inside Jenkins k8s container.
I think it's important to mention that k8s cluster has no internet access but local private docker + maven registries inside nexus.

The jenkins test job have the following output now:

T E S T S

Running SimplePostgreSQLTest
[main] INFO org.testcontainers.dockerclient.DockerClientProviderStrategy - Loaded org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy from ~/.testcontainers.properties, will try it first
[main] INFO org.testcontainers.dockerclient.EnvironmentAndSystemPropertyClientProviderStrategy - Found docker client settings from environment
[main] INFO org.testcontainers.dockerclient.DockerClientProviderStrategy - Found Docker environment with Environment variables, system properties and defaults. Resolved:
dockerHost=unix:///var/run/docker.sock
apiVersion='{UNKNOWN_VERSION}'
registryUrl='https://index.docker.io/v1/'
registryUsername='root'
registryPassword='null'
registryEmail='null'
dockerConfig='DefaultDockerClientConfig[dockerHost=unix:///var/run/docker.sock,registryUsername=root,registryPassword=,registryEmail=,registryUrl=https://index.docker.io/v1/,dockerConfigPath=/root/.docker,sslConfig=,apiVersion={UNKNOWN_VERSION},dockerConfig=]'

[testcontainers-netty-1-3] ERROR com.github.dockerjava.core.async.ResultCallbackTemplate - Error during callback
com.github.dockerjava.api.exception.InternalServerErrorException: {"message":"Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 10.144.143.34:53: no such host"}

at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:109)
at com.github.dockerjava.netty.handler.HttpResponseHandler.channelRead0(HttpResponseHandler.java:33)
at org.testcontainers.shaded.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at org.testcontainers.shaded.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:438)
at org.testcontainers.shaded.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
at org.testcontainers.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
at org.testcontainers.shaded.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:253)
at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at org.testcontainers.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1342)
at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at org.testcontainers.shaded.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at org.testcontainers.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:934)
at org.testcontainers.shaded.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:979)
at org.testcontainers.shaded.io.netty.channel.epoll.EpollDomainSocketChannel$EpollDomainUnsafe.epollInReady(EpollDomainSocketChannel.java:138)
at org.testcontainers.shaded.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:404)
at org.testcontainers.shaded.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:307)
at org.testcontainers.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
at org.testcontainers.shaded.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
at java.lang.Thread.run(Thread.java:748)

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.884 sec <<< FAILURE!
testSimple(SimplePostgreSQLTest) Time elapsed: 0.027 sec <<< ERROR!
com.github.dockerjava.api.exception.InternalServerErrorException: {"message":"Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 10.144.143.34:53: no such host"}

I really don't understand why it tries to reach registry-1.docker.io instead of local nexus where all images/jars are stored.

@bsideup
Copy link
Member

bsideup commented Apr 24, 2018

Hi @sjvatgit,

This line says that your Docker environment is not configured to use local Nexus:

[main] INFO org.testcontainers.dockerclient.DockerClientProviderStrategy - Found Docker environment with Environment variables, system properties and defaults. Resolved:
dockerHost=unix:///var/run/docker.sock
apiVersion='{UNKNOWN_VERSION}'
registryUrl='https://index.docker.io/v1/'

I would suggest configuring registryURL in your Docker's config or use properties to override default images we use, see Customizing images section:
https://www.testcontainers.org/usage/properties.html

@sjvatgit
Copy link
Author

Thank you for the reply!
I've resolved this problem by providing the images names in the .testcontainers.properties file (by the way, there is no any info at testcontainers.org about ryuk.container.image which, imho, is a must for the testcontainers core)
In my case the path to the file was not the ~/.docker/.testcontainers.properties but /root/.docker/.testcontainers.properties at the jenkins slave.

Currently I've stuck with this:
"
[main] INFO org.testcontainers.DockerClientFactory - Docker host IP address is 172.17.0.1
[main] INFO org.testcontainers.DockerClientFactory - Connected to docker:
Server Version: 17.12.0-ce
API Version: 1.35
Operating System: Red Hat Enterprise Linux Server 7.4 (Maipo)
Total Memory: 32013 MB
[main] INFO org.testcontainers.DockerClientFactory - Ryuk started - will monitor and terminate Testcontainers containers on JVM exit
ℹ︎ Checking the system...
✔ Docker version should be at least 1.6.0
✔ Docker environment should have more than 2GB free disk space
✔ File should be mountable
[main] INFO 🐳 [localhost:5000/postgres:10.2] - Creating container for image: localhost:5000/postgres:10.2
[main] INFO 🐳 [localhost:5000/postgres:10.2] - Starting container with ID: 784e891424a02348f6290eda0381d9b15dcc042d308f0768be17292bfadbfcbc
[main] INFO 🐳 [localhost:5000/postgres:10.2] - Container localhost:5000/postgres:10.2 is starting: 784e891424a02348f6290eda0381d9b15dcc042d308f0768be17292bfadbfcbc
[main] INFO 🐳 [localhost:5000/postgres:10.2] - Container localhost:5000/postgres:10.2 started
URL: jdbc:postgresql://172.17.0.1:32777/test
User: test
Password: test
HikariConfig: com.zaxxer.hikari.HikariConfig@18271936
[main] INFO com.zaxxer.hikari.HikariDataSource - HikariCP pool HikariPool-0 is starting.
[main] INFO com.zaxxer.hikari.pool.HikariPool - HikariCP pool HikariPool-0 is shutting down.
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 309.088 sec <<< FAILURE!
testSimple(SimplePostgreSQLTest) Time elapsed: 305.051 sec <<< ERROR!
com.zaxxer.hikari.pool.PoolInitializationException: Exception during pool initialization
at com.zaxxer.hikari.pool.BaseHikariPool.initializeConnections(BaseHikariPool.java:544)
at com.zaxxer.hikari.pool.BaseHikariPool.(BaseHikariPool.java:171)
at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:60)
at com.zaxxer.hikari.pool.HikariPool.(HikariPool.java:48)
at com.zaxxer.hikari.HikariDataSource.(HikariDataSource.java:80)
at SimplePostgreSQLTest.testSimple(SimplePostgreSQLTest.java:59)
"

172.17.0.1 is the docker host node address inside k8s.
Do you have any suggestions about this?

@bsideup
Copy link
Member

bsideup commented Apr 24, 2018

@sjvatgit are you running Testcontainers 1.7.1? There was a fix ( #648 ) for similar issue

@sjvatgit
Copy link
Author

sjvatgit commented Apr 24, 2018

Yes, I have 1.7.1 already. I'ts the latest one and if I understand the fix correctly the previous version retruns with localhost:port. But I suppose current 172.17.0.1:port is the correct one.
Update
After some investigation I've realised that 172.17.0.1 is not a container IP but the gateway IP (to be precise, a docker0 bridge in my case).
Also found a same issue in the testcontainers/testcontainers-scala project:
containerIpAddress returns the gateway
It is something to be fixed, imho...
Update2
There was a wrong dependency in my test project. The cause of the error is the postgres 9.2 jdbc driver which is incompatible with the 10.2 database. After update, everything is working fine.
Thank you!

@sjvatgit
Copy link
Author

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants