Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker-compose v3 format is not supported #531

Closed
alexcase52 opened this issue Dec 22, 2017 · 11 comments
Closed

Docker-compose v3 format is not supported #531

alexcase52 opened this issue Dec 22, 2017 · 11 comments

Comments

@alexcase52
Copy link

alexcase52 commented Dec 22, 2017

TestContainers 1.5.1

18:47:36.971 [main] ERROR ? [docker/compose:1.8.0] - Could not start container
java.lang.IllegalStateException: Container did not start correctly.
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:241) [testcontainers-1.5.1.jar:na]
	at org.testcontainers.containers.GenericContainer.lambda$start$0(GenericContainer.java:194) [testcontainers-1.5.1.jar:na]
	at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:76) ~[duct-tape-1.0.6.jar:na]
	at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:192) [testcontainers-1.5.1.jar:na]
	at org.testcontainers.containers.ContainerisedDockerCompose.invoke(DockerComposeContainer.java:442) ~[testcontainers-1.5.1.jar:na]
	at org.testcontainers.containers.DockerComposeContainer.runWithCompose(DockerComposeContainer.java:152) ~[testcontainers-1.5.1.jar:na]
	at org.testcontainers.containers.DockerComposeContainer.finished(DockerComposeContainer.java:245) ~[testcontainers-1.5.1.jar:na]
	at org.testcontainers.containers.FailureDetectingExternalResource$1.evaluate(FailureDetectingExternalResource.java:36) ~[testcontainers-1.5.1.jar:na]
	at org.junit.rules.RunRules.evaluate(RunRules.java:20) ~[junit-4.11.jar:na]
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309) ~[junit-4.11.jar:na]
	at org.junit.runner.JUnitCore.run(JUnitCore.java:160) ~[junit-4.11.jar:na]
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69) ~[junit-rt.jar:na]
	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234) ~[junit-rt.jar:na]
	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74) ~[junit-rt.jar:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_112]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_112]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_112]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_112]
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144) ~[idea_rt.jar:na]
18:47:36.972 [main] ERROR ? [docker/compose:1.8.0] - Container log output (if any) will follow:
18:47:36.974 [dockerjava-netty-1-7] INFO  ? [docker/compose:1.8.0] - STDERR: Version in "[...]docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a version of "2" (or "2.0") and place your service definitions under the `services` key, or omit the `version` key and place your service definitions at the root of the file to use version 1.
18:47:36.975 [dockerjava-netty-1-7] INFO  ? [docker/compose:1.8.0] - STDERR: For more on the Compose file format versions, see https://docs.docker.com/compose/compose-file/
----

And I use docker-compose.yml with version: "3"

@bsideup
Copy link
Member

bsideup commented Dec 22, 2017

Hi @alexcase52,

Yes, v3 is a new format, and we use Docker Compose 1.8.0 by default.
You can either use withLocalCompose() or set compose.container.image to docker/compose:1.18.0 with configuration property (i.e. via testcontainers.properties file on classpath):
https://www.testcontainers.org/usage/properties.html

@alexcase52
Copy link
Author

thanks @bsideup, tried both options - neither worked well. I tried to reuse development env docker-compose.yml with several services. but as I understand docker-compose attempted to start everything from this file which is not what I needed. I need just a single service.
Worked around for now via building GenericContainer with my only service in the code.

@bsideup
Copy link
Member

bsideup commented Dec 22, 2017

@alexcase52 good to hear that you got it working :) Frankly, I would always recommend GenericContainer over Docker Compose, especially together with Networks, because it's so much better and flexible :)

@alexcase52
Copy link
Author

alexcase52 commented Dec 22, 2017

@bsideup, a question to close with this and have simplest possible GenericContainer building code:
I have Dockerfile and a bunch of files in its dir refferred by this Dockerfile. Can I say: this is Dir, there's all what you need, build an image? Something like new ImageFromDockerfile().withFilesFromPath(Dir) or even new ImageFromDockerfile(Dir)?

@jonashackt
Copy link

jonashackt commented Oct 3, 2018

Thanks @bsideup for your comment - i had the same issue in https://github.com/jonashackt/spring-boot-rest-clientcertificates-docker-compose like @alexcase52 :

STDERR: Version in "[...]docker-compose.yml" is unsupported. You might be seeing this error because you're using the wrong Compose file version. Either specify a version of "2" (or "2.0") and place your service definitions under the serviceskey, or omit theversion key and place your service definitions at the root of the file to use version 1.

First I used .withLocalCompose(true) which worked fine locally (with a current Docker Compose installation on Mac). But this failed on the TravisCI build.

Then I switched to the testcontainers.properties solution, where I upgraded the docker compose version of testcontainers:

compose.container.image=docker/compose:1.22.0

Now everything runs fine - both locally and on TravisCI. Here´s the full docker-compose.yml:

version: '3.7'

services:

 server-alice:
  build: ./server-alice
  ports:
   - "8443"
  tty:
    true
  restart:
    unless-stopped

 server-tom:
  build: ./server-tom
  ports:
   - "8443"
  tty:
    true
  restart:
    unless-stopped

 client-bob:
  build: ./client-bob
  ports:
   - "8080:8080"
  tty:
    true
  restart:
unless-stopped

I would suggest a general update in testcontainers core to docker/compose:1.22.0 since there are already 3.0, 3.1, 3.2, 3.3, 3.4, 3.5, 3.6 & 3.7 as of today. And I think it´s a common use case to just re-use an existing docker-compose.yml, which is also used in other contexts (this is also the case in my example project). And in that case you don´t want to be restricted to such old Compose file versions IMHO. For example restart policys aren´t available in lower versions than 3.0.

But as I don´t have any glue about testcontainer internals, that´s something the maintainers have to decide.

@kiview
Copy link
Member

kiview commented Oct 4, 2018

@jonashackt Just to clear out some confusion (or induce even more 😅), restart_policy (as most of the v3 exclusive features) only takes effect when deploying stacks to a swarm (also see Compose file version 3 reference.

TBH I'm also not sure, if v3 is really intended to replace v2 for classic Docker-Compose use.

Still it also might worth discussing if it would make sense to support v3 as well a stack deployment to swarm.

@jonashackt
Copy link

jonashackt commented Oct 8, 2018

@kiview your right, I also don´t see, why Docker Stack features should be of any use in a testcontainers context. BUT: The problem I see is that I want to be able to re-use my existing docker-compose.yml with testcontainers - without the need to change it. In this case the v3 features are used - not with testcontainers, but in another use case. E.g. my stage or prod deployments.

But it´s also completely ok for me to use the testcontainers.properties file. I was just curious if it would be a problem to just upgrade the base image in the testcontainers core.

rnorth pushed a commit that referenced this issue Dec 24, 2018
Bumps [influxdb-java](https://github.com/influxdata/influxdb-java) from 2.10 to 2.14.
<details>
<summary>Changelog</summary>

*Sourced from [influxdb-java's changelog](https://github.com/influxdata/influxdb-java/blob/master/CHANGELOG.md).*

> ## 2.14 [2018-10-12]
> 
> ### Fixes
> 
> - Fixed chunked query exception handling [Issue #523](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/523)
> - Memory leak in StringBuilder cache for Point.lineprotocol() [Issue #526](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/521)
> 
> ## 2.13 [2018-09-12]
> 
> ### Fixes
> - MessagePack queries: Exception during parsing InfluxDB version [macOS] [PR #487](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/487)
> - The InfluxDBResultMapper is able to handle results with a different time precision [PR #501](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/501)
> - UDP target host address is cached [PR #502](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/502)
> - Error messages from server not parsed correctly when using msgpack [PR #506](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/506)
> - Response body must be closed properly in case of JSON response [PR #514](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/514)
> - Time is serialized not consistently in MsgPack and Json, missing millis and nanos in MsgPack[PR #517](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/517)
> 
> ### Features
> 
> - Support for Basic Authentication [PR #492](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/492)
> - Added possibility to reuse client as a core part of [influxdb-java-reactive](https://github.com/bonitoo-io/influxdb-java-reactive) client [PR #493](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/493)
> - Retry capability for writing of BatchPoints [PR #503](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/503)
> - Added `BiConsumer` with capability to discontinue a streaming query [Issue #515](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/515)
> - Added `onComplete` action that is invoked after successfully end of streaming query [Issue #515](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/515)
> 
> ## 2.12 [2018-07-31]
> 
> ### Fixes
> 
> - Remove code which checks for unsupported influxdb versions [PR #474](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/474)
> - Unpredictable errors when OkHttpClient.Builder instance is reused [PR #478](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/478)
> 
> ### Features
> 
> - Support for MessagePack [PR #471](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/471)
> - Cache version per influxdb instance and reduce ping() calls for every query call [PR #472](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/472)
> - FAQ list for influxdb-java [PR #475](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/475)
> 
> ### Improvements
> 
> - Test: Unit test to ensure tags should be sorted by key in line protocol (to reduce db server overheads) [PR #476](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/476)
> 
> ## 2.11 [2018-07-02]
> 
> ### Features
> 
> - Allow write precision of TimeUnit other than Nanoseconds [PR #321](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/321)
> - Support dynamic measurement name in InfluxDBResultMapper [PR #423](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/423)
> - Debug mode which allows HTTP requests being sent to the database to be logged [PR #450](https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/450)
> - Fix problem of connecting to the influx api with URL which does not points to the url root (e.g. localhots:80/influx-api/) [PR #400] (https://github-redirect.dependabot.com/influxdata/influxdb-java/pull/400)
></table> ... (truncated)
</details>
<details>
<summary>Commits</summary>

- [`91d0f09`](influxdata/influxdb-java@91d0f09) [maven-release-plugin] prepare release influxdb-java-2.14
- [`8ffaeb9`](influxdata/influxdb-java@8ffaeb9) Revert "[maven-release-plugin] prepare release influxdb-java-2.14"
- [`2781da2`](influxdata/influxdb-java@2781da2) [maven-release-plugin] prepare release influxdb-java-2.14
- [`19c69ed`](influxdata/influxdb-java@19c69ed) [maven-release-plugin] prepare for next development iteration
- [`c6d7f25`](influxdata/influxdb-java@c6d7f25) [maven-release-plugin] prepare release influxdb-java-2.14
- [`2f4c594`](influxdata/influxdb-java@2f4c594) Merge pull request [#531](https://github-redirect.dependabot.com/influxdata/influxdb-java/issues/531) from heshengbang/master
- [`f653e62`](influxdata/influxdb-java@f653e62) Easy to use try-with-resources, add README.md
- [`c7be9b0`](influxdata/influxdb-java@c7be9b0) Easy to use try-with-resources
- [`4590d18`](influxdata/influxdb-java@4590d18) - added automated SNAPSHOT publishing to Maven Central repository
- [`ce65a41`](influxdata/influxdb-java@ce65a41) - added automated SNAPSHOT publishing to Maven Central repository
- Additional commits viewable in [compare view](influxdata/influxdb-java@influxdb-java-2.10...influxdb-java-2.14)
</details>
<br />

[![Dependabot compatibility score](https://api.dependabot.com/badges/compatibility_score?dependency-name=org.influxdb:influxdb-java&package-manager=gradle&previous-version=2.10&new-version=2.14)](https://dependabot.com/compatibility-score.html?dependency-name=org.influxdb:influxdb-java&package-manager=gradle&previous-version=2.10&new-version=2.14)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

**Note:** This repo was added to Dependabot recently, so you'll receive a maximum of 5 PRs for your first few update runs. Once an update run creates fewer than 5 PRs we'll remove that limit.

You can always request more updates by clicking `Bump now` in your [Dependabot dashboard](https://app.dependabot.com).

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme

Additionally, you can set the following in the `.dependabot/config.yml` file in this repo:
- Update frequency (including time of day and day of week)
- Automerge options (never/patch/minor, and dev/runtime dependencies)
- Pull request limits (per update run and/or open at any time)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)

Finally, you can contact us by mentioning @dependabot.

</details>
@stale
Copy link

stale bot commented Jan 6, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you believe this is a mistake, please reply to this comment to keep it open. If there isn't one already, a PR to fix or at least reproduce the problem in a test case will always help us get back on track to tackle this.

@stale stale bot added the stale label Jan 6, 2019
@rnorth
Copy link
Member

rnorth commented Jan 6, 2019

I think I’d like to keep this open - we should be able to upgrade the image. IIRC the last time I tried it wasn’t so simple and there were other issues, but I can’t remember the details.

Sent with GitHawk

@kiview
Copy link
Member

kiview commented Jan 6, 2019

I've added a label so stablebot won't remove it in the future, since I also agree that we should fix it at some point.

Maybe it would be best to support 3.x and 2.x, since both are in widespread use and serve slightly different use cases.

rnorth pushed a commit that referenced this issue Sep 9, 2019
…ge pull

Together with using Compose file 2.1 syntax, this is a solution to network cleanup issue described in:
 * #1767
 * #739
 * testcontainers/moby-ryuk#2
 * docker/compose#6636

Solution to general credential helper authenticated pull issues in:
 * docker/compose#5854

Tangentially should add support for v3 syntax (not yet tested) re #531
rnorth added a commit that referenced this issue Nov 23, 2019
* Upgrade docker-compose image to latest version and perform direct image pull

Together with using Compose file 2.1 syntax, this is a solution to network cleanup issue described in:
 * #1767
 * #739
 * testcontainers/moby-ryuk#2
 * docker/compose#6636

Solution to general credential helper authenticated pull issues in:
 * docker/compose#5854

Tangentially should add support for v3 syntax (not yet tested) re #531
@rnorth
Copy link
Member

rnorth commented Dec 7, 2019

@rnorth rnorth closed this as completed Dec 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants