Skip to content

e2e: fix test resource isolation and make standalone parallelism configurable - #14073

Merged
ndeloof merged 1 commit into
docker:mainfrom
glours:e2e-ci-optimizations-wave1-wave2
Aug 19, 2026
Merged

e2e: fix test resource isolation and make standalone parallelism configurable#14073
ndeloof merged 1 commit into
docker:mainfrom
glours:e2e-ci-optimizations-wave1-wave2

Conversation

@glours

@glours glours commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

What I did

Several e2e tests used fixed host ports (8070, 8080, 8090, etc.), global buildx
builders registered with --use, and a hardcoded IPC container name. Under
parallel execution these daemon-level resources caused port conflicts,
buildkitd container-name collisions, and intermittent failures. Separately,
make e2e-compose-standalone hardcoded -parallel=1, forcing the entire
standalone suite to run serially even though the same tests run in parallel in
plugin mode.

Changes:

  • Convert fixed host ports in four fixtures (network-test, sentences,
    build-test, volume-test) to ephemeral bindings. Add a ServicePublishedPort
    helper to framework.go to resolve the actual mapped port at test runtime.
  • Give each buildx-builder test a unique daemon-scoped name via a BuilderName
    helper, preventing container-name collisions between parallel goroutines.
    Covers TestBuildPlatformsWithCorrectBuildxConfig, TestBuildPrivileged,
    TestBuildBuilder, TestBuildEntitlements, TestBuildTLS.
  • Replace -parallel=1 in make e2e-compose-standalone with
    E2E_STANDALONE_PARALLEL?=4, aligning standalone with plugin mode on
    4-core GitHub Actions runners. E2E_PARALLEL_PLUGIN?=4 makes the
    plugin-mode knob explicit too. Both are overridable (E2E_STANDALONE_PARALLEL=1
    restores serial local debugging).
  • Raise the poll delay in watch_test.go long-running polls (90 s / 120 s
    timeout) from the default 100 ms to 1 s; bump cascade_test.go and
    publish_test.go from 100 ms to 500 ms. Timeouts are unchanged.

The logging-driver fixture keeps its fixed port 24224: the Docker fluentd
log driver encodes fluentd-address at service-start time, so ephemeral
resolution would require a multi-step compose up. TestLoggingDriver is
serial, so there is no collision risk in practice.

Related issue

No related issue — this is a test-infrastructure improvement.

(not mandatory) A picture of a cute animal, if possible in relation to what you did

A panda who has learned not to share its bamboo with parallel neighbours 🐼

panda

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟢 APPROVE

This PR correctly implements two waves of e2e CI optimisations. The diff was reviewed end-to-end across all 17 changed files.

Key changes verified:

  • ServicePublishedPort — Correctly omits -f in its compose port call. docker compose port resolves the project purely from the daemon by the com.docker.compose.project label when -p is provided (toProjectName returns early, backend.Port() never reads a compose file). The net.SplitHostPort parse handles both IPv4 and IPv6 address formats produced by Compose. t.Fatalf calls runtime.Goexit() so the zero-value return after it is unreachable — no silent zero-port risk.

  • BuilderName — Sanitisation and 60-char truncation are correct. All five call sites pass a top-level t (t.Name() = function name, e.g. TestBuildPlatformsWithCorrectBuildxConfig = 40 chars): the longest composed name is 56 chars, well below the truncation threshold. The resulting names are globally unique across parallel tests, compatible with Docker container-name and context-name character rules.

  • IPC fixtureipc: "container:ipc_e2e-src" matches exactly what the test creates (projectName+"-src" = "ipc_e2e-src"). The --rm flag on the source container and the final rm -f in cleanup coexist safely because the latter's result is discarded.

  • Makefile-parallel=$(E2E_PARALLEL_PLUGIN) and -parallel=$(E2E_STANDALONE_PARALLEL) are correctly positioned before ./pkg/e2e (test flags before package). --tags=standalone is also before the package path. TEST_FLAGS is only appended with -run/-skip, so no double -parallel issue in practice.

  • Ephemeral ports — All removed fixed-port bindings (8070, 8090, 9090, 90, 95, 8080 in fixtures) have corresponding call-site updates to ServicePublishedPort or relaxed assertions (0.0.0.0: vs 0.0.0.0:8080). No residual hardcoded port numbers remain for the changed fixtures.

  • Poll-delay calibrations — Raising delays from 100 ms → 500 ms–1 s reduces compose exec storm during long-running watch polls without changing test semantics.

@codecov

codecov Bot commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@glours
glours force-pushed the e2e-ci-optimizations-wave1-wave2 branch from fd49a3c to f8b90f3 Compare August 17, 2026 13:08
@glours glours self-assigned this Aug 17, 2026
@glours
glours force-pushed the e2e-ci-optimizations-wave1-wave2 branch from f8b90f3 to 75a7df2 Compare August 17, 2026 13:12
@glours glours changed the title e2e CI: parallelize standalone, increase plugin parallelism, fix shared state (wave 1+2) e2e: fix test resource isolation and make standalone parallelism configurable Aug 17, 2026
@glours
glours force-pushed the e2e-ci-optimizations-wave1-wave2 branch from 75a7df2 to 057be6f Compare August 17, 2026 13:20
@glours
glours marked this pull request as ready for review August 17, 2026 13:30
@glours
glours requested review from a team as code owners August 17, 2026 13:30
@glours
glours requested a review from ndeloof August 17, 2026 13:30

@docker-agent docker-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 No issues found — LGTM! View logs.

…igurable

Several e2e tests used fixed host ports (8070, 8080, 8090, etc.), global
buildx builders registered with --use, and a hardcoded IPC container name.
Under parallel execution these shared daemon-level resources caused port
conflicts, buildkitd container-name collisions, and intermittent failures.
Separately, the standalone make target hardcoded -parallel=1, forcing the
entire standalone suite to run serially even though the same tests run in
parallel in plugin mode.

Changes:
- Convert fixed host ports in four fixtures (network-test, sentences,
  build-test, volume-test) to ephemeral bindings; add ServicePublishedPort
  helper to framework.go to resolve the actual mapped port at runtime.
- Give each buildx-builder test a unique daemon-scoped name via BuilderName
  helper, preventing container-name collisions between parallel goroutines.
  Applies to TestBuildPlatformsWithCorrectBuildxConfig, TestBuildPrivileged,
  TestBuildBuilder, TestBuildEntitlements, TestBuildTLS.
- Rename the fixed ipc_mode_container to a project-scoped name
  (ipc_e2e-src) and update the ipc-test fixture accordingly.
- Replace the hardcoded -parallel=1 in make e2e-compose-standalone with
  E2E_PARALLEL_PLUGIN?=4 and E2E_STANDALONE_PARALLEL?=4, aligning both
  modes with the effective GOMAXPROCS on 4-core GitHub Actions runners.
  Both variables are overridable (E2E_STANDALONE_PARALLEL=1 restores
  serial local debugging).
- Raise the poll delay in watch_test.go long-running polls (90s/120s
  timeout) from the default 100ms to 1s; bump cascade_test.go and
  publish_test.go from 100ms to 500ms. Reduces CPU pressure without
  affecting effective test duration.

The logging-driver fixture retains its fixed port 24224: the Docker fluentd
log driver encodes fluentd-address at service-start time, making ephemeral
resolution non-trivial. TestLoggingDriver is serial so there is no
collision risk in practice.

Signed-off-by: Guillaume Lours <glours@users.noreply.github.com>
@glours
glours force-pushed the e2e-ci-optimizations-wave1-wave2 branch from 057be6f to 797c4a3 Compare August 19, 2026 13:39
@ndeloof
ndeloof merged commit 8bd79da into docker:main Aug 19, 2026
47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants