Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 113 additions & 0 deletions .base/.github/workflows/build-worker.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,25 @@ on:
build_contexts: |
repo_root=.
third_party=./vendor
extra_stages:
required: false
type: string
default: ""
description: |
Comma-separated list of extra Dockerfile stages to build after
the standard devel/runtime pipeline. For each stage <name>, if
a corresponding <name>-test stage exists in the Dockerfile, it
is built first (ephemeral, same as devel-test / runtime-test
convention).

Each extra stage gets its own GHA cache scope
(<cache-key>-<name>-cache / <cache-key>-<name>-test-cache).

Stages in the standard pipeline blocklist (sys, devel-base,
devel, devel-test, runtime-test, runtime) are rejected.

Default empty string preserves existing behavior — no extra
stages are built. Closes #415.
cache_variant:
required: false
type: string
Expand Down Expand Up @@ -455,6 +474,100 @@ jobs:
USER_GID=1000
${{ inputs.build_args }}

# ────────────────────────────────────────────────────────────
# #415 — Extra stages loop.
#
# Iterates over the comma-separated `extra_stages` input and
# builds each stage (plus its <name>-test companion if present
# in the Dockerfile). Uses `docker buildx build` directly
# because GHA YAML does not support dynamic step generation.
# GHA cache (`type=gha`) works because docker/setup-buildx-action
# already configured the builder with the cache backend.
#
# Blocklist prevents re-building stages owned by the standard
# pipeline. Test-stage detection reuses the same `FROM ... AS`
# regex pattern as setup.sh's _parse_dockerfile_stages().
# ────────────────────────────────────────────────────────────
- name: Build extra stages
if: ${{ inputs.extra_stages != '' }}
env:
EXTRA_STAGES: ${{ inputs.extra_stages }}
CACHE_KEY: ${{ steps.cache.outputs.key }}
CONTEXT_PATH: ${{ inputs.context_path }}
DOCKERFILE: ${{ inputs.dockerfile_path || format('{0}/Dockerfile', inputs.context_path) }}
BUILD_CONTEXTS_INPUT: ${{ inputs.build_contexts }}
BUILD_ARGS_INPUT: ${{ inputs.build_args }}
PLATFORM: ${{ matrix.platform }}
HARDWARE: ${{ matrix.hardware }}
TEST_TOOLS_IMAGE: "ghcr.io/ycpss91255-docker/test-tools:${{ inputs.test_tools_version }}"
run: |
set -euo pipefail

blocklist="sys devel-base devel devel-test runtime-test runtime"

available_stages=$(grep -iE '^FROM[[:space:]]+[^[:space:]]+[[:space:]]+AS[[:space:]]+' \
"${DOCKERFILE}" | \
sed -E 's/^FROM[[:space:]]+[^[:space:]]+[[:space:]]+AS[[:space:]]+([^[:space:]#]+).*/\1/i')

build_arg_flags=(
--build-arg "USER_NAME=ci"
--build-arg "USER_GROUP=ci"
--build-arg "USER_UID=1000"
--build-arg "USER_GID=1000"
--build-arg "HARDWARE=${HARDWARE}"
--build-arg "TEST_TOOLS_IMAGE=${TEST_TOOLS_IMAGE}"
)
while IFS= read -r line; do
[ -z "${line}" ] && continue
build_arg_flags+=(--build-arg "${line}")
done <<< "${BUILD_ARGS_INPUT}"

build_ctx_flags=()
if [ -n "${BUILD_CONTEXTS_INPUT}" ]; then
while IFS= read -r line; do
[ -z "${line}" ] && continue
build_ctx_flags+=(--build-context "${line}")
done <<< "${BUILD_CONTEXTS_INPUT}"
fi

IFS=',' read -ra stages <<< "${EXTRA_STAGES}"
for stage in "${stages[@]}"; do
stage="$(echo "${stage}" | tr -d '[:space:]')"
[ -z "${stage}" ] && continue

for blocked in ${blocklist}; do
if [ "${stage}" = "${blocked}" ]; then
echo "::error::extra_stages contains blocked stage '${stage}'. Use the standard pipeline inputs instead."
exit 1
fi
done

test_stage="${stage}-test"
if echo "${available_stages}" | grep -qx "${test_stage}"; then
echo "::group::Build ${test_stage} (extra stage test)"
docker buildx build \
--file "${DOCKERFILE}" \
--target "${test_stage}" \
--platform "${PLATFORM}" \
--cache-from "type=gha,scope=${CACHE_KEY}-${test_stage}-cache" \
--cache-to "type=gha,scope=${CACHE_KEY}-${test_stage}-cache,mode=max" \
"${build_arg_flags[@]}" "${build_ctx_flags[@]}" \
"${CONTEXT_PATH}"
echo "::endgroup::"
fi

echo "::group::Build ${stage} (extra stage)"
docker buildx build \
--file "${DOCKERFILE}" \
--target "${stage}" \
--platform "${PLATFORM}" \
--cache-from "type=gha,scope=${CACHE_KEY}-${stage}-cache" \
--cache-to "type=gha,scope=${CACHE_KEY}-${stage}-cache,mode=max" \
"${build_arg_flags[@]}" "${build_ctx_flags[@]}" \
"${CONTEXT_PATH}"
echo "::endgroup::"
done

# ────────────────────────────────────────────────────────────────
# Stable-name aggregator so downstream branch-protection rules
# keep working. Keeping this job's name as `docker-build`
Expand Down
13 changes: 4 additions & 9 deletions .base/.hadolint.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,7 @@
---
ignored:
- DL3003 # Use WORKDIR to switch to a directory
- DL3004 # Do not use sudo (we need it for user setup)
- DL3006 # Always tag the version of an image explicitly
- DL3007 # Using latest is prone to errors
- DL3008 # Pin versions in apt get install
- DL3013 # Pin versions in pip install
- DL3018 # Pin versions in apk add
- DL3046 # useradd without flag -l
- DL4006 # Set SHELL option -o pipefail (alpine stages use default shell)
- DL3004 # Do not use sudo (dev container passwordless sudo is by design)
- DL3008 # Pin versions in apt get install (impractical for dev images on rolling base)
- DL3013 # Pin versions in pip install (downstream repo responsibility)
- DL3018 # Pin versions in apk add (aligned with DL3008 strategy)

2 changes: 1 addition & 1 deletion .base/.version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
v0.34.0
v0.35.0
4 changes: 2 additions & 2 deletions .base/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -285,7 +285,7 @@ Allowlist (v1 — keys that can be overridden per-stage):
|---|---|
| `[deploy]` | `gpu_mode`, `gpu_count`, `gpu_capabilities`, `runtime` |
| `[gui]` | `mode` |
| `[network]` | `mode`, `ipc`, `network_name`, `port_<N>`, `port_inherit` |
| `[network]` | `mode`, `ipc`, `pid`, `network_name`, `port_<N>`, `port_inherit` |
| `[security]` | `privileged` |
| `[volumes]` | `mount_<N>`, `mount_inherit` |
| `[environment]` | `env_<N>`, `env_inherit` |
Expand Down Expand Up @@ -348,7 +348,7 @@ two derived artifacts.
[build] apt_mirror_ubuntu, apt_mirror_debian # Dockerfile build args
[deploy] gpu_mode (auto|force|off), gpu_count, gpu_capabilities
[gui] mode (auto|force|off)
[network] mode (host|bridge|none), ipc, privileged
[network] mode (host|bridge|none), ipc, pid (host|private), privileged
[volumes] mount_1 (workspace, auto-populated on first run)
mount_2..mount_N (extra host mounts; devices via /dev path)
[logging] driver (json-file default), max_size, max_file, compress
Expand Down
1 change: 1 addition & 0 deletions .base/config/docker/setup.conf
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ mode = auto
[network]
mode = host
ipc = host
pid = private
network_name =

# ═════════════════════════════════════════════════════════════════════
Expand Down
36 changes: 36 additions & 0 deletions .base/doc/changelog/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,42 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [v0.35.0] - 2026-05-27

### Added
- **`[network] pid` setting** for PID namespace mode (`host` / `private`). Default `private` (Docker default). Set `pid = host` when running multiple GPU-rendering containers on the same GPU to avoid NVIDIA driver pthread robust mutex failures (`ESRCH`). Follows the `[network] ipc` precedent: setup.conf -> `.env` `PID_MODE` -> compose.yaml `pid:`. Per-stage override and TUI selection (4 languages) included. Closes #412.
- **`build-worker.yaml` `extra_stages` input** — opt-in comma-separated list of extra Dockerfile stages to build after the standard pipeline. For each stage `<name>`, if a corresponding `<name>-test` stage exists in the Dockerfile it is built first (same convention as `devel-test` / `runtime-test`). Each extra stage gets its own GHA cache scope. Blocklist validation rejects attempts to re-build standard pipeline stages. Closes #415.

### Changed
- **`.hadolint.yaml` cleanup** — removed 5 globally ignored rules (DL3003, DL3006, DL3007, DL3046, DL4006) and properly fixed the underlying violations: pinned bats/alpine versions via `ARG` in `Dockerfile.test-tools`, added `-l` flag to `useradd` in `Dockerfile.example`, replaced `RUN cd` with `WORKDIR`, and moved DL4006 to inline ignore on the Alpine `RUN` with pipe. Closes #405.

### Removed
- **`dockerfile/setup/` pip scaffolding** — removed entirely (reverses #261). `python3-pip` dropped from `Dockerfile.example` apt install, `SETUP_DIR` ARG and all COPY/RUN pip references removed. Downstream repos that need pip handle it independently in their own Dockerfiles. Closes #407.

### Fixed
- **Makefile forwarding: absolute container paths** — `make exec -- /root/demo/test.sh` failed with `No rule to make target` because GNU Make's built-in implicit rules stat every goal on the host filesystem. Added `--no-builtin-rules` + `.SUFFIXES:` so the `%:` catch-all fires correctly for arbitrary absolute paths. Closes #414 (case 2).
- **Makefile forwarding: VAR=VALUE args silently lost** — `make setup set build.arg_4 ROS2_DISTRO=jazzy` dropped the `ROS2_DISTRO=jazzy` token because Make treats any `KEY=VALUE` CLI token as a variable override, not a goal. Added a `MAKEOVERRIDES` guard that detects swallowed args and aborts with a clear error message pointing users to the underlying script. Closes #414 (case 1).
- `upgrade.sh` #399 idempotency regex false-positive: `COPY script/*.sh /lint/script/` was matching the already-patched check because `/lint/` is a prefix of `/lint/script/`. Anchored the regex with `$` so only the exact `/lint/` destination triggers the skip. Closes #403.

## [v0.34.1] - 2026-05-25

Patch release: single feature from #399 / PR #400.

### Added

- **`upgrade.sh` auto-patches downstream Dockerfile `COPY *.sh /lint/`
→ `COPY script/*.sh /lint/`** (closes #399). v0.31.0 (#330) moved
user-facing wrappers from the repo root into a `script/` subfolder;
`init.sh` migrates the symlinks but the Dockerfile's COPY directive
is user-owned and stayed anchored at root, pulling zero files after
the migration. Every post-#330 fanout hit the same smoke-test
failure (`build.sh -h exits 0` → `assert_success` failed because
`/lint/build.sh` did not exist). `upgrade.sh` now detects the stale
`COPY *.sh /lint/` pattern and rewrites it to `COPY script/*.sh
/lint/` in the same chore commit, so the next fanout is clean.
Idempotent: already-patched Dockerfiles are skipped. Modelled on
the #348 `COPY .base/script/docker/lib` sibling patch.

## [v0.34.0] - 2026-05-21

Stable v0.34.0 minor feature release, promoting v0.34.0-rc1 (#396) with no follow-up fixes — RC tag CI (`Self Test` + `release-test-tools`) was green.
Expand Down
4 changes: 2 additions & 2 deletions .base/doc/readme/README.ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,7 @@ Override 可能な key (v1):
|---|---|
| `[deploy]` | `gpu_mode`, `gpu_count`, `gpu_capabilities`, `runtime` |
| `[gui]` | `mode` |
| `[network]` | `mode`, `ipc`, `network_name`, `port_<N>`, `port_inherit` |
| `[network]` | `mode`, `ipc`, `pid`, `network_name`, `port_<N>`, `port_inherit` |
| `[security]` | `privileged` |
| `[volumes]` | `mount_<N>`, `mount_inherit` |
| `[environment]` | `env_<N>`, `env_inherit` |
Expand Down Expand Up @@ -314,7 +314,7 @@ assertion helpers のセットを提供します。ダウンストリーム repo
[build] apt_mirror_ubuntu、apt_mirror_debian # Dockerfile build args
[deploy] gpu_mode (auto|force|off)、gpu_count、gpu_capabilities
[gui] mode (auto|force|off)
[network] mode (host|bridge|none)、ipc、privileged
[network] mode (host|bridge|none)、ipc、pid (host|private)、privileged
[volumes] mount_1(workspace、初回 setup.sh 実行時に自動記入)
mount_2..mount_N(ユーザ定義の追加 host mount;/dev デバイスは path 指定)
[logging] driver(デフォルト json-file)、max_size、max_file、compress
Expand Down
4 changes: 2 additions & 2 deletions .base/doc/readme/README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ deploy.gpu_capabilities = gpu compute utility graphics video
|---|---|
| `[deploy]` | `gpu_mode`, `gpu_count`, `gpu_capabilities`, `runtime` |
| `[gui]` | `mode` |
| `[network]` | `mode`, `ipc`, `network_name`, `port_<N>`, `port_inherit` |
| `[network]` | `mode`, `ipc`, `pid`, `network_name`, `port_<N>`, `port_inherit` |
| `[security]` | `privileged` |
| `[volumes]` | `mount_<N>`, `mount_inherit` |
| `[environment]` | `env_<N>`, `env_inherit` |
Expand Down Expand Up @@ -302,7 +302,7 @@ assertion helpers。下游 repo 应优先使用这些 helper 而非原生的
[build] apt_mirror_ubuntu、apt_mirror_debian # Dockerfile build args
[deploy] gpu_mode (auto|force|off)、gpu_count、gpu_capabilities
[gui] mode (auto|force|off)
[network] mode (host|bridge|none)、ipc、privileged
[network] mode (host|bridge|none)、ipc、pid (host|private)、privileged
[volumes] mount_1(workspace,首次 setup.sh 执行时自动填入)
mount_2..mount_N(用户自定义额外 host mount;/dev 设备走 path)
[logging] driver(默认 json-file)、max_size、max_file、compress
Expand Down
4 changes: 2 additions & 2 deletions .base/doc/readme/README.zh-TW.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ deploy.gpu_capabilities = gpu compute utility graphics video
|---|---|
| `[deploy]` | `gpu_mode`, `gpu_count`, `gpu_capabilities`, `runtime` |
| `[gui]` | `mode` |
| `[network]` | `mode`, `ipc`, `network_name`, `port_<N>`, `port_inherit` |
| `[network]` | `mode`, `ipc`, `pid`, `network_name`, `port_<N>`, `port_inherit` |
| `[security]` | `privileged` |
| `[volumes]` | `mount_<N>`, `mount_inherit` |
| `[environment]` | `env_<N>`, `env_inherit` |
Expand Down Expand Up @@ -302,7 +302,7 @@ assertion helpers。下游 repo 應優先使用這些 helper 而非原生的
[build] apt_mirror_ubuntu、apt_mirror_debian # Dockerfile build args
[deploy] gpu_mode (auto|force|off)、gpu_count、gpu_capabilities
[gui] mode (auto|force|off)
[network] mode (host|bridge|none)、ipc、privileged
[network] mode (host|bridge|none)、ipc、pid (host|private)、privileged
[volumes] mount_1(workspace,首次 setup.sh 執行時自動填入)
mount_2..mount_N(使用者自訂額外 host mount;/dev 裝置走 path)
[logging] driver(預設 json-file)、max_size、max_file、compress
Expand Down
Loading