Skip to content

[CI][AMD] Support build-scoped ROCm images - #447

Closed
AndreasKaratzas wants to merge 11 commits into
mainfrom
akaratza_rocm_image_promotion_infra
Closed

AndreasKaratzas wants to merge 11 commits into
mainfrom
akaratza_rocm_image_promotion_infra

Conversation

@AndreasKaratzas

@AndreasKaratzas AndreasKaratzas commented Aug 3, 2026 •

Copy link
Copy Markdown
Member

The vLLM pipeline needs a main-only, globally serialized promotion job, but the current V2 generator does not preserve its condition and concurrency fields through every rendering path. The existing timeout logic also assumes a Python-only base-refresh job is always short, while the new selector can turn any confirmed registry miss into a full rebuild. This adds validated if_condition, concurrency, and concurrency_group fields and carries them through normal commands, direct AMD jobs, and AMD mirrors. if_condition is emitted as Buildkite's if field. The promotion job is always selected, with its condition excluding PR builds when the pipeline is uploaded.

Every non-skipped base-selection job now gets the 540-minute timeout. Native and DinD AMD jobs use rocm/vllm-dev:ci_base-build-$BUILDKITE_BUILD_ID, which is published by the companion vLLM build. This keeps each evaluation on the ci_base it selected instead of a stable or cross-build tag.

@AndreasKaratzas
AndreasKaratzas marked this pull request as ready for review August 3, 2026 03:47
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
@AndreasKaratzas
AndreasKaratzas force-pushed the akaratza_rocm_image_promotion_infra branch from 7ee41ef to e1506ae Compare August 3, 2026 04:02
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Resolve ROCm pipeline conflicts while preserving current build-scoped behavior and the promotion-capable handoff.

Assisted-by: OpenAI Codex
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
Raise the native AMD tmpfs /dev/shm limit from 16 GiB to 32 GiB in both pipeline generators.

Assisted-by: OpenAI Codex
Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
@AndreasKaratzas
AndreasKaratzas force-pushed the akaratza_rocm_image_promotion_infra branch from 896ec73 to e51373b Compare August 17, 2026 06:44
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
@AndreasKaratzas
AndreasKaratzas force-pushed the akaratza_rocm_image_promotion_infra branch from e51373b to 32d2108 Compare August 17, 2026 08:02
AndreasKaratzas and others added 4 commits August 17, 2026 21:46
…romotion_infra

Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>

# Conflicts:
#	buildkite/bootstrap-amd.sh
#	buildkite/pipeline_generator/amd.py
#	buildkite/tests/pipeline_generator/test_amd.py
Keep the PR's build-scoped images and promotion step alongside main's
native image verification and current pipeline generator behavior.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Recognize if_condition as an upload-time Buildkite gate so conditional steps are not force-selected as unknown fields. Allow the promotion step to be absent only on vLLM checkouts without promotion support, while retaining drift checks for established AMD steps and promotion-capable checkouts.

Add regression coverage for conditional-step preflight and legacy/promotion-capable checkout contracts.

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
@AndreasKaratzas

Copy link
Copy Markdown
Member Author

@AndreasKaratzas
AndreasKaratzas deleted the akaratza_rocm_image_promotion_infra branch October 19, 2026 05:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant