Skip to content

ci(hermes): add swap for image exports #7380

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ericksoa merged 1 commit into from
Jul 22, 2026
Merged

ci(hermes): add swap for image exports #7380

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/workflows/sandbox-images-and-e2e.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,24 @@ jobs:
- name: Resolve Hermes base image
uses: ./.github/actions/resolve-hermes-base-image

- &hermes-export-swap
name: Add swap for Hermes image export
shell: bash
run: |
set -euo pipefail
swap_file=/mnt/nemoclaw-hermes-image-export.swap
free -h
df -h / /mnt
swapon --show
sudo fallocate -l 32G "$swap_file"
sudo chmod 0600 "$swap_file"
sudo mkswap "$swap_file"
sudo swapon "$swap_file"
swapon --show
free -h
df -h / /mnt
docker system df

- name: Build Hermes production image
env:
HERMES_BASE_IMAGE: ${{ env.HERMES_BASE_IMAGE }}
Expand Down Expand Up @@ -330,6 +348,8 @@ jobs:
node --experimental-strip-types scripts/check-messaging-plan-image-boundary.mts verify \
nemoclaw-openclaw-plan-boundary openclaw

- *hermes-export-swap

- name: Build and verify Hermes messaging plan boundary
env:
HERMES_BASE_IMAGE: ${{ env.HERMES_BASE_IMAGE }}
Expand Down
15 changes: 8 additions & 7 deletions test/e2e/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -410,13 +410,14 @@ base-build reason before the allowance applies. Published-image runs retain the
normal limits, and output silence, first-turn, and all other phase requirements
remain unchanged.

The two Hermes rebuild jobs add a bounded 32 GiB swap file on their ephemeral
hosted runners before invoking the live fixture. The fixture verifies that
floor and provisions the same swap file on GitHub Actions when a trusted
control-plane run uses the workflow definition from `main`. Those jobs build
both old and current Hermes image layers and can otherwise exhaust the runner's
default memory and swap during Docker layer export. Other E2E jobs keep the
standard runner memory configuration.
The two Hermes rebuild jobs and both reusable-workflow Hermes image exporters
add a bounded 32 GiB swap file on their ephemeral hosted runners before the
memory-heavy image build. The rebuild fixture verifies that floor and
provisions the same swap file on GitHub Actions when a trusted control-plane
run uses the workflow definition from `main`. Those paths build large Hermes
image layers and can otherwise exhaust the runner's default memory and swap
during Docker layer export. Other E2E jobs keep the standard runner memory
configuration.
Comment on lines +413 to +420

@coderabbitai coderabbitai Bot Jul 22, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Clarify the swap-floor sentence.

“Verifies that floor” is ambiguous. Name the contract explicitly so the documentation matches the validator:

Proposed wording 
-The rebuild fixture verifies that floor and provisions the same swap file
+The rebuild fixture verifies the 32 GiB swap floor and provisions the same swap file
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
The two Hermes rebuild jobs and both reusable-workflow Hermes image exporters
add a bounded 32 GiB swap file on their ephemeral hosted runners before the
memory-heavy image build. The rebuild fixture verifies that floor and
provisions the same swap file on GitHub Actions when a trusted control-plane
run uses the workflow definition from `main`. Those paths build large Hermes
image layers and can otherwise exhaust the runner's default memory and swap
during Docker layer export. Other E2E jobs keep the standard runner memory
configuration.
The two Hermes rebuild jobs and both reusable-workflow Hermes image exporters
add a bounded 32 GiB swap file on their ephemeral hosted runners before the
memory-heavy image build. The rebuild fixture verifies the 32 GiB swap floor and provisions the same swap file on GitHub Actions when a trusted control-plane
run uses the workflow definition from `main`. Those paths build large Hermes
image layers and can otherwise exhaust the runner's default memory and swap
during Docker layer export. Other E2E jobs keep the standard runner memory
configuration.
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/README.md` around lines 413 - 420, Update the rebuild fixture
sentence to explicitly state that it verifies a 32 GiB swap-file floor on GitHub
Actions, rather than saying it “verifies that floor.” Keep the existing trusted
control-plane and main-workflow conditions unchanged.


These assertions run inside the existing `full-e2e` lifecycle instead of a
second standalone onboarding run. This keeps the measurement on the job's first
Expand Down
16 changes: 16 additions & 0 deletions test/e2e/support/sandbox-images-workflow-boundary.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,22 @@ describe("sandbox image workflow boundary", () => {
);
});

it("requires bounded swap before every hosted Hermes image export", () => {
const { imageWorkflow, mainWorkflow } = readWorkflows();
for (const jobName of ["build-hermes-sandbox-image", "messaging-plan-image-boundary"]) {
const job = imageWorkflow.jobs[jobName];
const swap = job.steps!.find((step) => step.name === "Add swap for Hermes image export")!;
swap.run = swap.run!.replace('sudo swapon "$swap_file"', 'echo "swap omitted"');
}

expect(validateSandboxImagesWorkflow(imageWorkflow, mainWorkflow)).toEqual(
expect.arrayContaining([
'build-hermes-sandbox-image Hermes export swap must include sudo swapon "$swap_file"',
'messaging-plan-image-boundary Hermes export swap must include sudo swapon "$swap_file"',
]),
);
});

it("rejects coupling, rebuilding, or failing to reuse the OpenClaw image artifact", () => {
const { imageWorkflow, mainWorkflow } = readWorkflows();
const producer = imageWorkflow.jobs["build-sandbox-images"];
Expand Down
35 changes: 35 additions & 0 deletions tools/e2e/sandbox-images-workflow-boundary.mts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ const CLEANUP_RUN = "bash .github/scripts/docker-auth-cleanup.sh";
const HERMES_SECRET_BOUNDARY_STEP_ID = "hermes-secret-boundary";
const HERMES_ROOT_AFTER_SECRET_CONDITION =
"${{ !cancelled() && (steps.hermes-secret-boundary.outcome == 'success' || steps.hermes-secret-boundary.outcome == 'failure') }}";
const HERMES_EXPORT_SWAP_STEP_NAME = "Add swap for Hermes image export";
const MESSAGING_PLAN_IMAGE_BOUNDARY_JOB = "messaging-plan-image-boundary";
const IMAGE_BUILD_JOBS = [
"build-sandbox-images",
Expand Down Expand Up @@ -610,6 +611,39 @@ function validateMessagingPlanImageBoundary(
}
}

function validateHermesExportSwap(errors: string[], workflow: SandboxImagesWorkflow): void {
for (const [jobName, buildStepName] of [
["build-hermes-sandbox-image", "Build Hermes production image"],
[MESSAGING_PLAN_IMAGE_BOUNDARY_JOB, "Build and verify Hermes messaging plan boundary"],
] as const) {
const job = workflow.jobs[jobName] ?? {};
const swapSteps = steps(job).filter((step) => step.name === HERMES_EXPORT_SWAP_STEP_NAME);
if (swapSteps.length !== 1) {
errors.push(`${jobName} must provision Hermes export swap exactly once`);
continue;
}
const run = swapSteps[0]?.run ?? "";
for (const fragment of [
"swap_file=/mnt/nemoclaw-hermes-image-export.swap",
'sudo fallocate -l 32G "$swap_file"',
'sudo chmod 0600 "$swap_file"',
'sudo mkswap "$swap_file"',
'sudo swapon "$swap_file"',
"swapon --show",
"free -h",
"df -h / /mnt",
"docker system df",
]) {
if (!run.includes(fragment)) {
errors.push(`${jobName} Hermes export swap must include ${fragment}`);
}
}
if (stepIndex(job, HERMES_EXPORT_SWAP_STEP_NAME) >= stepIndex(job, buildStepName)) {
errors.push(`${jobName} must provision swap before the Hermes image build`);
}
}
}

function validateRuntimeImageReuse(errors: string[], workflow: SandboxImagesWorkflow): void {
const producerName = "build-sandbox-images";
const producer = workflow.jobs[producerName] ?? {};
Expand Down Expand Up @@ -969,6 +1003,7 @@ export function validateSandboxImagesWorkflow(
validateSecretScopeAndRegistryWrites(errors, workflow);
validateGuardedProductionBuildContracts(errors, workflow);
validateNodeTarImageScans(errors, workflow);
validateHermesExportSwap(errors, workflow);
validateMessagingPlanImageBoundary(errors, workflow);
validateRuntimeImageReuse(errors, workflow);
validateHermesImageReuse(errors, workflow);
Expand Down
Loading