Skip to content

fix(OMN-13946): add libatomic1 to runner image (Pyright node runtime defect) - #2207

Merged
jonahgabriel merged 6 commits into
devfrom
jonah/omn-13946-runner-image-libatomic
Jul 9, 2026
Merged

jonahgabriel merged 6 commits into
devfrom
jonah/omn-13946-runner-image-libatomic

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Jul 5, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Self-hosted CI runner image's bundled GH Actions node runtime (used by node-based actions, e.g. pyright) dlopens libatomic.so.1 at load time; Ubuntu 22.04 doesn't pull libatomic1 in transitively. Every node-based action on the runner image fails deterministically with node: libatomic.so.1: cannot open shared object file — real, not transient. Blocks omniclaude PR chore(OMN-12667): bump zone-filter workflow ref #1860 (OMN-13502) and any omniclaude PR that runs pyright.
  • Adds libatomic1 to docker/runners/Dockerfile apt install list.
  • Bumps image_version 5->6 in docker/runners/runner-image.lock.json (real image content change) and regenerates the bound identity_digest via scripts/ci/runner_image_identity.py --mode generate. The Dockerfile apt-list is not itself a binding input, so identity_digest only moves because image_version does; shared_env_digest is unchanged.
  • Updates the OMN-12765 release-backmerge identity-pin test (tests/integration/infra/test_omn_12765_release_backmerge_identity.py) to the regenerated digest.

Scope

Code fix + identity lock only. Image rebuild + fleet redeploy on .201 is a separate gated live-runtime step, out of scope here.

Evidence-Ticket: OMN-13946
Evidence-Source: OCC#3608

Test plan

  • uv run pytest tests/ci/test_runner_image_identity.py tests/ci/test_validate_runner_image.py tests/ci/test_runner_image_node24_floor.py -v — 36 passed
  • uv run pytest tests/integration/infra/test_omn_12765_release_backmerge_identity.py -v — 2 passed
  • uv run pytest tests/ -n auto — 24808 passed / 19 failed / 78 errors, all pre-existing environment-dependent (live Kafka/Postgres/Infisical/runtime container) failures unrelated to this change
  • uv run mypy src/omnibase_infra/ — no issues
  • pre-commit run --files docker/runners/Dockerfile docker/runners/runner-image.lock.json — all applicable hooks pass

Summary by CodeRabbit

  • Bug Fixes
    • Improved workflow reliability by retrying dependency installation with backoff, reducing failures from transient network or package issues.
    • Updated the runner environment so required system libraries are available for actions that depend on them.
  • Chores
    • Refreshed the runner image reference to a newer version.
  • Tests
    • Adjusted integration coverage to match the updated runner image and environment expectations.

@coderabbitai

coderabbitai Bot commented Jul 5, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 15 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 22f6a6c2-b867-4d07-bd80-aea1f082c5b6

📥 Commits

Reviewing files that changed from the base of the PR and between 32ea48e and 788f41e.

📒 Files selected for processing (5)
  • docker/runners/Dockerfile
  • docker/runners/runner-image.lock.json
  • src/omnibase_infra/cli/cli_skill.py
  • src/omnibase_infra/cli/omnimarket_drift_guard.py
  • tests/integration/infra/test_omn_12765_release_backmerge_identity.py
📝 Walkthrough

Walkthrough

This PR adds retry-with-backoff logic to uv sync --all-extras steps in three CI workflows and sets a global UV_HTTP_TIMEOUT env var. Separately, it adds libatomic1 to the runner Dockerfile, bumps the runner image lock's digest/version, and updates the corresponding integration test.

Changes

CI Workflow Retry Logic

Layer / File(s) Summary
Retry loop and timeout for uv sync
.github/workflows/url-authority-gate.yml, .github/workflows/validate-validator-requirements.yml, .github/workflows/validator-runtime-profiles.yml
Replaces single uv sync --all-extras calls with 3-attempt retry loops using incremental sleep backoff, failing the step on the third failed attempt; also adds UV_HTTP_TIMEOUT: "600" env var to one workflow.

Runner Image libatomic1 Fix

Layer / File(s) Summary
Dockerfile dependency addition
docker/runners/Dockerfile
Adds libatomic1 to the apt-get install list with comments explaining the Node runtime's need for libatomic.so.1.
Lock and test update
docker/runners/runner-image.lock.json, tests/integration/infra/test_omn_12765_release_backmerge_identity.py
Bumps identity_digest and image_version (5→6) in the lock file and updates the corresponding test assertion and comment.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

  • OmniNode-ai/omnibase_infra#2115: Both PRs update docker/runners/runner-image.lock.json and the corresponding integration test assertions to match regenerated runner lock digests.
  • OmniNode-ai/omnibase_infra#2122: Both PRs update the runner image identity digest in the lock file with corresponding test assertion updates.
  • OmniNode-ai/omnibase_infra#1887: Both PRs update the runner Dockerfile to add a runtime dependency and the resulting lock file digest and tests.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: adding libatomic1 to the runner image to fix the Node runtime defect.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jonah/omn-13946-runner-image-libatomic

Comment @coderabbitai help to get the list of available commands.

@jonahgabriel
jonahgabriel enabled auto-merge July 5, 2026 04:24
@jonahgabriel
jonahgabriel added this pull request to the merge queue Jul 5, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 5, 2026
@jonahgabriel
jonahgabriel added this pull request to the merge queue Jul 5, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Jul 5, 2026
@jonahgabriel
jonahgabriel force-pushed the jonah/omn-13946-runner-image-libatomic branch from 17a6d3e to 375ed99 Compare July 5, 2026 07:21
@jonahgabriel
jonahgabriel enabled auto-merge July 5, 2026 07:21
@jonahgabriel
jonahgabriel added this pull request to the merge queue Jul 5, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 5, 2026
@jonahgabriel
jonahgabriel added this pull request to the merge queue Jul 5, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 5, 2026
@jonahgabriel
jonahgabriel added this pull request to the merge queue Jul 5, 2026
@jonahgabriel
jonahgabriel removed this pull request from the merge queue due to a manual request Jul 5, 2026
@jonahgabriel
jonahgabriel force-pushed the jonah/omn-13946-runner-image-libatomic branch from 375ed99 to 32e96b5 Compare July 5, 2026 09:00
@jonahgabriel
jonahgabriel added this pull request to the merge queue Jul 5, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 5, 2026
@jonahgabriel
jonahgabriel enabled auto-merge July 5, 2026 09:30

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
.github/workflows/validate-validator-requirements.yml (2)

54-60: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Consider also setting UV_HTTP_TIMEOUT here for consistency.

url-authority-gate.yml bumps UV_HTTP_TIMEOUT to "600" alongside its retry loop, but this workflow's uv sync --all-extras still relies on uv's default 30s HTTP read timeout. If the underlying flakiness is timeout-related, this workflow could still fail on slow reads before the retry loop even gets a chance to matter.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/validate-validator-requirements.yml around lines 54 - 60,
The retry loop around uv sync in the workflow should also set UV_HTTP_TIMEOUT
for consistency with the other workflow. Update the
validate-validator-requirements job so the environment used by the uv sync
--all-extras step includes UV_HTTP_TIMEOUT set to 600, matching the retry
behavior and preventing uv’s default read timeout from causing premature
failures.

54-60: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Retry loop is duplicated across three workflows; consider extracting.

The identical 6-line retry block is now copy-pasted in url-authority-gate.yml, validate-validator-requirements.yml, and validator-runtime-profiles.yml. Consider extracting it into a reusable composite action (e.g. .github/actions/uv-sync-with-retry) to avoid drift when backoff/attempt tuning changes later.

♻️ Example composite action
# .github/actions/uv-sync-with-retry/action.yml
name: uv sync with retry
runs:
  using: composite
  steps:
    - shell: bash
      run: |
        for attempt in 1 2 3; do
          uv sync --all-extras && break
          if [ "$attempt" = "3" ]; then
            exit 1
          fi
          sleep "$((attempt * 10))"
        done
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/validate-validator-requirements.yml around lines 54 - 60,
The retry loop for `uv sync --all-extras` is duplicated and should be
centralized. Extract the shared logic from the workflow step into a reusable
composite action such as `.github/actions/uv-sync-with-retry/action.yml`, then
update the workflow that contains this block to call that action instead of
inlining the loop. Keep the behavior and symbols around the `uv sync` step, but
make the backoff/attempt handling live in one place so the other workflows can
reuse it consistently.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In @.github/workflows/validate-validator-requirements.yml:
- Around line 54-60: The retry loop around uv sync in the workflow should also
set UV_HTTP_TIMEOUT for consistency with the other workflow. Update the
validate-validator-requirements job so the environment used by the uv sync
--all-extras step includes UV_HTTP_TIMEOUT set to 600, matching the retry
behavior and preventing uv’s default read timeout from causing premature
failures.
- Around line 54-60: The retry loop for `uv sync --all-extras` is duplicated and
should be centralized. Extract the shared logic from the workflow step into a
reusable composite action such as
`.github/actions/uv-sync-with-retry/action.yml`, then update the workflow that
contains this block to call that action instead of inlining the loop. Keep the
behavior and symbols around the `uv sync` step, but make the backoff/attempt
handling live in one place so the other workflows can reuse it consistently.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b96e6c7c-a750-4739-a9a4-b2f1b8651987

📥 Commits

Reviewing files that changed from the base of the PR and between 15ba4f3 and 32ea48e.

📒 Files selected for processing (6)
  • .github/workflows/url-authority-gate.yml
  • .github/workflows/validate-validator-requirements.yml
  • .github/workflows/validator-runtime-profiles.yml
  • docker/runners/Dockerfile
  • docker/runners/runner-image.lock.json
  • tests/integration/infra/test_omn_12765_release_backmerge_identity.py

…defect)

The self-hosted CI runner image's bundled GitHub Actions node runtime
(used by node-based actions, e.g. pyright) dlopens libatomic.so.1 at
load time. Ubuntu 22.04 does not pull libatomic1 in transitively, so
every node-based action fails with "node: libatomic.so.1: cannot open
shared object file" — deterministic, not transient. Blocks omniclaude
PR #1860 (OMN-13502) and any omniclaude PR that runs pyright.

Add libatomic1 to the runner image apt install list, bump
image_version 5->6 (real image content change), and regenerate the
bound identity (docker/runners/runner-image.lock.json). The Dockerfile
apt-list is not itself a binding input, so identity_digest only moves
because image_version does; shared_env_digest is unchanged.

Updates the OMN-12765 release-backmerge identity-pin test to the
regenerated digest.

Rebuild + fleet redeploy on .201 is a separate gated live-runtime step,
out of scope here.

Evidence-Source: OCC#TBD
Evidence-Ticket: OMN-13946
@jonahgabriel
jonahgabriel force-pushed the jonah/omn-13946-runner-image-libatomic branch from 32ea48e to eb557d3 Compare July 8, 2026 07:01
@jonahgabriel
jonahgabriel disabled auto-merge July 8, 2026 17:01
@jonahgabriel
jonahgabriel merged commit 4e9655d into dev Jul 9, 2026
142 of 146 checks passed
@jonahgabriel
jonahgabriel deleted the jonah/omn-13946-runner-image-libatomic branch July 9, 2026 09:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant