Skip to content

fix(OMN-20255): the .201 dev deploy agent stages workspace builds from the deploy-source tree, not the lanes' canonical clones - #4401

Merged
jonahgabriel merged 2 commits into
devfrom
jonah/omn-20255-deploy-agent-omni-home
Oct 1, 2026
Merged

jonahgabriel merged 2 commits into
devfrom
jonah/omn-20255-deploy-agent-omni-home

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Ticket OMN-20255 (parent OMN-17291). On .201 the dev-lane deploy agent staged every workspace build from the lanes' canonical omni_home tree instead of the deploy-source tree, so each dev job left omnimarket, omnibase_core and omnibase_compat in that tree on a detached HEAD (and ran clean -ffdx in them). While omnimarket is detached, the clone guard refuses every in-process onex delegate on the host. This pins the agent's OMNI_HOME to the deploy-source root that reconcile-host converges and locks under.

Cause (evidence)

The deploy-agent-dev systemd user unit sources DEPLOY_AGENT_ENV_FILE=~/.omnibase/.env, which sets OMNI_HOME=/data/omninode/omni_home. The deploy-source tree reconcile-host converges hourly on /data/omninode (where OMNI_HOME=/data/omninode is set in /data/omninode/omnibase_infra/.env). The agent takes its OMN-20154 build lock at /data/omninode/omni_home/.onex-reconcile-host.lock, while reconcile-host takes its lock at /data/omninode/.onex-reconcile-host.lock, so the interlock never engaged.

Evidence from 2026-10-01 shows dev jobs accepted at 03:44:10Z, 04:12:03Z, 04:38:23Z, and 05:05:13Z were each followed 13-14 seconds later by a 'checkout: moving from dev to ' entry in the reflogs of the clones under OMNI_HOME (omnibase_core, omnibase_compat, omnimarket). The agent journal logs 'staging: /data/omninode/omni_home/omnimarket -> workspace/sibling-repos/omnimarket'. The RT-1 step (stage_workspace.sh -> deploy_source_ref.py clean checkout) runs checkout --force, reset --hard, and clean -ffdx on each vendored sibling under OMNI_HOME, leaving them on a detached HEAD. While omnimarket is detached, the clone guard refuses every in-process onex delegate on the host.

Change

The deploy-agent-dev.service unit now declares Environment=OMNI_HOME=/data/omninode and adds OMNI_HOME to DEPLOY_AGENT_ENV_PROTECTED so the sourced env file cannot override it. A new test script scripts/deploy-agent/tests/unit/test_dev_unit_omni_home_deploy_source_omn20255.py is added with 4 tests: two positive controls verifying the launcher sources the env store and that both locks derive from OMNI_HOME, plus tests for the declared value and its protection.

Out of scope

DEPLOY_AGENT_REPO_DIR still defaults to /data/omninode/omni_home/omnibase_infra (the dev lane compose tree and its bind mounts). The git phase still runs reset --hard on that clone's dev branch; it does not detach it.

Verification

The new test is RED against the origin/dev unit (2 failed, 2 controls passed) and GREEN with the related unit tests (82 passed: OMN-19522 PATH, launcher OMN-18073, unit drift, instances-as-data OMN-19543, service source). The deploy-source clones /data/omninode/{omnimarket,omnibase_core,omnibase_compat} were verified clean, on dev, with no ignored files, worktrees, or processes using them, and deploy_source_ref.py reconcile re-attaches a detached clone.

Lab proof (.201, 2026-10-01)

  • Actor confirmed a fourth time before the fix: dev job c667989c (accepted 05:05:13Z, gha/omnibase_infra/pr-4385) was followed at 05:05:27-28Z by checkout: moving from dev to <sha> in all three lanes' clones.
  • This unit was installed on .201 at the agent's first idle moment (05:56:12Z) and the agent restarted. Readback: /proc/613981/environ held OMNI_HOME=/data/omninode.
  • Six seconds later, the agent's own self_update[boundary=pre_accept] found the installed unit drifted from the tracked copy at origin/dev and restored it (synced unit deploy-agent-dev.service; restarting via systemd). The next process (pid 666772) read OMNI_HOME=/data/omninode/omni_home again. So no hand install survives before this merges.
  • That same path is the delivery mechanism after merge: the agent pulls origin/dev, syncs this unit and restarts itself. Readback of the post-merge agent environ, and the three lanes' clones staying on dev across the next two dev-lane jobs, will be posted on OMN-20255.

The backup of the pre-install unit is ~/.config/systemd/user/deploy-agent-dev.service.bak-pre-omn20255 on .201.

PR text drafted by onex delegate run 633f948f-c8fe-4750-b3ec-d9475693cd81 (local Qwen3.8-27B), reviewed by the lane.

Evidence-Ticket: OMN-20255
Evidence-Source: OCC#12072

…m the deploy-source tree, not the lanes' canonical clones

deploy-agent-dev took OMNI_HOME from the operator env store
(/data/omninode/omni_home, the lanes' tree), so RT-1's clean checkout
detached and clean -ffdx'd omnimarket, omnibase_core and omnibase_compat
there on every dev job, and the clone guard refused every in-process
delegate on the host. The unit now declares OMNI_HOME=/data/omninode,
the root reconcile-host converges and locks under (OMN-20154), and
protects the name from the env store.
@jonahgabriel
jonahgabriel marked this pull request as ready for review October 1, 2026 05:59
@jonahgabriel
jonahgabriel enabled auto-merge October 1, 2026 05:59
jonahgabriel pushed a commit to OmniNode-ai/onex_change_control that referenced this pull request Oct 1, 2026
…12072)

* evidence(OMN-20255): author OCC companion for OmniNode-ai/omnibase_infra#4401

OCC companion by node_pr_lifecycle_fix_effect (OMN-13317 F1 / OMN-13990 / OMN-14285). Product PR head b389da4b81517f73c2af79eeac681178a8f247e0.

* evidence(OMN-20255): self-bind OCC#12072 + rebind contract_sha256

---------

Co-authored-by: omnimarket-bot <bot@omninode.ai>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hostile Reviewer — adversarial findings (OMN-17492)

Models succeeded: qwen3-review, gpt-oss-review
Models failed: none
New finding threads: 5
Deduped (already posted on this PR): 0
Below quorum (one model only, reported not threaded): 4
Nit-level findings suppressed: 0

The model is the FINDER, never the gate: merge is gated only by the
deterministic Hostile Review Thread Gate, which blocks while
hostile-reviewer threads are unresolved. Resolve each thread after
addressing (or rejecting, with a reply) its finding.

Below quorum: 4 finding(s) raised by one model only (OMN-18479)

These are reported and NOT dropped, but they get no thread and do not block: a single model's finding no other model reproduced is not evidence enough to stop a merge. Read them; act on them if they are right.

  • [MAJOR] scripts/deploy-agent/tests/unit/test_dev_unit_omni_home_deploy_source_omn20255.py (qwen3-review) — Hardcoded absolute path in unit test couples test to deployment layout | The test file hardcodes _DEPLOY_SOURCE_ROOT = "/data/omninode" and _LANES_TREE = "/data/omninode/omni_home" as expected val
  • [MAJOR] scripts/deploy-agent/tests/unit/test_dev_unit_omni_home_deploy_source_omn20255.py (qwen3-review) — Test regex for Environment= lines is fragile and may misparse | The regex ^Environment="?([A-Za-z_][A-Za-z0-9_]*)=(.*?)"?$ uses a non-greedy match for the value and optional quotes. For lines like `
  • [MAJOR] scripts/deploy-agent/tests/unit/test_dev_unit_omni_home_deploy_source_omn20255.py (qwen3-review) — No test verifies that the env store file does not set OMNI_HOME to the lanes tree | The test suite verifies that the service file declares OMNI_HOME and protects it, and that the launcher sources the
  • [MINOR] scripts/deploy-agent/deploy/deploy-agent-dev.service (qwen3-review) — OMNI_HOME is protected from the env store but the protection is only as strong as the launcher's implementation | The service file adds OMNI_HOME to DEPLOY_AGENT_ENV_PROTECTED, which presumably causes

Findings demoted from threads (anchor rejected)

  • [MINOR] hostile-reviewer (qwen3-review)

    Service file comment block is too long and mixes operational history with configuration | The 10-line comment block in the service file embeds incident history (job IDs, dates, specific detach events) and cross-references other tickets (OMN-20154, RT-1). This is operational context that belongs in a runbook, a commit message, or a design document, not in a systemd unit file. Unit files are parsed by systemd and consumed by operators; a dense narrative comment degrades readability and will become stale as th

    Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).

  • [MINOR] hostile-reviewer (qwen3-review)

    Test for lock file path uses string containment rather than structural parsing | The test test_the_build_lock_and_reconcile_host_lock_share_one_root_name asserts that the lock file path string ".onex-reconcile-host.lock" appears in the lock module and that "$OMNI_HOME/.onex-reconcile-host.lock" appears in the reconcile-host script. This is a string containment check, not a structural verification that both files derive the lock path from the same OMNI_HOME variable. If the lock module hardcodes a diff

    Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).

  • [MINOR] hostile-reviewer (gpt-oss-review)

    Unit test hard‑codes absolute filesystem paths | The new test defines _DEPLOY_SOURCE_ROOT and _LANES_TREE as literal strings "/data/omninode" and "/data/omninode/omni_home". This ties the test to a specific host layout and will cause failures on any environment where those directories do not exist, reducing portability and increasing false negatives. | Evidence: Lines in test_dev_unit_omni_home_deploy_source_omn20255.py: _DEPLOY_SOURCE_ROOT = "/data/omninode" and _LANES_TREE = "/data/omninode/omni_home"

    Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).

  • [MINOR] hostile-reviewer (gpt-oss-review)

    DEPLOY_AGENT_ENV_PROTECTED list contains itself | The protected environment variable list now includes the token DEPLOY_AGENT_ENV_PROTECTED, creating a self‑reference. If the env‑store processing logic expands or validates this list, the self‑reference could lead to unnecessary recursion or misinterpretation of protection semantics. | Evidence: Service file line: Environment="DEPLOY_AGENT_ENV_PROTECTED=PATH OMNI_HOME DEPLOY_AGENT_ENV_FILE DEPLOY_AGENT_PYTHON DEPLOY_AGENT_ENV_PROTECTED ...". | Fix: Remove

    Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).

  • [MINOR] hostile-reviewer (gpt-oss-review)

    Path resolution in test is brittle to repository layout changes | The test computes paths using Path(__file__).resolve().parents[2] and similar constructs. Any modification to the directory hierarchy (e.g., moving the test file) will break these calculations, causing the test to read incorrect files or raise errors unrelated to the functionality under test. | Evidence: Lines defining _DEPLOY_DIR, _LAUNCHER, _LOCK, _RECONCILE_HOST using parents[2] and parents[3] in the test file. | Fix: Derive

    Resolve this thread when addressed — the Hostile Review Thread Gate blocks while hostile-reviewer threads are unresolved (OMN-17492).

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

✅ Hostile Reviewer — REVIEWED

Critical findings: 1
Major findings: 0
Total findings: 3
Models succeeded: qwen3-review,gpt-oss-review
Models unavailable: none

Action required: findings were posted as review threads. Address or reject each one, then resolve its thread — the Hostile Review Thread Gate fails while hostile-reviewer threads are unresolved (OMN-17492).


Semantics (OMN-17492 — the model finds, thread resolution gates)

Surface Meaning Blocks merge?
Review threads Per-finding, posted by the reviewer No (informational)
Hostile Review Thread Gate Deterministic: unresolved hostile-reviewer threads exist Fails until resolved (not yet a required context)
degraded verdict Fewer than 2 models succeeded (infra) Fails this job with a named reason

Powered by omniintelligence.review_pairing.cli_review — multi-model adversarial review: qwen3-review (Qwen3.8-27B), gpt-oss-review (gpt-oss-120b) (OMN-8468/OMN-8524/OMN-17492)

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hostile Reviewer — adversarial findings (OMN-17492)

Models succeeded: qwen3-review, gpt-oss-review
Models failed: none
New finding threads: 0
Deduped (already posted on this PR): 0
Below quorum (one model only, reported not threaded): 3
Nit-level findings suppressed: 0

The model is the FINDER, never the gate: merge is gated only by the
deterministic Hostile Review Thread Gate, which blocks while
hostile-reviewer threads are unresolved. Resolve each thread after
addressing (or rejecting, with a reply) its finding.

Below quorum: 3 finding(s) raised by one model only (OMN-18479)

These are reported and NOT dropped, but they get no thread and do not block: a single model's finding no other model reproduced is not evidence enough to stop a merge. Read them; act on them if they are right.

  • [CRITICAL] scripts/deploy-agent/deploy/deploy-agent-dev.service (gpt-oss-review) — OMNI_HOME path change breaks lock interlock | The added Environment=OMNI_HOME=/data/omninode sets OMNI_HOME to a different root than the previously documented lanes tree (/data/omninode/omni_home). Re
  • [MINOR] scripts/deploy-agent/tests/unit/test_dev_unit_omni_home_deploy_source_omn20255.py (gpt-oss-review) — Hard‑coded absolute paths in new unit test reduce portability | The newly added test file defines _DEPLOY_SOURCE_ROOT and _LANES_TREE as literal strings and constructs file paths relative to the repos
  • [MINOR] scripts/deploy-agent/deploy/deploy-agent-dev.service (gpt-oss-review) — Self‑reference in DEPLOY_AGENT_ENV_PROTECTED list may cause parsing loops | The updated DEPLOY_AGENT_ENV_PROTECTED string now includes the token DEPLOY_AGENT_ENV_PROTECTED itself, in addition to other

@jonahgabriel
jonahgabriel added this pull request to the merge queue Oct 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 1, 2026
@jonahgabriel
jonahgabriel added this pull request to the merge queue Oct 1, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 1, 2026
@jonahgabriel
jonahgabriel added this pull request to the merge queue Oct 1, 2026
Merged via the queue into dev with commit c83f3b5 Oct 1, 2026
143 checks passed
@jonahgabriel
jonahgabriel deleted the jonah/omn-20255-deploy-agent-omni-home branch October 1, 2026 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant