Skip to content

docs: clarify arc-dind tool staging and --mount rw/ro output-path pitfalls - #8457

Merged
lpcox merged 3 commits into
mainfrom
copilot/awf-arc-dind-fix-threat-detect
Sep 11, 2026
Merged

docs: clarify arc-dind tool staging and --mount rw/ro output-path pitfalls#8457
lpcox merged 3 commits into
mainfrom
copilot/awf-arc-dind-fix-threat-detect

Conversation

Copilot AI commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

On runner.topology: arc-dind, AWF only auto-stages the invoking engine binary (Copilot/Claude/Codex) into the daemon-visible filesystem. Any other tool a workflow calls inside the sandbox (e.g. a threat-detection binary) is invisible to the chroot sysroot unless separately staged, so it fails with exit 127. Even once staged, a :ro mount of a parent directory doesn't grant write access to output paths beneath it, and producer/consumer steps can silently disagree on the effective path — both failure modes surfaced in gh-aw#59935. The actual codegen fix belongs in gh-aw; this PR covers the documentation/guidance gap on the AWF side.

  • docs/arc-dind.md

    • New "Staging additional CLI tools" section generalizing the existing Copilot CLI staging pattern to any tool invoked inside AWF, with a worked staging example.
    • New subsection on making output paths writable (specific :rw mount needed even under a :ro parent) and keeping producer/consumer paths consistent.
  • docs/environment.md

    • New "--mount and read-only vs. writable output paths" subsection: a :ro mount of a parent never grants write access to a subpath; add a more specific :rw mount for the tool's output directory.
  • docs/selective-mounting.md

    • Cross-linked the existing read-only-mounts best practice to the new guidance.

Example pattern now documented for tools needing both staged input and writable output:

awf --mount /tmp/gh-aw:/tmp/gh-aw:ro \
    --mount /tmp/gh-aw/my-tool:/tmp/gh-aw/my-tool:rw \
    --allow-domains github.com \
    -- my-tool --output /tmp/gh-aw/my-tool/result.json

…dance

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix threat-detect not staged and output path issues docs: clarify arc-dind tool staging and --mount rw/ro output-path pitfalls Sep 11, 2026
Copilot AI requested a review from lpcox September 11, 2026 15:51
@lpcox
lpcox marked this pull request as ready for review September 11, 2026 16:27
Copilot AI balanced review requested due to automatic review settings September 11, 2026 16:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The staging guidance contains inaccurate claims and examples that fail without additional mounts or directories.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Clarifies ARC/DinD staging and writable-output mount requirements.

Changes:

  • Documents staging additional CLI tools.
  • Explains nested read-write mounts and consistent output paths.
  • Cross-links selective-mounting guidance.
File summaries
File Description
docs/arc-dind.md Adds tool staging and output-path guidance.
docs/environment.md Documents read-only versus writable mounts.
docs/selective-mounting.md Links to the expanded guidance.
Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 5
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread docs/arc-dind.md Outdated
Comment on lines +88 to +90
AWF only auto-stages the **invoking CLI binary** (`copilot`, `claude`, `codex`,
etc. — see `dind.stageEngineBinary` and "What AWF handles automatically"
below) into the daemon-visible filesystem. It does **not** discover or stage
Comment thread docs/arc-dind.md Outdated
Comment on lines +106 to +110
**Fix:** copy any additional tool binary your workflow invokes inside AWF to
a path under the same daemon-visible shared directory used for other ARC/DinD
staging (e.g. `${RUNNER_TEMP}/gh-aw` when `--docker-host-path-prefix` points
there), *before* the AWF-wrapped step runs, and pass that staged path (not the
runner-only install path) as the command to run:
Comment thread docs/arc-dind.md
Comment on lines +119 to +124
- name: Run my-tool under AWF
run: |
sudo awf --docker-host-path-prefix /host \
--allow-domains api.example.com \
-- "${RUNNER_TEMP}/gh-aw/bin/my-tool" --output /tmp/gh-aw/my-tool/result.json
```
Comment thread docs/arc-dind.md Outdated
Comment on lines +143 to +144
--mount /tmp/gh-aw:/tmp/gh-aw:ro \
--mount /tmp/gh-aw/my-tool:/tmp/gh-aw/my-tool:rw
Comment thread docs/environment.md Outdated
the writable subpath explicitly as `:rw` in addition to the `:ro` parent:

```bash
awf --mount /tmp/gh-aw:/tmp/gh-aw:ro \
@lpcox

lpcox commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

@copilot address the review feedback

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>

Copilot AI commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

@copilot address the review feedback

Addressed the review feedback in 9d0c797:

  • Auto-staging conditions are now stated precisely: first-command staging runs only when --docker-host-path-prefix is a /tmp-rooted shared path (shouldUseDockerHostStaging()), a /host prefix does not trigger it, and dind.stageEngineBinary is described as explicit config rather than discovery.
  • The command -v recipe is scoped to self-contained executables; wrappers, setup-action SDK tools and package-managed linters are directed to stage/mount the whole install tree plus runtime, or use chroot.binariesSourcePath (mounted at /tmp/awf-runner-bin and prepended to PATH).
  • The worked example now stages under the canonical /tmp/gh-aw tree and explicitly mounts it (noting that --docker-host-path-prefix only rewrites bind-mount sources), so the invocation resolves instead of failing with 127.
  • Producer/consumer guidance now uses the same canonical /tmp/gh-aw path in the staging step and the mounts, removing the ${RUNNER_TEMP} vs /tmp/gh-aw mismatch.
  • Both --mount examples are preceded by mkdir -p, since AWF rejects non-existent host paths with Host path does not exist.

@github-actions

Copy link
Copy Markdown
Contributor

✅ Copilot review passed with no inline comments.

@copilot Add the ready-for-aw label to this PR to trigger agentic CI smoke tests.

@github-actions

Copy link
Copy Markdown
Contributor

Documentation Preview

Documentation has been built for this PR.

Download preview artifact

To view locally:

  1. Download the docs-preview-pr-8457 artifact from the workflow run
  2. Unzip and open index.html in your browser

Built from commit 8313abb

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Smoke Gemini reports failed. Facets need polishing...

💎 Faceted by Smoke Gemini

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK AOAI (api-key) reports failed. AOAI BYOK (api-key) mode investigation needed...

🔑 BYOK (AOAI api-key) report filed by Smoke Copilot BYOK AOAI (api-key)

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

📰 DEVELOPING STORY: Smoke Docker Sbx reports failed. Our correspondents are investigating the incident...

📰 BREAKING: Report filed by Smoke Docker Sbx

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Smoke Claude passed

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by Smoke Claude for #8457

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

🔌 Smoke Services — All services reachable! ✅

🔌 Service connectivity validated by Smoke Services

@github-actions

Copy link
Copy Markdown
Contributor

🚀 Security Guard has started processing this pull request

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • example.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Smoke Copilot BYOK AOAI (Entra) reports failed. AOAI BYOK (Entra) mode investigation needed...

🪪 BYOK (AOAI Entra) report filed by Smoke Copilot BYOK AOAI (Entra)

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded.

Tested by Smoke Chroot

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • msfeed25.pkgs.visualstudio.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "msfeed25.pkgs.visualstudio.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Build Test Suite completed successfully!

Generated by Build Test Suite for #8457

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Claude Engine Validation

Check Result
API status ✅ PASS
gh check ✅ PASS
File status ✅ PASS

Overall result: PASS

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by Smoke Claude for #8457 · claude · haiku45 · 55.9 AIC · ⊞ 4.5K ·
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test Results

  • Redis PING: ✅ PONG
  • PostgreSQL pg_isready: ✅ accepting connections
  • PostgreSQL SELECT 1: ✅ 1

Overall: PASS

🔌 Service connectivity validated by Smoke Services
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot BYOK (Direct)

GitHub MCP Testing — PR #8451, #8442 verified
GitHub.com Connectivity — HTTP 200
File Write/Read — Test file confirmed
BYOK Inference — Direct BYOK mode active (api-proxy → api.githubcopilot.com)

Status: PASS

Running in direct BYOK mode via api-proxy sidecar with real COPILOT_PROVIDER_API_KEY held server-side.

🔑 BYOK report filed by Smoke Copilot BYOK
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

EGRESS_RESULT allow=pass deny=pass

✅ Allowed domain (github.com) reachable: allowed=200
✅ Blocked domain (example.com) denied: CONNECT tunnel failed 403

Overall: PASS

@lpcox

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • example.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "example.com"

See Network Configuration for more information.

🛡️ Egress verdict from Smoke Copilot Network Isolation
Add label ready-for-aw to run again

@github-actions github-actions Bot added the smoke-copilot-network-isolation Copilot network-isolation egress smoke test label Sep 11, 2026
@lpcox
lpcox deployed to aoai-model September 11, 2026 19:11 — with GitHub Actions Active
@lpcox
lpcox deployed to aoai-model September 11, 2026 19:11 — with GitHub Actions Active
@github-actions

Copy link
Copy Markdown
Contributor

Smoke Test: Copilot Engine @lpcox

Overall: PASS

📰 BREAKING: Report filed by Smoke Copilot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Chroot Version Comparison

Runtime Host Version Chroot Version Match?
Python Python 3.12.14 Python 3.12.14 ✅ YES
Node.js v24.20.0 v2.98.0 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

Overall: FAILED — Node.js version inside the chroot (v2.98.0) does not match the host (v24.20.0). smoke-chroot label not added since not all tests passed.

Tested by Smoke Chroot
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color passed ✅ PASS
Go env passed ✅ PASS
Go uuid passed ✅ PASS
Java gson N/A ❌ FAIL
Java caffeine N/A ❌ FAIL
Node.js clsx passed ✅ PASS
Node.js execa passed ✅ PASS
Node.js p-limit passed ✅ PASS
Rust fd passed ✅ PASS
Rust zoxide passed ✅ PASS

Overall: 7/8 ecosystems passed — FAIL

❌ Java Failure Details

Both gson and caffeine failed at the mvn compile step with:

[ERROR] Could not create local repository at /home/runner/.m2/repository -> [Help 1]
[ERROR] LocalRepositoryNotAccessibleException

Root cause: ~/.m2 is pre-owned by root:root (mode drwxr-xr-x) on this runner, so the runner user cannot create ~/.m2/repository to store downloaded artifacts. This is a pre-existing host/runner permission issue, unrelated to the firewall's proxy configuration itself — the Maven proxy settings in settings.xml were applied correctly and JAVA_TOOL_OPTIONS proxy env vars were picked up as expected.

Generated by Build Test Suite for #8457 · copilot · auto · 29.3 AIC · ⊞ 10.6K ·
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

Merged PRs: chore(deps): bump Astro docs integrations; [docs] auth: docs: correct /models GET auto API-version claim in auth-matrix
PR details: chore(deps): bump eslint and typescript-eslint; docs: clarify arc-dind tool staging and --mount rw/ro output-path pitfalls
Playwright title check: ❌ (playwright not installed)
File write/readback: ✅
Discussion comment: ❌ (discussions:write unavailable for #8101)
Build (npm ci && npm run build): ✅
Overall: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • msfeed25.pkgs.visualstudio.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "msfeed25.pkgs.visualstudio.com"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex
Add label ready-for-aw to run again

@github-actions

Copy link
Copy Markdown
Contributor

📡 OTel Tracing Smoke Test Results

# Scenario Result
1 Module Loading otel.js loaded; isEnabled: true; exports startRequestSpan, setTokenAttributes, setBudgetAttributes, endSpan, endSpanError, shutdown, isEnabled, ...
2 Test Suite ✅ 3 suites, 68/68 tests passed (0 failed)
3 Env Var Forwarding GITHUB_AW_OTEL_TRACE_ID/GITHUB_AW_OTEL_PARENT_SPAN_ID in env-passthrough.ts; GH_AW_OTLP_ENDPOINTS, OTEL_EXPORTER_OTLP_ENDPOINT, trace/span IDs in api-proxy-env-config.ts
4 Token Tracker Integration onUsage callback present in token-tracker-http.js
5 OTEL Diagnostics ⚪ No otel.jsonl span file found in this run (no live LLM traffic exercised through api-proxy in this sandbox — expected, not a regression)

Overall: ✅ Pass. All code-level checks and the full OTEL unit test suite (68 tests) succeeded. Scenario 5 had no exported spans because this run didn't route API traffic through the api-proxy sidecar, which is expected behavior, not a failure.

📡 OTel tracing validated by Smoke OTel Tracing
Add label ready-for-aw to run again

@lpcox
lpcox merged commit d020050 into main Sep 11, 2026
167 of 173 checks passed
@lpcox
lpcox deleted the copilot/awf-arc-dind-fix-threat-detect branch September 11, 2026 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[awf] arc-dind: threat-detect not staged, output path read-only and unread

3 participants