feat(0025): subagent process isolation via PreToolUse hooks

[maruiz93]

Summary

• Adds experiment 0025: intercepts Agent() calls via a PreToolUse hook to run subagent work in isolated claude CLI processes within the sandbox
• Echo-through pattern: hook spawns a separate claude -p process, captures its output, replaces the built-in subagent's prompt with an echo instruction
• Hook config injected via host_files to CLAUDE_CONFIG_DIR — target repo stays clean (can be empty)
• Policy referenced from fullsend-ai/agents via SHA-pinned URL instead of local copy

Key discoveries

• --dangerously-skip-permissions skips settings.local.json — must use settings.json for hooks in sandbox
• allowed_remote_resources only needed in harness YAML; config.yaml gets defaults merged automatically
• Process group kill (start_new_session + os.killpg) prevents orphaned grandchildren on timeout

Test plan

• fullsend run with empty target repo — 3 transcripts, validation passed
• Hook intercept confirmed via echo-through prompt in subagent transcript
• Process isolation confirmed via distinct session IDs across transcripts
• Reviewer: verify HOW_TO.md steps reproduce successfully

🤖 Generated with Claude Code

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 5:22 PM UTC · Completed 5:42 PM UTC
Commit: 1032337 · View workflow run →

[qodo-code-review]

PR Summary by Qodo

Add experiment 0025: isolate subagent work via PreToolUse Agent hook

✨ Enhancement 📝 Documentation ⚙️ Configuration changes 🕐 20-40 Minutes

AI Description

• Add experiment 0025 demonstrating subagent process isolation using a PreToolUse hook.
• Run intercepted Agent() prompts in a separate claude -p process and echo results back.
• Document setup, expected transcripts, and validate required output artifacts.

Diagram

graph TD
  P["Parent agent"] --> A["Agent() call"] --> H["PreToolUse hook"] --> C["claude -p process"]
  H --> E["Echo subagent"] --> P
  S[("settings.json (hooks)")] --> H

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Separate evaluation as a second fullsend run (no Agent hook)

• ➕ True process isolation without relying on Claude Code hook semantics
• ➕ Avoids the unavoidable third “echo” session/inference
• ➖ More orchestration complexity (two runs, artifact passing)
• ➖ Less representative of real Agent()-based subagent workflows

2. Block built-in subagent via tool denial and inject result

• ➕ Could eliminate the extra echo-through subagent session if reliable
• ➖ Denial behavior is documented as unreliable in practice
• ➖ Higher risk of brittle behavior across model/tooling updates

3. Implement isolation natively in fullsend (first-class subagent runner)

• ➕ Eliminates hook workarounds and reduces operational complexity
• ➕ Better observability/control over subprocess lifecycle and resources
• ➖ Requires product code changes beyond an experiment
• ➖ Longer lead time; not suitable for quick validation

Recommendation: For an experiment intended to validate feasibility quickly, the PR’s PreToolUse echo-through approach is appropriate and well-contained (no target repo pollution via host_files). If this graduates beyond an experiment, prefer a native fullsend-level implementation to remove the third echo session and reduce dependence on Claude Code hook edge cases.

Files changed (11) +465 / -0

Enhancement (2) +124 / -0

judge-parent.mdAdd parent agent that triggers an intercepted Agent() evaluation +32/-0

Add parent agent that triggers an intercepted Agent() evaluation

• Defines the experiment agent workflow: write topic content to FULLSEND_OUTPUT_DIR, invoke 'Agent()' for evaluation, and persist evaluation/summary artifacts. The prompt explicitly includes absolute paths to support sandbox execution and validation.

0025-subagent-process-isolation/.fullsend/agents/judge-parent.md

pretooluse.pyImplement Agent() interception by spawning isolated claude CLI processes +92/-0

Implement Agent() interception by spawning isolated claude CLI processes

• Implements the PreToolUse command hook that detects 'tool_name == Agent', spawns 'claude -p' with the original prompt, and rewrites the Agent prompt to echo the isolated result back verbatim. Includes recursion prevention via 'FULLSEND_HOOK_SPAWNED' and process-group kill on timeout to avoid orphaned subprocesses.

0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py

Tests (1) +21 / -0

validate-output.shValidate required output artifacts exist after the run +21/-0

Validate required output artifacts exist after the run

• Adds a validation script that checks for 'topic.md', 'evaluation.md', and 'summary.md' under the output directory and fails the run if missing. Reports file sizes to aid debugging.

0025-subagent-process-isolation/.fullsend/scripts/validate-output.sh

Documentation (3) +269 / -0

HOW_TO.mdDocument end-to-end reproduction steps and verification checks +137/-0

Document end-to-end reproduction steps and verification checks

• Provides a step-by-step guide to run the experiment, including prerequisites, environment variables, and fullsend invocation. Documents how to verify hook interception and process isolation via transcript inspection and session ID comparison.

0025-subagent-process-isolation/HOW_TO.md

README.mdAdd experiment write-up: hypothesis, approach, and key findings +131/-0

Add experiment write-up: hypothesis, approach, and key findings

• Explains the rationale for PreToolUse interception and the echo-through pattern, including why three sessions occur. Captures operational discoveries (settings.json vs .local, env exports, policy format) and summarizes results/costs.

0025-subagent-process-isolation/README.md

README.mdRegister experiment 0025 in the root experiments index +1/-0

Register experiment 0025 in the root experiments index

• Adds experiment 0025 to the top-level README table with status Active and a link to its directory. Keeps the experiment catalog up to date.

README.md

Other (5) +51 / -0

config.yamlRegister the experiment harness as a runnable agent source +2/-0

Register the experiment harness as a runnable agent source

• Adds the minimal '.fullsend/config.yaml' needed to load the harness definition for 'judge-parent'. This makes the experiment runnable via 'fullsend run judge-parent'.

0025-subagent-process-isolation/.fullsend/config.yaml

claude-hooks.jsonConfigure a PreToolUse hook to match and intercept Agent tool calls +16/-0

Configure a PreToolUse hook to match and intercept Agent tool calls

• Adds Claude Code hook configuration wiring 'PreToolUse' events for the 'Agent' tool to a command hook. This file is intended to be injected as '/sandbox/claude-config/settings.json'.

0025-subagent-process-isolation/.fullsend/config/claude-hooks.json

gcp-vertex.envProvide Vertex AI environment exports for sandbox runs +5/-0

Provide Vertex AI environment exports for sandbox runs

• Adds an env file that exports the variables needed to run Claude Code via Vertex ('CLAUDE_CODE_USE_VERTEX=1') and pass through GCP configuration/credentials. Uses 'export' so child processes (including 'claude') inherit variables.

0025-subagent-process-isolation/.fullsend/env/gcp-vertex.env

judge-parent.yamlAdd harness wiring: policy fetch, host_files injection, and validation +26/-0

Add harness wiring: policy fetch, host_files injection, and validation

• Defines the experiment harness (image, model, role) and references a SHA-pinned remote policy URL. Injects env, credentials, the PreToolUse hook script, and hook settings.json via 'host_files', and runs a single-iteration validation script.

0025-subagent-process-isolation/.fullsend/harness/judge-parent.yaml

.gitignoreIgnore experiment outputs and fullsend cache +2/-0

Ignore experiment outputs and fullsend cache

• Ignores 'results/' artifacts and '.fullsend/.fullsend-cache/' to keep runs from polluting git status. Scoped to the experiment directory.

0025-subagent-process-isolation/.gitignore

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📜 Skill insights (0)

1. Validator uses wrong env ✓ Resolved 🐞 Bug ≡ Correctness

Description

validate-output.sh derives the output path from TARGET_REPO_DIR (falling back to ./output) instead
of the documented FULLSEND_OUTPUT_DIR, so validation can fail even when the agent writes all files
to FULLSEND_OUTPUT_DIR as instructed. This makes experiment runs brittle against the runner’s env
contract and working directory.

Code

0025-subagent-process-isolation/.fullsend/scripts/validate-output.sh[4]

+output_dir="${TARGET_REPO_DIR:-.}/output"

Relevance

●●● Strong

Env-var contract mismatch is a concrete correctness bug affecting runs; likely fixed to use
documented vars.

PR-#10

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The new validator checks ${TARGET_REPO_DIR:-.}/output, but the repo’s sandbox env var
documentation defines FULLSEND_OUTPUT_DIR and FULLSEND_TARGET_REPO_DIR as the canonical
variables; this mismatch makes the validator dependent on an undocumented variable and/or a specific
working directory layout.

0025-subagent-process-isolation/.fullsend/scripts/validate-output.sh[1-5]
0018-runner-hello-world/README.md[84-90]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The validator script computes `output_dir` from `TARGET_REPO_DIR`, but the sandbox contract documented in this repo exposes `FULLSEND_OUTPUT_DIR` (and `FULLSEND_TARGET_REPO_DIR`) instead. If `TARGET_REPO_DIR` is not set, the script validates `./output`, which may not be where the agent wrote files.

## Issue Context
- The agent is instructed to write outputs under `$FULLSEND_OUTPUT_DIR`.
- The validator should therefore validate the same directory (or a clearly documented extracted/mirrored location).

## Fix Focus Areas
- 0025-subagent-process-isolation/.fullsend/scripts/validate-output.sh[4-4]

## Implementation notes
- Prefer `output_dir="${FULLSEND_OUTPUT_DIR:-output}"` (or another fallback that matches the runner’s documented contract).
- If you need a host-side extracted directory, document and use the exact env var the runner actually provides (but avoid inventing a new `TARGET_REPO_DIR` contract just for this one experiment).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. HOW_TO has extra sections ✓ Resolved 📜 Skill insight ✧ Quality

Description

The HOW_TO includes additional sections beyond the required ordered set (Purpose, Requirements,
Steps, Expected Output). This violates the requirement that the document contain exactly those
four sections in order.

Code

0025-subagent-process-isolation/HOW_TO.md[R104-106]

+## File Layout
+
+```

Relevance

●●● Strong

Removing extra HOW_TO sections is a straightforward compliance change; convention enforcement has
been accepted before.

PR-#36

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 1062129 requires the HOW_TO to contain exactly four sections in order. The file
includes ## File Layout and ## Troubleshooting after ## Expected Output, so it is not limited
to the required four sections.

0025-subagent-process-isolation/HOW_TO.md[3-9]
0025-subagent-process-isolation/HOW_TO.md[10-19]
0025-subagent-process-isolation/HOW_TO.md[28-33]
0025-subagent-process-isolation/HOW_TO.md[90-103]
0025-subagent-process-isolation/HOW_TO.md[104-137]
Skill: writing-how-to

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The HOW_TO document contains additional sections after `Expected Output`, but the compliance rule requires exactly four sections in order: Purpose, Requirements, Steps, Expected Output.

## Issue Context
`## File Layout` and `## Troubleshooting` appear after `## Expected Output`.

## Fix Focus Areas
- 0025-subagent-process-isolation/HOW_TO.md[1-137]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. Hardcoded GOOGLE_APPLICATION_CREDENTIALS path ✗ Dismissed 📜 Skill insight ⛨ Security

Description

GOOGLE_APPLICATION_CREDENTIALS is given a concrete/example path in docs and is hardcoded in an env
file, effectively providing a default for a secret-bearing environment variable. This can lead to
accidental credential/path leakage and violates the requirement to avoid defaults for secret env
vars.

Code

0025-subagent-process-isolation/.fullsend/env/gcp-vertex.env[4]

+export GOOGLE_APPLICATION_CREDENTIALS=/tmp/.gcp-credentials.json

Relevance

●●● Strong

Team has accepted security-hardening changes around secrets/credentials handling; hardcoded secret
defaults likely removed.

PR-#10

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 1062127 forbids providing default values for secret environment variables. The env
file sets GOOGLE_APPLICATION_CREDENTIALS to a fixed path, and the HOW_TO also provides a concrete
example/export value for it.

0025-subagent-process-isolation/.fullsend/env/gcp-vertex.env[1-5]
0025-subagent-process-isolation/HOW_TO.md[20-27]
Skill: writing-how-to

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`GOOGLE_APPLICATION_CREDENTIALS` is being assigned/provided with a default/example path, which violates the requirement to never hardcode default values for secret environment variables.

## Issue Context
This PR introduces both an env file and a HOW_TO that specify a concrete credentials path (including an explicit export command). The compliance rule requires descriptions/placeholders only, not defaults.

## Fix Focus Areas
- 0025-subagent-process-isolation/.fullsend/env/gcp-vertex.env[1-5]
- 0025-subagent-process-isolation/HOW_TO.md[20-38]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

4. Steps include explanatory context ✓ Resolved 📜 Skill insight ⚙ Maintainability

Description

Several Steps include narrative/explanatory context (e.g., parentheticals and rationale) rather than
only short action instructions or commands. This violates the rule that explanatory context must not
appear in HOW_TO steps.

Code

0025-subagent-process-isolation/HOW_TO.md[R45-47]

+4. Create an empty target repo (the hook and settings live in the
+   agent files, not the target repo):
+   ```bash

Relevance

●●● Strong

Mechanical doc cleanup (remove narrative from steps) is low-risk and consistent with enforced
conventions.

PR-#36

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 1062133 requires steps to contain only commands or short action instructions. Step
4 includes explanatory rationale text about why the target repo is empty, which is narrative
context.

0025-subagent-process-isolation/HOW_TO.md[45-50]
Skill: writing-how-to

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The HOW_TO `Steps` section includes explanatory context inside steps, which should be moved to the README (or elsewhere) so each step remains only an action/command.

## Issue Context
Example: step 4 includes rationale about why the target repo is empty.

## Fix Focus Areas
- 0025-subagent-process-isolation/HOW_TO.md[28-60]
- 0025-subagent-process-isolation/HOW_TO.md[66-88]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

5. Hook JSON parse crash ✓ Resolved 🐞 Bug ☼ Reliability

Description

pretooluse.py calls json.load(sys.stdin) without handling empty/malformed stdin, so the hook can
terminate with an uncaught exception instead of deterministically allowing (or blocking) the tool
call. This is inconsistent with other hooks in the repo that guard JSON parsing to ensure controlled
behavior on bad input.

Code

0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py[R22-23]

+    hook_input = json.load(sys.stdin)
+    tool_name = hook_input.get("tool_name", "")

Relevance

●●● Strong

Defensive handling to avoid unexpected hook crashes is a straightforward reliability fix; likely
accepted.

PR-#10

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The new hook performs an unguarded JSON read from stdin, while existing PreToolUse hooks in this
repo explicitly handle empty/malformed input to avoid unexpected termination and to enforce a
consistent fail-open/fail-closed policy.

0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py[22-23]
0009-hermes-security-patterns/hooks/ssrf_pretool.py[141-150]
0017-reasoning-monitor/monitor/tool_allowlist.py[105-120]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`pretooluse.py` uses `json.load(sys.stdin)` without a `try/except`. If stdin is empty, truncated, or malformed, the hook process will crash before it can emit a response, making behavior runner-dependent and potentially breaking the Agent call path.

## Issue Context
Other hooks in this repo implement explicit parse error handling and choose a clear fail-open or fail-closed policy.

## Fix Focus Areas
- 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py[22-23]

## Implementation notes
- Wrap JSON parsing in `try/except (json.JSONDecodeError, EOFError, Exception)`.
- Decide policy:
 - Fail-open: print a concise stderr message and `sys.exit(0)` with no stdout.
 - Or fail-closed: emit a structured JSON block response (if that matches the hook protocol for this hook type).
- Keep behavior deterministic on parse failures (avoid uncaught exceptions).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

6. Requirements missing install link ✓ Resolved 📜 Skill insight ✧ Quality

Description

The Requirements table includes an entry whose Link column is not an installation documentation link
(it is build instructions text). This violates the requirement that every tool in the Requirements
table include an installation link.

Code

0025-subagent-process-isolation/HOW_TO.md[R12-15]

+| Requirement | Link |
+|-------------|------|
+| fullsend CLI v0.33.0+ | Built from source: `go build -o ~/.local/bin/fullsend ./cmd/fullsend/` |
+| OpenShell 0.0.83+ | https://github.com/NVIDIA/OpenShell |

Relevance

●●● Strong

Documentation/compliance formatting fix (proper install link) is mechanical and likely required by
linting.

PR-#36

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 1062130 requires each Requirements table row to include a non-empty installation
link. The fullsend CLI v0.33.0+ row's Link cell is not a link to installation documentation.

0025-subagent-process-isolation/HOW_TO.md[12-18]
Skill: writing-how-to

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A Requirements table entry does not provide an installation documentation link in the Link column.

## Issue Context
The `fullsend CLI v0.33.0+` row uses build instructions text rather than a link to installation docs.

## Fix Focus Areas
- 0025-subagent-process-isolation/HOW_TO.md[12-18]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

7. Purpose is multiple sentences ✓ Resolved 📜 Skill insight ✧ Quality

Description

The Purpose section contains more than one sentence. This violates the requirement that Purpose be
exactly one sentence.

Code

0025-subagent-process-isolation/HOW_TO.md[R5-8]

+Run a fullsend agent that spawns a subagent via `Agent()`, where
+the subagent's work is transparently intercepted by a PreToolUse
+hook and executed as a separate `claude` CLI process. Verify the
+isolation through transcript analysis.

Relevance

●●● Strong

Repo appears to enforce doc/lint conventions; single-sentence Purpose is a low-risk compliance edit.

PR-#36

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

PR Compliance ID 1062132 requires the Purpose section to be exactly one sentence. The Purpose text
spans multiple sentences describing both running the experiment and verifying isolation.

0025-subagent-process-isolation/HOW_TO.md[3-9]
Skill: writing-how-to

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The HOW_TO `Purpose` section contains multiple sentences, but it must be exactly one sentence.

## Issue Context
Purpose currently describes the run and then separately instructs to verify isolation.

## Fix Focus Areas
- 0025-subagent-process-isolation/HOW_TO.md[3-9]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Context used

✅ Compliance rules (platform): 19 rules

✅ Skills: writing-how-to

_{To customize comments, go to the Qodo configuration screen, or learn more in the docs.}

[fullsend-ai-review]

Review

Findings

Low

• [edge-case] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:93 — If the hook framework kills the hook process at the 300s timeout while communicate is blocking (before the 270s inner timeout fires), the spawned process group (start_new_session=True) will be orphaned with no cleanup. The 30-second buffer mitigates this under normal conditions. Within the sandbox this is low-impact since all processes are terminated on container exit.

• [command-injection] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:88 — The spawned claude subprocess runs with --dangerously-skip-permissions, granting unrestricted tool execution. The original_prompt is passed directly as stdin to this unrestricted process. The sandbox container boundary provides mitigation, and the design is intentional and documented in README.md.

• [prompt-injection] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:127 — The output from the spawned subprocess (agent_output) is interpolated directly into the parent agent's prompt via f-string. The echo-through instruction provides no reliable defense against prompt injection, though the risk is bounded by the sandbox.

• [secret-exposure] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:81 — The spawned subprocess inherits the full parent environment via os.environ.copy(), including all credentials. Both parent and child run in the same sandbox, limiting additional exposure.

• [fail-open] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:49 — The sandbox directory check and recursion guard silently return without JSON output, causing the Agent tool to proceed without isolation. This is intentional for local development and recursion prevention.

• [secret-exposure] 0025-subagent-process-isolation/.fullsend/harness/judge-parent.yaml:15 — GCP credentials file is copied into the sandbox and accessible to the unrestricted agent. For production use, prefer workload identity federation or short-lived tokens.

• [information-disclosure] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:121 — When the spawned process exits with a non-zero return code and produces no stdout, stderr content (truncated to 500 chars) is included in agent_output and interpolated into the parent agent's prompt. Within the sandbox this is low risk.

• [naming-convention] 0025-subagent-process-isolation/.fullsend/scripts/post-emit-cost.py:55 — Inside build_otlp_payload, local variables start_ns and end_ns hold string values but the _ns suffix suggests nanosecond integers. The OTLP field naming uses the same convention, making this a minor style observation.

Previous run

Review

Findings

Medium

• [race-condition] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:96 — os.killpg(proc.pid, signal.SIGKILL) can raise ProcessLookupError if the spawned process exits between the TimeoutExpired exception and the killpg call. This uncaught exception crashes the hook without producing JSON output on stdout, causing the parent agent's Agent tool call to fail in an uncontrolled way.
  Remediation: Wrap the os.killpg call in a try/except ProcessLookupError: pass block.

Low

• [command-injection] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:85 — The spawned claude subprocess runs with --dangerously-skip-permissions, granting unrestricted tool execution. The original_prompt is passed directly as stdin to this unrestricted process. The sandbox container boundary provides mitigation, and the design is intentional and documented in README.md.

• [env-variable-mismatch] 0025-subagent-process-isolation/.fullsend/scripts/validate-output.sh:4 — The validation script resolves the output directory via TARGET_REPO_DIR (${TARGET_REPO_DIR:-.}/output), while the agent writes files to FULLSEND_OUTPUT_DIR. If these don't resolve to the same path at runtime, validation checks the wrong directory.

• [edge-case] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:94 — If the hook framework kills the hook process at the 300s timeout while communicate is blocking, the spawned process group will be orphaned. Within the sandbox this is low-impact since the container terminates all processes on exit.

• [prompt-injection] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:119 — The output from the spawned subprocess (agent_output) is interpolated directly into the parent agent's prompt via f-string. The echo-through instruction provides no reliable defense against prompt injection, though the risk is bounded by the sandbox.

• [secret-exposure] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:78 — The spawned subprocess inherits the full parent environment via os.environ.copy(), including all credentials. Both parent and child run in the same sandbox, limiting additional exposure.

• [fail-open] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:49 — The sandbox directory check and recursion guard silently return without JSON output, causing the Agent tool to proceed without isolation. This is intentional for local development and recursion prevention.

• [secret-exposure] 0025-subagent-process-isolation/.fullsend/harness/judge-parent.yaml:15 — GCP credentials file is copied into the sandbox and accessible to the unrestricted agent. For production use, prefer workload identity federation or short-lived tokens.

• [error-handling-idiom] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:71 — claude_bin resolution prioritizes shutil.which('claude') over os.environ.get('CLAUDE_BIN'), inverting the typical convention where an explicit env var override takes priority over PATH.

• [error-handling-idiom] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:114 — Exception handling catches only FileNotFoundError; other subprocess exceptions (PermissionError, OSError) would propagate unhandled and crash the hook without JSON output.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run (2)

Review

Findings

Medium

• [race-condition] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:91 — os.killpg(proc.pid, signal.SIGKILL) can raise ProcessLookupError if the spawned process exits between the TimeoutExpired exception and the killpg call. This uncaught exception crashes the hook without producing JSON output on stdout, causing the parent agent's Agent tool call to fail in an uncontrolled way.
  Remediation: Wrap the os.killpg call in a try/except (ProcessLookupError, OSError): pass block.

• [command-injection] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:80 — The spawned claude subprocess runs with --dangerously-skip-permissions, granting unrestricted tool execution within the sandbox. While mitigated by the sandbox container boundary, the combination of arbitrary prompt input and unrestricted permissions deserves explicit documentation as an accepted security trade-off.

Low

• [env-variable-mismatch] 0025-subagent-process-isolation/.fullsend/scripts/validate-output.sh:4 — The validation script resolves the output directory via TARGET_REPO_DIR (${TARGET_REPO_DIR:-.}/output), while the agent writes files to FULLSEND_OUTPUT_DIR. If these don't resolve to the same path at runtime, validation checks the wrong directory.

• [prompt-injection] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:119 — The output from the spawned subprocess (agent_output) is interpolated directly into the parent agent's prompt via f-string without sanitization. The echo-through instruction provides no reliable defense against prompt injection, though the risk is bounded by the sandbox.

• [secret-exposure] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:75 — The spawned subprocess inherits the full parent environment via os.environ.copy(), including all credentials. Both parent and child run in the same sandbox, limiting additional exposure, but constructing a minimal environment would be more defensive.

• [docstring-format] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:2 — Module docstring omits the hook's JSON protocol specification (stdin format, stdout format). Installation instructions are N/A since the harness handles injection, but protocol documentation would improve maintainability.

• [error-handling] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:68 — claude_bin resolution prioritizes shutil.which('claude') over os.environ.get('CLAUDE_BIN'), inverting the typical convention where an explicit env var override takes priority over PATH.

• [edge-case] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:89 — If the hook framework kills the process at the 300s timeout while communicate is blocking, the spawned process group will be orphaned. Within the sandbox this is low-impact since the container terminates all processes on exit.

• [fail-open] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:46 — The sandbox directory check and recursion guard silently return without JSON output, causing the Agent tool to proceed without isolation. This is intentional for local development and recursion prevention.

• [secret-exposure] 0025-subagent-process-isolation/.fullsend/harness/judge-parent.yaml:16 — GCP credentials file is copied into the sandbox and accessible to the unrestricted agent. For production use, prefer workload identity federation or short-lived tokens.

• [error-handling-idiom] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:110 — Exception handling catches only FileNotFoundError; other subprocess exceptions (PermissionError, OSError) would propagate unhandled and crash the hook without JSON output.

---

Next steps:

• /fs-fix — agent addresses review findings automatically
• /fs-fix <your instruction> — agent fixes with your specific guidance
• Push commits directly — review re-runs automatically on push
• /fs-fix-stop — disable automatic fix runs for this PR

Previous run (3)

Review

Findings

Medium

• [command-injection] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:49 — The spawned claude subprocess runs with --dangerously-skip-permissions, granting unrestricted tool execution within the sandbox. While this is by design for the experiment and mitigated by the sandbox container boundary, the reliance on --dangerously-skip-permissions with arbitrary prompt input deserves explicit documentation as an accepted security trade-off.

• [prompt-injection] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:79 — The output from the spawned subprocess (agent_output) is interpolated directly into the parent agent's prompt via f-string without sanitization. If the spawned process returns adversarial content, the parent agent receives it as the subagent's response. The echo-through instruction provides no reliable defense against prompt injection. Consider documenting this as an accepted risk or using structured output.

Low

• [error-handling] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:37 — claude_bin resolution prioritizes shutil.which('claude') over os.environ.get('CLAUDE_BIN'), inverting the typical convention where an explicit env var override takes priority over PATH. Consider: os.environ.get('CLAUDE_BIN') or shutil.which('claude') or 'claude'.

• [edge-case] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:55 — If the hook framework kills the process at the 300s timeout while communicate is blocking, the spawned process group will be orphaned. Within the sandbox this is low-impact, but a try/finally cleanup would be more robust.

• [fail-open] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:14 — The sandbox directory check silently returns without JSON output, causing the Agent tool to proceed unmodified (no isolation). Consider whether this fail-open behavior is intentional for local development or should fail explicitly.

• [fail-open] 0025-subagent-process-isolation/.fullsend/hooks/pretooluse.py:18 — The FULLSEND_HOOK_SPAWNED env var recursion guard is intentional but could theoretically be bypassed by code that sets this var externally. The risk is low since such code would already have execution access.

• [secret-exposure] 0025-subagent-process-isolation/.fullsend/harness/judge-parent.yaml:16 — GCP credentials file is copied into the sandbox and accessible to the unrestricted agent. Consider using workload identity federation or short-lived tokens for production use.

• [missing-authorization] This non-trivial change has no linked issue. While AGENTS.md does not require issue linkage for experiments, linking one improves traceability.

• [cross-repository-dependency] 0025-subagent-process-isolation/.fullsend/harness/judge-parent.yaml — Policy referenced from fullsend-ai/agents via SHA-pinned URL. The SHA pin and integrity hash mitigate supply-chain risk, but vendoring locally would improve self-containment.

---

Labels: PR adds a new experiment (feat commit type)

[maruiz93]

I'm checking how to add the cost of the new spawned agent to mlflow

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 5:55 PM UTC · Completed 6:15 PM UTC
Commit: e69bee7 · View workflow run →

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 6:22 PM UTC · Ended 6:32 PM UTC
Commit: 476473f · View workflow run →

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 6:33 PM UTC · Ended 6:36 PM UTC
Commit: ff0a39e · View workflow run →

[fullsend-ai-review]

🤖 Review · ⚠️ Cancelled · Started 6:36 PM UTC · Ended 6:39 PM UTC
Commit: b78d508 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 6:39 PM UTC · Completed 7:17 PM UTC
Commit: 2f03f1e · View workflow run →

[ralphbean]

Went through 2f03f1e. The echo-through hook and the traceparent-stitched cost span both check out against the README/HOW_TO. Approving.

[maruiz93]

Acknowledged all low-severity findings as accepted trade-offs for this experiment:

• command-injection / --dangerously-skip-permissions: Intentional — sandbox boundary provides isolation, documented in README.
• prompt-injection: Accepted risk bounded by sandbox. The echo-through pattern is a workaround for missing PostToolUse support.
• secret-exposure (env copy): Both processes run in the same sandbox with identical access.
• fail-open: Intentional for local development and recursion prevention.
• edge-case (orphaned process): Container cleanup handles this.
• secret-exposure (GCP credentials): Experiment-only; production would use workload identity federation.

Addressed the medium race-condition (killpg), error-handling (json.load guard, OSError), env-variable-mismatch (validate-output.sh), and CLAUDE_BIN precedence in c122319.

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 10:11 PM UTC · Completed 10:26 PM UTC
Commit: c122319 · View workflow run →

Superseded by updated review

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 7:19 AM UTC · Completed 7:33 AM UTC
Commit: c122319 · View workflow run →

[fullsend-ai-retro]

Retro: PR #48 — feat(0025): subagent process isolation via PreToolUse hooks

What happened

PR #48 was a human-authored experiment by maruiz93 (with Claude Opus 4.6 co-authorship) adding experiment 0025: intercepting Agent() calls via a PreToolUse hook to run subagent work in isolated claude CLI processes. The PR received 4 review cycles from fullsend-ai-review, 7 findings from qodo-code-review, and an architectural approval from ralphbean. Agents repo: fullsend-ai/agents at SHA 5e98b13.

Review quality was strong. The agents collectively identified 5 confirmed bugs (killpg race condition, validate-output.sh env var mismatch, JSON parse crash, narrow exception handling, CLAUDE_BIN precedence) and drove 9 total code/documentation fixes. The human reviewer (ralphbean) verified architectural correctness of the echo-through pattern and traceparent cost stitching — a complementary "does this design make sense" review that neither bot performed.

Main inefficiencies:

1. 67% finding duplication across 4 review cycles — 6 findings repeated verbatim in all 4 reviews, with no acknowledgment of resolved findings.
2. 32 wasted pull_request_review workflow runs — the author submitted 28 individual reply comments (not batched), each triggering a fullsend.yaml shim run that resolved to a no-op at the dispatch level. 21 of these were cancelled.
3. Severity oscillation — command-injection was rated medium in R1/R2 then low in R3/R4 with essentially the same description.
4. Inline comment escalation — the agent posted 7 inline comments on R3 then 14 on R4, despite the author having addressed the medium-severity findings between those commits.

Evidence for existing issues

• #2959 / #5007: 67% duplication rate; 6 findings repeated in all 4 reviews; agent silently dropped resolved findings without acknowledgment.
• #5760: Inline comment count escalated 7→14 between R3→R4 despite fixes being applied; 14 duplicate inline comments posted on the commit that was then approved.
• #4313: Severity oscillated for command-injection (medium→medium→low→low) and prompt-injection (medium→low→low→low) across reviews with near-identical descriptions.
• #3073 / #5265: Agent never acknowledged the killpg race condition fix or env-var-mismatch fix — findings were silently dropped rather than noted as resolved.
• #2599: 32 wasted fullsend.yaml runs from author COMMENTED reviews; a per-PR review budget would cap but not eliminate this class of waste.

Proposals

One proposal filed — see below. The remaining improvement opportunities are well-covered by the existing issues listed above.

Proposals filed

• Filter pull_request_review events in fullsend.yaml shim to changes_requested state (in fullsend-ai/fullsend)