test(browser): retain Agent Task process teardown evidence after failure

CHANGELOG.md

@@ -6,6 +6,7 @@ All notable changes to OriginWeave are documented in this file. The format follo
 
 ### Added
 
+- Failed controlled Agent Task runs now report whether their original browser process ended after shutdown, alongside temporary-profile cleanup; a failed task never becomes a pass merely because cleanup succeeded. If process observation itself fails, termination remains unproven. This covers the original browser process only, not all descendants or arbitrary browser recovery.
 - Controlled pinned-Chromium Agent Task success now binds the ChromeDriver browser PID to its Linux `/proc/<pid>/stat` start-time identity and fails closed unless that exact root process terminates after session/driver shutdown; PID reuse counts only as termination of the original identity, and this does not yet prove termination of every Chromium descendant or process ownership outside the controlled runner.
 - Failed ordinary and forced-close Agent Task browser trials now retain credential-free temporary-profile cleanup evidence after bounded browser errors, and separate aggregate compatibility gates require cleanup proof from every trial rather than filtering unsuccessful trials out; this does not attest adversarial filesystem erasure, process termination, or arbitrary browser recovery.
 - Failed Manifest V3 restart trials now retain credential-free temporary-profile cleanup evidence after bounded browser errors, including reviewed ChromeDriver process-teardown `TimeoutExpired` failures; successful trials record the same cleanup fact, and an aggregate compatibility gate requires teardown proof from every MV3 trial before repeatability acceptance without retaining exception messages or command paths; this does not attest adversarial filesystem erasure, browser-process termination, or cleanup outside the controlled temporary profile.

docs/doctoring.md

@@ -90,6 +90,14 @@ TRINITY uses a compact learned coordinator to select models and assign Thinker,
 
 These results motivate explicit OriginWeave configuration for model routing, workflow stage, decomposition, recursion depth, permitted access, role assignment, and role-specific reasoning effort. They do not justify always using multiple agents. OriginWeave must compare bounded single-model, routed-model, and deeper multi-agent configurations through task-success, safety, variance, token, and compute ablations. No learned coordinator may expand browser capabilities, origins, destinations, approvals, secrets, or deterministic policy.
 
+### Failed Agent Task cleanup evidence
+
+PR #143's implementation at `c1dd380be91c0604b797b6914f8cfef2e96f99b7` retains a bounded browser-failure type after the controlled browser's PID/start-time identity has been captured, shuts down the session and driver, then observes whether that exact root identity ended. The isolated trial separately removes its temporary profile before returning failed-trial evidence. A true or false process-termination result remains distinct from profile cleanup; neither changes a failed task into a successful one, and success-only surface checks cannot override the aggregate pass-count gate.
+
+Process-observation errors leave termination unproven: a read failure escapes the browser-pass observer and the outer trial records that bounded error type with profile cleanup, without a process-termination flag. The current runner reports one failure type, so the original browser failure type is not retained in that fallback record. This limit is explicit; no catch-all, invented successful cleanup, timeout increase or causal-error-chain claim is added. Observing one controlled root does not attest every Chromium descendant, adversarial filesystem erasure or ownership of unrelated host processes.
+
+The release-record regression first failed because this changed failure path had no corresponding Unreleased entry. A controlled behavioral regression now exercises the real browser-pass and trial functions with no browser launch: observed exit, surviving identity and observation error all remain failed trials, driver shutdown is required, and private exception text is absent from returned evidence. These controlled tests are not Linux process or pinned-Chromium runtime evidence; exact-head hosted compatibility remains required.
+
 ## References
 
 Amazon Web Services. (n.d.). *Set up the Amazon EKS Pod Identity Agent*. Retrieved August 6, 2026, from https://docs.aws.amazon.com/eks/latest/userguide/pod-id-agent-setup.html

scripts/ci/run_mv3_compatibility.py

@@ -930,6 +930,7 @@ def _run_agent_task_browser_pass(
     session_id: str | None = None
     browser_process_id: int | None = None
     browser_process_start_time_ticks: int | None = None
+    browser_failure_type: str | None = None
     result: dict[str, Any] | None = None
     driver = subprocess.Popen(
         [str(chromedriver_bin), f"--port={driver_port}", "--allowed-ips=127.0.0.1"],
@@ -1143,6 +1144,10 @@ def _run_agent_task_browser_pass(
             "task_duration_ms": task_duration_ms,
             "duration_ms": round(task_duration_ms),
         }
+    except (OSError, ValueError, RuntimeError, json.JSONDecodeError) as exc:
+        browser_failure_type = type(exc).__name__
+        if browser_process_id is None or browser_process_start_time_ticks is None:
+            raise
     finally:
         if session_id is not None:
             with contextlib.suppress(Exception):
@@ -1159,14 +1164,20 @@ def _run_agent_task_browser_pass(
             driver.kill()
             driver.wait(timeout=5)
 
-    if result is None:
-        raise RuntimeError("Agent Task browser pass returned no result after shutdown")
     if browser_process_id is None or browser_process_start_time_ticks is None:
         raise RuntimeError("Agent Task browser process identity was not captured")
-    if not _wait_for_linux_process_identity_exit(
+    browser_process_terminated = _wait_for_linux_process_identity_exit(
         browser_process_id,
         browser_process_start_time_ticks,
-    ):
+    )
+    if browser_failure_type is not None:
+        return {
+            "failure_type": browser_failure_type,
+            "browser_process_terminated": browser_process_terminated,
+        }
+    if result is None:
+        raise RuntimeError("Agent Task browser pass returned no result after shutdown")
+    if not browser_process_terminated:
         raise RuntimeError("Agent Task browser process did not terminate")
     result["browser_process_terminated"] = True
     return result
@@ -1218,6 +1229,21 @@ def _run_agent_task_trial(
         }
     if result is None:
         raise RuntimeError("Agent Task browser pass returned no result")
+    returned_failure_type = result.get("failure_type")
+    if returned_failure_type is not None:
+        if not isinstance(returned_failure_type, str) or not returned_failure_type:
+            raise RuntimeError("Agent Task browser pass returned invalid failure evidence")
+        browser_process_terminated = result.get("browser_process_terminated")
+        if not isinstance(browser_process_terminated, bool):
+            raise RuntimeError("Agent Task browser pass returned invalid teardown evidence")
+        return {
+            "trial_number": trial_number,
+            "passed": False,
+            "failure_type": returned_failure_type,
+            "browser_process_terminated": browser_process_terminated,
+            "profile_cleaned": True,
+            "duration_ms": duration_ms,
+        }
 
     return {
         "trial_number": trial_number,
@@ -1774,4 +1800,4 @@ def main() -> int:
 
 
 if __name__ == "__main__":
-    raise SystemExit(main())
+    raise SystemExit(main())

tests/test_agent_task_failure_process_termination_contract.py

@@ -0,0 +1,166 @@
+"""Contract for browser-process termination evidence after Agent Task failure."""
+
+from __future__ import annotations
+
+import pathlib
+import runpy
+import unittest
+from unittest import mock
+
+ROOT = pathlib.Path(__file__).resolve().parents[1]
+RUNNER = ROOT / "scripts" / "ci" / "run_mv3_compatibility.py"
+
+
+class AgentTaskFailureProcessTerminationContractTests(unittest.TestCase):
+    """Require failed browser work to retain exact root-process teardown evidence."""
+
+    def _namespace(self, name: str) -> dict[str, object]:
+        return runpy.run_path(str(RUNNER), run_name=name)
+
+    def test_failure_cleanup_release_record_preserves_evidence_limits(self) -> None:
+        """The changed failure path needs its own release record and evidence limits."""
+
+        changelog = (ROOT / "CHANGELOG.md").read_text(encoding="utf-8")
+        doctoring = (ROOT / "docs" / "doctoring.md").read_text(encoding="utf-8")
+        self.assertIn(
+            "Failed controlled Agent Task runs now report whether their original browser process ended",
+            changelog,
+        )
+        self.assertIn(
+            "a failed task never becomes a pass merely because cleanup succeeded", changelog
+        )
+        self.assertIn("Process-observation errors leave termination unproven", doctoring)
+        self.assertIn(
+            "the original browser failure type is not retained in that fallback record",
+            doctoring,
+        )
+
+    def test_real_failure_path_distinguishes_observed_exit_from_observation_error(self) -> None:
+        """Exercise both owning helpers without launching a browser or trusting fake success."""
+
+        for exit_observation in (True, False, PermissionError("controlled read failure")):
+            with self.subTest(exit_observation=type(exit_observation).__name__):
+                namespace = self._namespace("agent_task_failure_observation_boundary")
+                browser_pass = namespace["_run_agent_task_browser_pass"]
+                run_trial = namespace["_run_agent_task_trial"]
+                driver = mock.Mock()
+
+                def request(_port, method, target, *_args):
+                    if method == "POST" and target == "/session":
+                        return {
+                            "value": {
+                                "sessionId": "controlled-session",
+                                "capabilities": {
+                                    "browserVersion": namespace["PINNED_CHROME_VERSION"],
+                                    "goog:processID": 321,
+                                },
+                            }
+                        }
+                    if method == "POST":
+                        raise RuntimeError("private controlled browser failure")
+                    return {}
+
+                exit_wait = mock.Mock(return_value=exit_observation)
+                if isinstance(exit_observation, Exception):
+                    exit_wait.side_effect = exit_observation
+                replacements = {
+                    "_free_loopback_port": lambda: 12345,
+                    "_wait_for_driver": lambda _port: None,
+                    "_json_request": request,
+                    "_read_linux_proc_stat_process_identity": lambda _pid: (321, 654),
+                    "_wait_for_linux_process_identity_exit": exit_wait,
+                }
+                with mock.patch.dict(browser_pass.__globals__, replacements), mock.patch.object(
+                    namespace["subprocess"], "Popen", return_value=driver
+                ):
+                    result = run_trial(
+                        pathlib.Path("controlled-chrome"),
+                        pathlib.Path("controlled-driver"),
+                        "http://127.0.0.1/controlled-fixture",
+                        21,
+                    )
+
+                driver.terminate.assert_called_once_with()
+                driver.wait.assert_called_once_with(timeout=5)
+                exit_wait.assert_called_once_with(321, 654)
+                self.assertIs(result["passed"], False)
+                self.assertIs(result["profile_cleaned"], True)
+                self.assertNotIn("private controlled browser failure", repr(result))
+                if isinstance(exit_observation, Exception):
+                    self.assertEqual(result["failure_type"], "PermissionError")
+                    self.assertNotIn("browser_process_terminated", result)
+                else:
+                    self.assertEqual(result["failure_type"], "RuntimeError")
+                    self.assertIs(result["browser_process_terminated"], exit_observation)
+
+    def test_browser_pass_retains_failure_process_termination_evidence(self) -> None:
+        """A browser-pass failure after identity capture must survive teardown as evidence."""
+
+        runner = RUNNER.read_text(encoding="utf-8")
+        start = runner.index("def _run_agent_task_browser_pass(")
+        end = runner.index("\ndef _run_agent_task_trial(", start)
+        browser_pass = runner[start:end]
+        for expected in (
+            "browser_failure_type: str | None = None",
+            "browser_failure_type = type(exc).__name__",
+            '"failure_type": browser_failure_type',
+            '"browser_process_terminated": browser_process_terminated',
+        ):
+            with self.subTest(expected=expected):
+                self.assertIn(expected, browser_pass)
+
+    def test_trial_preserves_failure_process_termination_evidence(self) -> None:
+        """The isolated trial must propagate failure teardown evidence after profile cleanup."""
+
+        namespace = self._namespace("agent_task_failure_process_termination_trial")
+        run_trial = namespace["_run_agent_task_trial"]
+
+        def fail_after_shutdown(*_args: object, **_kwargs: object) -> dict[str, object]:
+            return {
+                "failure_type": "RuntimeError",
+                "browser_process_terminated": True,
+            }
+
+        run_trial.__globals__["_run_agent_task_browser_pass"] = fail_after_shutdown
+        result = run_trial(
+            pathlib.Path("controlled-chrome"),
+            pathlib.Path("controlled-chromedriver"),
+            "http://127.0.0.1/controlled-fixture",
+            11,
+        )
+
+        self.assertEqual(result["trial_number"], 11)
+        self.assertIs(result["passed"], False)
+        self.assertEqual(result["failure_type"], "RuntimeError")
+        self.assertIs(result["browser_process_terminated"], True)
+        self.assertIs(result["profile_cleaned"], True)
+
+    def test_failed_trial_can_report_a_surviving_original_browser_process(self) -> None:
+        """Failure evidence must preserve a false result instead of inventing cleanup."""
+
+        namespace = self._namespace("agent_task_failure_process_survival_trial")
+        run_trial = namespace["_run_agent_task_trial"]
+
+        def fail_with_surviving_process(
+            *_args: object, **_kwargs: object
+        ) -> dict[str, object]:
+            return {
+                "failure_type": "RuntimeError",
+                "browser_process_terminated": False,
+            }
+
+        run_trial.__globals__["_run_agent_task_browser_pass"] = fail_with_surviving_process
+        result = run_trial(
+            pathlib.Path("controlled-chrome"),
+            pathlib.Path("controlled-chromedriver"),
+            "http://127.0.0.1/controlled-fixture",
+            12,
+        )
+
+        self.assertIs(result["passed"], False)
+        self.assertIs(result["browser_process_terminated"], False)
+        self.assertIs(result["profile_cleaned"], True)
+
+
+if __name__ == "__main__":
+    unittest.main()