diff --git a/AGENTS.md b/AGENTS.md index 6f747c38e..e9527a54d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -58,6 +58,10 @@ The organization currently documents a **solo-maintainer** governance condition. ## Architecture constraints - Keep Blink, V8, Skia, Viz, Dawn, Chromium sandboxing, Site Isolation, and Manifest V3 compatibility upstream-aligned. +- Pinned-Chromium presentation evidence may use only fixed CDP commands and declared static-fixture DOM outputs; page-provided scripts must never select commands or supply evaluation text. +- For hidden fixture outputs, read the bounded `textContent` property rather than rendered element text, and reject a probe baseline that already equals its target so an ACK cannot masquerade as a causal presentation transition. +- Browser-session failures may retain only a closed, credential-free WebDriver category; ChromeDriver startup/process diagnostics belong to their canonical owner lane and must not be captured here. Never emit remote or local diagnostic text into CI evidence. +- Hosted Ubuntu sandbox remediation is a canonical `.github` sandbox-helper workflow-contract dependency: do not add `--no-sandbox` or copy workflow setup into a product PR; adopt the reviewed immutable owner release and rerun the three browser trials. - New product logic belongs in Rust control-plane modules behind narrow adapters. - Rust crates must remain independently understandable and reusable. - Keep logical origin, resolved destination, operating-system TCP peer, TLS service identity, proxy route, and HTTP semantics as separate authority boundaries. diff --git a/CHANGELOG.md b/CHANGELOG.md index 118113e7f..4b6298b7c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,10 @@ All notable changes to OriginWeave are documented in this file. The format follo ## [Unreleased] +- Keep Agent Task evidence to the standard bounded WebDriver session-creation category; ChromeDriver startup/process diagnostics remain with their canonical owner. +- Preserve the bounded standard WebDriver session-creation failure category in Agent Task evidence while continuing to redact remote driver diagnostics. +- Recognize the standard session-creation category from a ChromeDriver HTTP error response before applying the generic HTTP failure boundary. +- Added a pinned-Chromium presentation probe to the controlled Agent Task evidence lane. It records a baseline, applies fixed viewport/DPR/timezone overrides before the observed navigation, then resets and proves the baseline returns through declared fixture DOM outputs. Hosted browser evidence remains required. - Keep WebDriver remote HTTP bodies, W3C error/message text, last-response startup detail, and mismatched remote `browserVersion` capability values out of CI exception strings while preserving fail-closed command/readiness/version decisions and the response-size bound. - Publish success-shaped MV3/Agent Task compatibility JSON only after both owned loopback fixture servers complete their shutdown post-conditions; browser/trial gate failures still emit bounded diagnostic evidence before raising. - Require loopback fixture-server cleanup to observe helper-thread termination after the bounded join, so a timed join cannot be treated as cleanup success while an owned server thread remains live. @@ -121,4 +125,4 @@ All notable changes to OriginWeave are documented in this file. The format follo - The hourly product agent has no Git metadata or repository authority. A separate post-verification publisher opens one PR and cannot approve or merge it. - The unprivileged OpenCode user is restricted to loopback egress during model execution, preventing runner-wide allow-listed endpoints from becoming direct source-exfiltration channels. -[Unreleased]: https://github.com/ContextualWisdomLab/OriginWeave/compare/main...HEAD \ No newline at end of file +[Unreleased]: https://github.com/ContextualWisdomLab/OriginWeave/compare/main...HEAD diff --git a/CLAUDE.md b/CLAUDE.md index ab08bc16f..4fde720e4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -11,4 +11,8 @@ Additional constraints: - Do not merge logical origin, destination authorization, direct TCP peer proof, TLS service identity, proxy routing, or HTTP resource policy into one ambient authority. - Do not add hostname reconnect, proxy-environment inheritance, dangerous certificate-verifier hooks, Common Name fallback, TLS 0-RTT, key logging, or secret extraction to a production TLS path. - Keep changes bounded to one product gap and preserve modular crate boundaries. +- Browser presentation probes use fixed CDP commands and static-fixture DOM observations only; never turn page content into executable input. +- Read hidden fixture observations through the bounded `textContent` property, and require every pre-override value to differ from its target before accepting a presentation transition. +- Preserve only a closed WebDriver category in browser evidence; ChromeDriver startup/process diagnostics remain in their canonical owner lane, and diagnostic text is never serialized. +- Treat hosted Ubuntu sandbox remediation as a `.github` workflow-owner dependency; preserve Chromium sandboxing and require a released helper contract before rerunning browser evidence. - Never claim a test, benchmark, browser integration, TLS identity, GPU execution, release, or merge succeeded without current exact-head evidence. diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 8a702c75f..132d6dedc 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -2,6 +2,13 @@ This is a dated delivery baseline, not a substitute for the PRD, TRD, roadmap, architecture decisions, or live GitHub state. It keeps buyer-visible gaps, current issues, active pull-request evidence, and commercial completion tracks in one discoverable place. Protected `main` is the implementation boundary: code in an open pull request is not shipped behavior. +## Live continuity note: 2026-09-09 + +- The next #292 evidence slice is stacked on the existing pinned-Chrome Agent Task owner. It uses fixed Chromium CDP viewport/DPR/timezone commands, static fixture DOM observations, an observed pre-override baseline, and observed explicit-reset restoration. It is active-PR evidence only until the exact Chrome for Testing job succeeds. +- A failed Agent Task session start records only the bounded credential-free WebDriver failure category; this consumer neither captures nor classifies ChromeDriver startup/process diagnostics. Those diagnostics belong to their canonical owner lane, and no remote or local diagnostic text enters CI evidence. +- Historical #299 browser job `34316793780` on `f8cb436c` classified all three pre-navigation failures as `sandbox`. The remediation remains the canonical `.github#1792` sandbox-helper workflow contract, not a leaf workflow copy or `--no-sandbox`; owner PR `.github#1857` and its CodeQL wake-race successor `.github#2056` remain unreleased, so consumer evidence is blocked pending immutable owner adoption and a fresh three-trial replay. +- This runner proves a narrow browser-evidence contract, not a product Browser Session implementation. It does not transfer #293's standard-BiDi capability boundary into protected main or claim full-profile admission. + ## Observed snapshot: 2026-08-26 ### Protected-main truth diff --git a/docs/traceability/agent-task-cleanup-failure-provenance.md b/docs/traceability/agent-task-cleanup-failure-provenance.md new file mode 100644 index 000000000..75c8fc627 --- /dev/null +++ b/docs/traceability/agent-task-cleanup-failure-provenance.md @@ -0,0 +1,33 @@ +# Agent Task cleanup-failure provenance + +## Problem + +Agent Task browser and profile cleanup wrappers preserve an earlier causal failure in Python exception chaining, but predecessor `d329f9e5dfbd5e4200eeec5fc40bf37d526fb77b` serialized only the wrapper `failure_type` into durable browser evidence. A page-observed/action failure followed by WebDriver-session cleanup failure, or a browser-pass failure followed by profile cleanup failure, therefore became indistinguishable from a cleanup-only failure once the process artifact was consumed. + +Review `5151447157` records the first exact-head finding. Test-first commit `70c710f00ad06a1437d74d10604820fb30272be2` requires the emitted failed-trial record to retain the bounded primary failure type and cleanup failure type while excluding hostile exception detail. Production commit `3f8f1cc15ec6c37c5b85e8cf6959c5d1c039b1fc` adds only closed exception-class metadata to the two cleanup wrappers and the Agent Task failure materializer. Commit `a297811635636942afa8f0d650f0ec3f9183dfca` binds the profile-cleanup regression to the same explicit primary-failure metadata used by production. Commit `d31a8d5e54a5b14fbc5e3a17980f97ec40b43c81` corrects the focused harness so cleanup exceptions and the `main()` materializer come from the same `runpy` module instance; otherwise Python class identity would make an artificial cross-module exception miss the production `isinstance` branch. + +Review `5151777477` found one remaining nested case: browser/action failure A can be wrapped by WebDriver-session cleanup failure B and then by profile cleanup failure C. The predecessor durable record retained only `BrowserProfileCleanupError`, `BrowserSessionCleanupError`, and C's class name, losing the original A type and B's cleanup type. Test-first `2320dd7f7313b443a87f5735266330ddd61f6053` requires the three-level chain to retain the root failure type plus both cleanup-stage type names while rejecting hostile messages. Production `08133c3166dd899fe5925ce96781023e1dce5ed7` propagates only those already-bounded class names through `BrowserProfileCleanupError` and the existing Agent Task materializer. + +Review `5156823718` found the analogous session-start path was still lossy: `WebDriverSessionNotCreatedError` is intentionally wrapped by `AgentTaskSessionStartError`, but a later profile-cleanup failure caused durable JSON to retain only the wrapper name and discard the already-bounded WebDriver category. Test-first `8e68cf8228a4d602defe1d23de3abfcf4cd86367` requires `session not created -> profile cleanup` evidence to keep `failure_type=BrowserProfileCleanupError`, `failure_cause_type=WebDriverSessionNotCreatedError`, and the outer cleanup type without exposing messages. Hosted CI `34374258739` is an executed RED for that test-first tree: the Rust-contract job reached Python repository contracts and failed there, while exact production coverage remained successful. Production `0050fffd1e5f50dca568db3eeeaba86b0c8f890f` minimally unwraps the existing `AgentTaskSessionStartError.session_error_type` metadata when profile cleanup wraps that error; it adds no new diagnostic source or taxonomy. + +Review `5158052125` found the remaining three-stage session-start path: `WebDriverSessionNotCreatedError -> AgentTaskSessionStartError -> BrowserSessionCleanupError -> BrowserProfileCleanupError`. The session-cleanup wrapper still reduced its primary error to `AgentTaskSessionStartError`, so the outer profile wrapper could no longer recover the already-bounded WebDriver category. Test-first `7e612800cf8069385c3aa62976a2c8268ecb340b` adds the real JSON-materialization regression and hosted CI `34386678497` is executed RED at Python repository contracts; production coverage on that test-only tree still passed. Production `960c1acc85c8e582a2a9c6a792eca29db39cfee0` minimally forwards `AgentTaskSessionStartError.session_error_type` at the first cleanup wrapper, without retaining an exception object, message, path, process diagnostic, or new category. + +## Constraints + +The artifact must not serialize `str(error)`, WebDriver response text, browser/page-controlled values, profile paths, ChromeDriver process diagnostics, credentials, or secret-shaped content. ChromeDriver startup/process classification remains owned by PR #148. Sandbox-helper workflow mechanics remain owned by the canonical `.github` path. Browser interaction, presentation apply/reset, URL stability, cleanup semantics, and the three-trial denominator must not change. + +## Alternatives + +Dropping the original browser failure was rejected because it destroys causal provenance after cleanup wraps the failure. Serializing exception messages or the full exception chain was rejected because remote, page-controlled, filesystem, or secret-bearing detail could cross the CI evidence boundary. Inferring a primary type from arbitrary chained exceptions at serialization time was rejected because cleanup-only and secondary-cleanup cases can have different chaining semantics. Flattening a nested cleanup chain to the wrapper class name was rejected because it discards the already-bounded root and session-cleanup types needed to distinguish A -> B -> C from an unrelated wrapper failure. Keeping separate `runpy` module instances in the contract was rejected because identical-looking exception classes from separate executions are distinct Python class objects and would test an impossible production boundary rather than the runner's real materializer. Treating `AgentTaskSessionStartError` itself as the durable root was rejected because the wrapper already carries a narrower closed WebDriver session-error category; losing it only when cleanup also fails makes RCA less precise without improving redaction. Reconstructing the WebDriver category later in `BrowserProfileCleanupError` was rejected because the first cleanup wrapper had already discarded it; preserving the bounded category at that first wrapper is the smaller causal repair and keeps the outer wrapper generic. + +## Decision + +`BrowserSessionCleanupError` retains `cleanup_error_type` and optional `primary_error_type`; when its primary is `AgentTaskSessionStartError`, it forwards that wrapper's already-bounded `session_error_type` rather than the wrapper class name. `BrowserProfileCleanupError` retains its own `cleanup_error_type`; when its primary error is the local `BrowserSessionCleanupError`, it also preserves that wrapper's bounded `cleanup_error_type` as `session_cleanup_error_type` and forwards the wrapper's bounded root `primary_error_type` when one exists. When the primary is `AgentTaskSessionStartError`, it forwards that wrapper's already-bounded `session_error_type` as `primary_error_type`. Failed Agent Task evidence publishes the root as `failure_cause_type`, the session cleanup as optional `session_cleanup_error_type`, and the outer cleanup as `cleanup_error_type`. Existing one-stage cleanup records remain compatible. No message text is added. + +## Risk and effect + +The additional behavior exposes only a Python exception class name already retained inside a bounded local wrapper; it does not retain the exception object or any raw diagnostic. This improves post-run RCA by preserving the standard session-creation category even when WebDriver-session deletion and profile deletion both fail, and by distinguishing the root browser/action failure, WebDriver-session cleanup failure, and profile-cleanup failure without broadening diagnostic authority. The repair does not establish real-browser GREEN; exact-head repository gates and pinned-Chromium execution remain independent acceptance evidence. + +## Acceptance + +The focused contract must prove single-stage session/profile cleanup still emit `failure_type`, `failure_cause_type`, and `cleanup_error_type`. The nested contract must additionally prove `browser/action A -> session cleanup B -> profile cleanup C` emits the closed root type, `session_cleanup_error_type`, and final `cleanup_error_type`, while hostile primary/session/profile messages remain absent from JSON output. A `WebDriverSessionNotCreatedError -> AgentTaskSessionStartError -> profile cleanup` chain must emit the underlying `WebDriverSessionNotCreatedError` as `failure_cause_type`. The three-stage `WebDriverSessionNotCreatedError -> AgentTaskSessionStartError -> session cleanup -> profile cleanup` chain must preserve that same root category plus both cleanup types and must not serialize cleanup detail. Contracts must exercise the same loaded runner namespace as the failure materializer. Exact-head repository CI must pass before this repair is considered repository GREEN. Pinned Chromium must still complete the existing three-trial causal browser sequence before #299 can claim browser acceptance. diff --git a/scripts/ci/run_mv3_compatibility.py b/scripts/ci/run_mv3_compatibility.py index d9dcd5883..2de4765ed 100755 --- a/scripts/ci/run_mv3_compatibility.py +++ b/scripts/ci/run_mv3_compatibility.py @@ -37,6 +37,10 @@ REPEATABILITY_TRIALS = 3 AGENT_TASK_REPEATABILITY_TRIALS = 3 AGENT_TASK_INPUT_VALUE = "originweave controlled input" +PRESENTATION_VIEWPORT_WIDTH = 1200 +PRESENTATION_VIEWPORT_HEIGHT = 800 +PRESENTATION_DEVICE_PIXEL_RATIO = 2 +PRESENTATION_TIMEZONE = "Pacific/Kiritimati" REQUEST_TIMEOUT_SECONDS = 5.0 STARTUP_TIMEOUT_SECONDS = 20.0 FIXTURE_TIMEOUT_SECONDS = 20.0 @@ -55,8 +59,18 @@ def log_message(self, _format: str, *args: object) -> None: class BrowserSessionCleanupError(RuntimeError): """Report bounded WebDriver-session cleanup failure without echoing remote text.""" - def __init__(self, cleanup_error: BaseException) -> None: + def __init__( + self, + cleanup_error: BaseException, + primary_error: BaseException | None = None, + ) -> None: self.cleanup_error_type = type(cleanup_error).__name__ + if isinstance(primary_error, AgentTaskSessionStartError): + self.primary_error_type = primary_error.session_error_type + else: + self.primary_error_type = ( + type(primary_error).__name__ if primary_error is not None else None + ) super().__init__( "WebDriver session cleanup failed; see the chained causal browser failure" ) @@ -65,21 +79,48 @@ def __init__(self, cleanup_error: BaseException) -> None: class BrowserProfileCleanupError(RuntimeError): """Report bounded profile cleanup failure without exposing filesystem details.""" - def __init__(self, cleanup_error: BaseException) -> None: + def __init__( + self, + cleanup_error: BaseException, + primary_error: BaseException | None = None, + ) -> None: self.cleanup_error_type = type(cleanup_error).__name__ + self.session_cleanup_error_type = ( + primary_error.cleanup_error_type + if isinstance(primary_error, BrowserSessionCleanupError) + else None + ) + if ( + isinstance(primary_error, BrowserSessionCleanupError) + and primary_error.primary_error_type is not None + ): + self.primary_error_type = primary_error.primary_error_type + elif isinstance(primary_error, AgentTaskSessionStartError): + self.primary_error_type = primary_error.session_error_type + else: + self.primary_error_type = ( + type(primary_error).__name__ if primary_error is not None else None + ) super().__init__( "browser profile cleanup failed; see the chained causal browser failure" ) class AgentTaskSessionStartError(RuntimeError): - """Classify a failed Agent Task browser session without exposing driver text.""" + """Record a failed Agent Task browser session without exposing driver text.""" def __init__(self, session_error: BaseException) -> None: self.session_error_type = type(session_error).__name__ super().__init__("Agent Task browser session failed to start") +class WebDriverSessionNotCreatedError(RuntimeError): + """Report the standard session-creation failure without remote diagnostics.""" + + def __init__(self) -> None: + super().__init__("WebDriver could not create a browser session") + + def _free_loopback_port() -> int: """Reserve and release one loopback TCP port for a short-lived local service.""" @@ -127,7 +168,7 @@ def _json_request( if method not in {"GET", "POST", "DELETE"}: raise ValueError("unsupported ChromeDriver method") if not path.startswith("/") or "://" in path or any(char in path for char in "\r\n"): - raise ValueError("invalid ChromeDriver path") + raise ValueError("invalid WebDriver path") body = None if payload is None else json.dumps(payload).encode("utf-8") connection = http.client.HTTPConnection("127.0.0.1", driver_port, timeout=timeout) @@ -143,6 +184,20 @@ def _json_request( if len(raw) > MAX_WEBDRIVER_RESPONSE_BYTES: raise RuntimeError("WebDriver response exceeded the bounded JSON limit") if response.status >= 400: + try: + error_response = json.loads(raw.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + raise RuntimeError( + f"WebDriver HTTP request failed with status {response.status}" + ) from None + error_value = ( + error_response.get("value") if isinstance(error_response, dict) else None + ) + if ( + isinstance(error_value, dict) + and error_value.get("error") == "session not created" + ): + raise WebDriverSessionNotCreatedError() raise RuntimeError( f"WebDriver HTTP request failed with status {response.status}" ) @@ -153,6 +208,8 @@ def _json_request( if not isinstance(decoded, dict): raise RuntimeError("WebDriver returned a non-object JSON payload") value = decoded.get("value") + if isinstance(value, dict) and value.get("error") == "session not created": + raise WebDriverSessionNotCreatedError() if isinstance(value, dict) and value.get("error"): raise RuntimeError("WebDriver command failed") return decoded @@ -230,6 +287,96 @@ def _get_element_semantics( return role, label +def _presentation_cdp_path(session_id: str) -> str: + """Return ChromeDriver's fixed vendor endpoint for this exact session.""" + + return _webdriver_path(session_id, "/goog/cdp/execute") + + +def _apply_presentation_probe(driver_port: int, session_id: str) -> None: + """Apply only the pinned viewport, DPR, and timezone probe before navigation.""" + + _json_request( + driver_port, + "POST", + _presentation_cdp_path(session_id), + { + "cmd": "Emulation.setDeviceMetricsOverride", + "params": { + "width": PRESENTATION_VIEWPORT_WIDTH, + "height": PRESENTATION_VIEWPORT_HEIGHT, + "deviceScaleFactor": PRESENTATION_DEVICE_PIXEL_RATIO, + "mobile": False, + }, + }, + ) + _json_request( + driver_port, + "POST", + _presentation_cdp_path(session_id), + { + "cmd": "Emulation.setTimezoneOverride", + "params": {"timezoneId": PRESENTATION_TIMEZONE}, + }, + ) + + +def _reset_presentation_probe(driver_port: int, session_id: str) -> None: + """Remove the exact probe overrides before reusing the browser session.""" + + _json_request( + driver_port, + "POST", + _presentation_cdp_path(session_id), + {"cmd": "Emulation.clearDeviceMetricsOverride", "params": {}}, + ) + _json_request( + driver_port, + "POST", + _presentation_cdp_path(session_id), + {"cmd": "Emulation.setTimezoneOverride", "params": {"timezoneId": ""}}, + ) + + +def _presentation_probe_target() -> dict[str, str]: + """Return the exact fixed page-observed target for this evidence probe.""" + + return { + "viewport": f"{PRESENTATION_VIEWPORT_WIDTH}x{PRESENTATION_VIEWPORT_HEIGHT}", + "device_pixel_ratio": str(PRESENTATION_DEVICE_PIXEL_RATIO), + "timezone": PRESENTATION_TIMEZONE, + } + + +def _validate_presentation_probe_baseline(baseline: dict[str, str]) -> None: + """Require an observable transition for every presentation surface under test.""" + + target = _presentation_probe_target() + if any(baseline.get(key) == value for key, value in target.items()): + raise RuntimeError("presentation probe baseline already matched target") + + +def _read_presentation_probe(driver_port: int, session_id: str) -> dict[str, str]: + """Read only declared fixture observations through bounded element endpoints.""" + + observed: dict[str, str] = {} + for key, selector in { + "viewport": "#presentation-viewport", + "device_pixel_ratio": "#presentation-device-pixel-ratio", + "timezone": "#presentation-timezone", + }.items(): + element_id = _find_element(driver_port, session_id, selector) + value = _json_request( + driver_port, + "GET", + _element_command_path(session_id, element_id, "/property/textContent"), + ).get("value") + if not isinstance(value, str): + raise RuntimeError("presentation probe observation was malformed") + observed[key] = value + return observed + + def _cleanup_browser_session(driver_port: int, session_id: str) -> None: """Delete one WebDriver session through the fixed loopback authority.""" @@ -256,7 +403,7 @@ def _cleanup_browser_session_preserving_primary( http.client.HTTPException, json.JSONDecodeError, ) as cleanup_error: - bounded_error = BrowserSessionCleanupError(cleanup_error) + bounded_error = BrowserSessionCleanupError(cleanup_error, primary_error) if primary_error is None: raise bounded_error from cleanup_error raise bounded_error from primary_error @@ -567,7 +714,11 @@ def _run_agent_task_browser_pass( driver_port = _free_loopback_port() session_id: str | None = None driver = subprocess.Popen( - [str(chromedriver_bin), f"--port={driver_port}", "--allowed-ips=127.0.0.1"], + [ + str(chromedriver_bin), + f"--port={driver_port}", + "--allowed-ips=127.0.0.1", + ], stdout=subprocess.DEVNULL, stderr=subprocess.STDOUT, text=True, @@ -634,6 +785,19 @@ def _run_agent_task_browser_pass( ).get("value") if initial_url != fixture_url: raise RuntimeError("Agent Task initial URL mismatch") + baseline_presentation = _read_presentation_probe(driver_port, session_id) + _validate_presentation_probe_baseline(baseline_presentation) + target_presentation = _presentation_probe_target() + _apply_presentation_probe(driver_port, session_id) + _json_request( + driver_port, + "POST", + _webdriver_path(session_id, "/url"), + {"url": fixture_url}, + ) + applied_presentation = _read_presentation_probe(driver_port, session_id) + if applied_presentation != target_presentation: + raise RuntimeError("presentation probe post-condition failed") input_element = _find_element(driver_port, session_id, "#task-text") input_role, input_name = _get_element_semantics( driver_port, @@ -745,6 +909,16 @@ def _run_agent_task_browser_pass( url_unchanged = url_unchanged and accepted_outcome_url == initial_url if not url_unchanged: raise RuntimeError("Agent Task URL changed before accepted outcome") + _reset_presentation_probe(driver_port, session_id) + _json_request( + driver_port, + "POST", + _webdriver_path(session_id, "/url"), + {"url": fixture_url}, + ) + cleanup_presentation = _read_presentation_probe(driver_port, session_id) + if cleanup_presentation != baseline_presentation: + raise RuntimeError("presentation probe cleanup post-condition failed") return { "browser_version": browser_version, "pre_action_baseline_verified": True, @@ -757,6 +931,8 @@ def _run_agent_task_browser_pass( "input_semantics_verified": True, "submit_semantics_verified": True, "extensions_disabled_requested": True, + "presentation_applied": True, + "presentation_cleanup_verified": True, "duration_ms": round((time.monotonic() - started) * 1000), } finally: @@ -806,7 +982,7 @@ def _run_agent_task_trial( try: temporary_profile.cleanup() except OSError as cleanup_error: - bounded_error = BrowserProfileCleanupError(cleanup_error) + bounded_error = BrowserProfileCleanupError(cleanup_error, primary_error) if primary_error is None: raise bounded_error from cleanup_error raise bounded_error from primary_error @@ -829,6 +1005,8 @@ def _run_agent_task_trial( "input_semantics_verified": result["input_semantics_verified"], "submit_semantics_verified": result["submit_semantics_verified"], "extensions_disabled_requested": result["extensions_disabled_requested"], + "presentation_applied": result["presentation_applied"], + "presentation_cleanup_verified": result["presentation_cleanup_verified"], "profile_cleaned": profile_cleaned, "duration_ms": round((time.monotonic() - trial_started) * 1000), } @@ -850,6 +1028,8 @@ def _agent_task_surfaces_complete(agent_task_trials: list[dict[str, Any]]) -> bo and trial.get("url_unchanged") is True and trial.get("input_semantics_verified") is True and trial.get("submit_semantics_verified") is True + and trial.get("presentation_applied") is True + and trial.get("presentation_cleanup_verified") is True and trial.get("profile_cleaned") is True for trial in agent_task_trials ) @@ -978,13 +1158,28 @@ def main() -> int: http.client.HTTPException, json.JSONDecodeError, ) as error: - agent_task_trials.append( - { - "trial_number": trial_number, - "passed": False, - "failure_type": type(error).__name__, - } - ) + failed_trial: dict[str, Any] = { + "trial_number": trial_number, + "passed": False, + "failure_type": type(error).__name__, + } + if isinstance(error, AgentTaskSessionStartError): + failed_trial["failure_cause_type"] = error.session_error_type + elif isinstance( + error, + (BrowserSessionCleanupError, BrowserProfileCleanupError), + ): + failed_trial["cleanup_error_type"] = error.cleanup_error_type + if ( + isinstance(error, BrowserProfileCleanupError) + and error.session_cleanup_error_type is not None + ): + failed_trial["session_cleanup_error_type"] = ( + error.session_cleanup_error_type + ) + if error.primary_error_type is not None: + failed_trial["failure_cause_type"] = error.primary_error_type + agent_task_trials.append(failed_trial) agent_task_successful_trials = sum( 1 for trial in agent_task_trials if trial.get("passed") is True @@ -1048,4 +1243,4 @@ def main() -> int: if __name__ == "__main__": - raise SystemExit(main()) \ No newline at end of file + raise SystemExit(main()) diff --git a/tests/fixtures/agent_task_basic/index.html b/tests/fixtures/agent_task_basic/index.html index 510b239f1..28358b8ca 100644 --- a/tests/fixtures/agent_task_basic/index.html +++ b/tests/fixtures/agent_task_basic/index.html @@ -17,6 +17,9 @@

Controlled Agent Task

idle + + +

Controlled Agent Task const taskText = document.getElementById("task-text"); const result = document.getElementById("task-result"); + document.getElementById("presentation-viewport").textContent = + `${window.innerWidth}x${window.innerHeight}`; + document.getElementById("presentation-device-pixel-ratio").textContent = + String(window.devicePixelRatio); + document.getElementById("presentation-timezone").textContent = + Intl.DateTimeFormat().resolvedOptions().timeZone; + form.addEventListener("submit", (event) => { event.preventDefault(); result.dataset.state = "submitted"; diff --git a/tests/test_agent_task_action_transition_evidence_contract.py b/tests/test_agent_task_action_transition_evidence_contract.py index 4dde74097..e5e144b04 100644 --- a/tests/test_agent_task_action_transition_evidence_contract.py +++ b/tests/test_agent_task_action_transition_evidence_contract.py @@ -160,8 +160,12 @@ def test_surface_completeness_requires_transition_baseline_evidence(self) -> Non trial["input_value_verified"] = True self.assertFalse(complete([trial])) trial["pre_click_baseline_verified"] = True + self.assertFalse(complete([trial])) + trial["presentation_applied"] = True + self.assertFalse(complete([trial])) + trial["presentation_cleanup_verified"] = True self.assertTrue(complete([trial])) if __name__ == "__main__": - unittest.main() \ No newline at end of file + unittest.main() diff --git a/tests/test_agent_task_chromium_sandbox_contract.py b/tests/test_agent_task_chromium_sandbox_contract.py index 3ea9258f2..6615d8591 100644 --- a/tests/test_agent_task_chromium_sandbox_contract.py +++ b/tests/test_agent_task_chromium_sandbox_contract.py @@ -22,5 +22,15 @@ def test_agent_task_browser_pass_does_not_disable_chromium_sandbox(self) -> None self.assertNotIn('"--no-sandbox"', browser_pass_source) + def test_agent_task_browser_pass_does_not_own_chromedriver_diagnostics(self) -> None: + """Keep ChromeDriver process diagnostics in their canonical owner lane.""" + + namespace = runpy.run_path(str(RUNNER), run_name="agent_task_sandbox_contract") + browser_pass_source = inspect.getsource(namespace["_run_agent_task_browser_pass"]) + + self.assertNotIn('"--verbose"', browser_pass_source) + self.assertNotIn('"--log-path=', browser_pass_source) + self.assertNotIn("_classify_chromedriver_startup_diagnostic", browser_pass_source) + if __name__ == "__main__": unittest.main() diff --git a/tests/test_agent_task_cleanup_failure_evidence_contract.py b/tests/test_agent_task_cleanup_failure_evidence_contract.py new file mode 100644 index 000000000..f339cd41a --- /dev/null +++ b/tests/test_agent_task_cleanup_failure_evidence_contract.py @@ -0,0 +1,296 @@ +"""Contract for bounded Agent Task cleanup-failure provenance in emitted evidence.""" + +from __future__ import annotations + +import io +import json +import os +import pathlib +import runpy +import unittest +from contextlib import redirect_stdout +from unittest.mock import patch + +ROOT = pathlib.Path(__file__).resolve().parents[1] +RUNNER = ROOT / "scripts" / "ci" / "run_mv3_compatibility.py" + + +class AgentTaskCleanupFailureEvidenceContractTests(unittest.TestCase): + """Keep the first causal failure distinguishable from secondary cleanup failure.""" + + def _assert_cleanup_failure_evidence( + self, + cleanup_kind: str, + *, + expected_failure_type: str, + expected_cleanup_type: str, + ) -> None: + namespace = runpy.run_path(str(RUNNER), run_name="agent_task_cleanup_evidence") + main_globals = namespace["main"].__globals__ + primary = RuntimeError("buyer-secret-primary-detail") + + if cleanup_kind == "session": + cleanup = namespace["_cleanup_browser_session_preserving_primary"] + cleanup_error_type = namespace["BrowserSessionCleanupError"] + + def failed_cleanup(*_args: object, **_kwargs: object) -> None: + raise OSError("buyer-secret-cleanup-detail") + + cleanup.__globals__["_cleanup_browser_session"] = failed_cleanup + try: + cleanup(9515, "session-1", primary) + except cleanup_error_type as raised: + cleanup_failure = raised + else: + self.fail("session cleanup double did not fail") + elif cleanup_kind == "profile": + cleanup_error_type = namespace["BrowserProfileCleanupError"] + cleanup_failure = cleanup_error_type( + OSError("buyer-secret-cleanup-detail"), + primary, + ) + else: + self.fail("unsupported cleanup evidence test kind") + + class FakeServer: + server_port = 9515 + + def start_fixture_server(_directory: pathlib.Path) -> tuple[FakeServer, object]: + return FakeServer(), object() + + def successful_restart_trial(*_args: object, **_kwargs: object) -> dict[str, object]: + return {"trial_number": 1, "passed": True, "surfaces": {"worker": True}} + + def failed_agent_task_trial(*_args: object, **_kwargs: object) -> dict[str, object]: + raise cleanup_failure + + main_globals.update( + { + "_start_fixture_server": start_fixture_server, + "_stop_fixture_server": lambda *_args: None, + "_run_restart_trial": successful_restart_trial, + "_run_agent_task_trial": failed_agent_task_trial, + "REPEATABILITY_TRIALS": 1, + "AGENT_TASK_REPEATABILITY_TRIALS": 1, + } + ) + + output = io.StringIO() + with patch.dict( + os.environ, + {"CHROME_BIN": "/bin/sh", "CHROMEDRIVER_BIN": "/bin/sh"}, + ), redirect_stdout(output), self.assertRaisesRegex( + RuntimeError, + r"^Agent Task repeatability gate failed: 0/1 trials passed$", + ): + namespace["main"]() + + evidence = json.loads(output.getvalue()) + failed_trial = evidence["agent_task"]["trial_results"][0] + self.assertEqual(failed_trial["failure_type"], expected_failure_type) + self.assertEqual(failed_trial["failure_cause_type"], "RuntimeError") + self.assertEqual(failed_trial["cleanup_error_type"], expected_cleanup_type) + self.assertNotIn("buyer-secret-primary-detail", output.getvalue()) + self.assertNotIn("buyer-secret-cleanup-detail", output.getvalue()) + + def test_session_cleanup_failure_retains_bounded_primary_and_cleanup_types(self) -> None: + """Durable evidence must retain the primary browser type across DELETE failure.""" + + self._assert_cleanup_failure_evidence( + "session", + expected_failure_type="BrowserSessionCleanupError", + expected_cleanup_type="OSError", + ) + + def test_profile_cleanup_failure_retains_bounded_primary_and_cleanup_types(self) -> None: + """Durable evidence must retain the primary browser type across profile cleanup.""" + + self._assert_cleanup_failure_evidence( + "profile", + expected_failure_type="BrowserProfileCleanupError", + expected_cleanup_type="OSError", + ) + + def test_profile_cleanup_after_session_cleanup_retains_bounded_chain(self) -> None: + """Durable evidence must retain root, session-cleanup, and profile-cleanup types.""" + + namespace = runpy.run_path(str(RUNNER), run_name="agent_task_nested_cleanup_evidence") + main_globals = namespace["main"].__globals__ + session_cleanup_error_type = namespace["BrowserSessionCleanupError"] + profile_cleanup_error_type = namespace["BrowserProfileCleanupError"] + primary = RuntimeError("buyer-secret-primary-detail") + session_cleanup_failure = session_cleanup_error_type( + OSError("buyer-secret-session-cleanup-detail"), + primary, + ) + profile_cleanup_failure = profile_cleanup_error_type( + PermissionError("buyer-secret-profile-cleanup-detail"), + session_cleanup_failure, + ) + + class FakeServer: + server_port = 9515 + + def start_fixture_server(_directory: pathlib.Path) -> tuple[FakeServer, object]: + return FakeServer(), object() + + def successful_restart_trial(*_args: object, **_kwargs: object) -> dict[str, object]: + return {"trial_number": 1, "passed": True, "surfaces": {"worker": True}} + + def failed_agent_task_trial(*_args: object, **_kwargs: object) -> dict[str, object]: + raise profile_cleanup_failure + + main_globals.update( + { + "_start_fixture_server": start_fixture_server, + "_stop_fixture_server": lambda *_args: None, + "_run_restart_trial": successful_restart_trial, + "_run_agent_task_trial": failed_agent_task_trial, + "REPEATABILITY_TRIALS": 1, + "AGENT_TASK_REPEATABILITY_TRIALS": 1, + } + ) + + output = io.StringIO() + with patch.dict( + os.environ, + {"CHROME_BIN": "/bin/sh", "CHROMEDRIVER_BIN": "/bin/sh"}, + ), redirect_stdout(output), self.assertRaisesRegex( + RuntimeError, + r"^Agent Task repeatability gate failed: 0/1 trials passed$", + ): + namespace["main"]() + + failed_trial = json.loads(output.getvalue())["agent_task"]["trial_results"][0] + self.assertEqual(failed_trial["failure_type"], "BrowserProfileCleanupError") + self.assertEqual(failed_trial["failure_cause_type"], "RuntimeError") + self.assertEqual(failed_trial["session_cleanup_error_type"], "OSError") + self.assertEqual(failed_trial["cleanup_error_type"], "PermissionError") + self.assertNotIn("buyer-secret-primary-detail", output.getvalue()) + self.assertNotIn("buyer-secret-session-cleanup-detail", output.getvalue()) + self.assertNotIn("buyer-secret-profile-cleanup-detail", output.getvalue()) + + def test_profile_cleanup_after_session_start_failure_retains_webdriver_category(self) -> None: + """Secondary profile cleanup must not hide the bounded session-start category.""" + + namespace = runpy.run_path(str(RUNNER), run_name="agent_task_start_cleanup_evidence") + main_globals = namespace["main"].__globals__ + profile_cleanup_error_type = namespace["BrowserProfileCleanupError"] + session_start_error_type = namespace["AgentTaskSessionStartError"] + webdriver_start_error_type = namespace["WebDriverSessionNotCreatedError"] + session_start_failure = session_start_error_type(webdriver_start_error_type()) + profile_cleanup_failure = profile_cleanup_error_type( + PermissionError("buyer-secret-profile-cleanup-detail"), + session_start_failure, + ) + + class FakeServer: + server_port = 9515 + + def start_fixture_server(_directory: pathlib.Path) -> tuple[FakeServer, object]: + return FakeServer(), object() + + def successful_restart_trial(*_args: object, **_kwargs: object) -> dict[str, object]: + return {"trial_number": 1, "passed": True, "surfaces": {"worker": True}} + + def failed_agent_task_trial(*_args: object, **_kwargs: object) -> dict[str, object]: + raise profile_cleanup_failure + + main_globals.update( + { + "_start_fixture_server": start_fixture_server, + "_stop_fixture_server": lambda *_args: None, + "_run_restart_trial": successful_restart_trial, + "_run_agent_task_trial": failed_agent_task_trial, + "REPEATABILITY_TRIALS": 1, + "AGENT_TASK_REPEATABILITY_TRIALS": 1, + } + ) + + output = io.StringIO() + with patch.dict( + os.environ, + {"CHROME_BIN": "/bin/sh", "CHROMEDRIVER_BIN": "/bin/sh"}, + ), redirect_stdout(output), self.assertRaisesRegex( + RuntimeError, + r"^Agent Task repeatability gate failed: 0/1 trials passed$", + ): + namespace["main"]() + + failed_trial = json.loads(output.getvalue())["agent_task"]["trial_results"][0] + self.assertEqual(failed_trial["failure_type"], "BrowserProfileCleanupError") + self.assertEqual( + failed_trial["failure_cause_type"], + "WebDriverSessionNotCreatedError", + ) + self.assertEqual(failed_trial["cleanup_error_type"], "PermissionError") + self.assertNotIn("buyer-secret-profile-cleanup-detail", output.getvalue()) + + def test_nested_session_and_profile_cleanup_after_session_start_retains_webdriver_category( + self, + ) -> None: + """Two cleanup failures must still retain the bounded WebDriver start category.""" + + namespace = runpy.run_path(str(RUNNER), run_name="agent_task_triple_cleanup_evidence") + main_globals = namespace["main"].__globals__ + session_cleanup_error_type = namespace["BrowserSessionCleanupError"] + profile_cleanup_error_type = namespace["BrowserProfileCleanupError"] + session_start_error_type = namespace["AgentTaskSessionStartError"] + webdriver_start_error_type = namespace["WebDriverSessionNotCreatedError"] + session_start_failure = session_start_error_type(webdriver_start_error_type()) + session_cleanup_failure = session_cleanup_error_type( + OSError("buyer-secret-session-cleanup-detail"), + session_start_failure, + ) + profile_cleanup_failure = profile_cleanup_error_type( + PermissionError("buyer-secret-profile-cleanup-detail"), + session_cleanup_failure, + ) + + class FakeServer: + server_port = 9515 + + def start_fixture_server(_directory: pathlib.Path) -> tuple[FakeServer, object]: + return FakeServer(), object() + + def successful_restart_trial(*_args: object, **_kwargs: object) -> dict[str, object]: + return {"trial_number": 1, "passed": True, "surfaces": {"worker": True}} + + def failed_agent_task_trial(*_args: object, **_kwargs: object) -> dict[str, object]: + raise profile_cleanup_failure + + main_globals.update( + { + "_start_fixture_server": start_fixture_server, + "_stop_fixture_server": lambda *_args: None, + "_run_restart_trial": successful_restart_trial, + "_run_agent_task_trial": failed_agent_task_trial, + "REPEATABILITY_TRIALS": 1, + "AGENT_TASK_REPEATABILITY_TRIALS": 1, + } + ) + + output = io.StringIO() + with patch.dict( + os.environ, + {"CHROME_BIN": "/bin/sh", "CHROMEDRIVER_BIN": "/bin/sh"}, + ), redirect_stdout(output), self.assertRaisesRegex( + RuntimeError, + r"^Agent Task repeatability gate failed: 0/1 trials passed$", + ): + namespace["main"]() + + failed_trial = json.loads(output.getvalue())["agent_task"]["trial_results"][0] + self.assertEqual(failed_trial["failure_type"], "BrowserProfileCleanupError") + self.assertEqual( + failed_trial["failure_cause_type"], + "WebDriverSessionNotCreatedError", + ) + self.assertEqual(failed_trial["session_cleanup_error_type"], "OSError") + self.assertEqual(failed_trial["cleanup_error_type"], "PermissionError") + self.assertNotIn("buyer-secret-session-cleanup-detail", output.getvalue()) + self.assertNotIn("buyer-secret-profile-cleanup-detail", output.getvalue()) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_agent_task_extension_isolation_evidence_contract.py b/tests/test_agent_task_extension_isolation_evidence_contract.py index 31eb60b6f..3674856f7 100644 --- a/tests/test_agent_task_extension_isolation_evidence_contract.py +++ b/tests/test_agent_task_extension_isolation_evidence_contract.py @@ -35,12 +35,16 @@ def test_requested_extension_isolation_is_metadata_not_success_evidence(self) -> "trial_number": 1, "passed": True, "pre_action_baseline_verified": True, + "clear_value_verified": True, + "input_value_verified": True, "pre_click_baseline_verified": True, "post_condition": True, "input_echo_verified": True, "url_unchanged": True, "input_semantics_verified": True, "submit_semantics_verified": True, + "presentation_applied": True, + "presentation_cleanup_verified": True, "profile_cleaned": True, } diff --git a/tests/test_agent_task_pinned_chrome_contract.py b/tests/test_agent_task_pinned_chrome_contract.py index 24c48776e..898698ed3 100644 --- a/tests/test_agent_task_pinned_chrome_contract.py +++ b/tests/test_agent_task_pinned_chrome_contract.py @@ -56,6 +56,38 @@ def test_agent_task_pass_uses_real_webdriver_input_and_post_condition(self) -> N with self.subTest(expected=expected): self.assertIn(expected, runner) + def test_presentation_probe_uses_fixed_cdp_commands_and_dom_observation(self) -> None: + """The Chromium probe must apply and reset fixed overrides without page-supplied code.""" + + namespace = runpy.run_path(str(RUNNER), run_name="presentation_probe_contract") + for expected in ( + "PRESENTATION_VIEWPORT_WIDTH", + "PRESENTATION_VIEWPORT_HEIGHT", + "PRESENTATION_DEVICE_PIXEL_RATIO", + "PRESENTATION_TIMEZONE", + "_apply_presentation_probe", + "_reset_presentation_probe", + "_read_presentation_probe", + ): + with self.subTest(expected=expected): + self.assertIn(expected, namespace) + + runner = RUNNER.read_text(encoding="utf-8") + self.assertIn('"Emulation.setDeviceMetricsOverride"', runner) + self.assertIn('"Emulation.setTimezoneOverride"', runner) + self.assertIn('"Emulation.clearDeviceMetricsOverride"', runner) + self.assertIn('"#presentation-viewport"', runner) + self.assertNotIn('"/execute/sync"', inspect.getsource(namespace["_read_presentation_probe"])) + + fixture = FIXTURE.read_text(encoding="utf-8") + for expected in ( + 'id="presentation-viewport"', + 'id="presentation-device-pixel-ratio"', + 'id="presentation-timezone"', + ): + with self.subTest(expected=expected): + self.assertIn(expected, fixture) + def test_agent_task_state_failure_does_not_echo_page_controlled_value(self) -> None: """A hostile DOM state must not become an exception or CI diagnostic payload.""" @@ -174,6 +206,7 @@ def failed_session_start( evidence = json.loads(output.getvalue()) failed_trial = evidence["agent_task"]["trial_results"][0] self.assertEqual(failed_trial["failure_type"], "AgentTaskSessionStartError") + self.assertEqual(failed_trial["failure_cause_type"], "RuntimeError") self.assertNotIn("host-controlled browser detail", output.getvalue()) def test_unexpected_cleanup_programming_failure_is_not_normalized(self) -> None: @@ -227,6 +260,8 @@ def test_agent_task_surface_completeness_is_non_vacuous(self) -> None: "input_semantics_verified": True, "submit_semantics_verified": True, "extensions_disabled_requested": True, + "presentation_applied": True, + "presentation_cleanup_verified": True, "profile_cleaned": True, } ] @@ -334,6 +369,8 @@ def successful_agent_task_trial(*_args: object, **_kwargs: object) -> dict[str, "input_semantics_verified": True, "submit_semantics_verified": True, "extensions_disabled_requested": True, + "presentation_applied": True, + "presentation_cleanup_verified": True, "profile_cleaned": True, } diff --git a/tests/test_fixture_server_evidence_publication_contract.py b/tests/test_fixture_server_evidence_publication_contract.py index f0056e0b6..fea3a7c54 100644 --- a/tests/test_fixture_server_evidence_publication_contract.py +++ b/tests/test_fixture_server_evidence_publication_contract.py @@ -73,12 +73,13 @@ def run_agent_task_trial( "passed": not fail_trials, } - namespace["_start_fixture_server"] = start_fixture_server - namespace["_stop_fixture_server"] = stop_fixture_server - namespace["_run_restart_trial"] = run_restart_trial - namespace["_run_agent_task_trial"] = run_agent_task_trial - namespace["_agent_task_surfaces_complete"] = lambda _trials: not fail_trials - namespace["print"] = lambda *_args, **_kwargs: events.append("evidence") + main_globals = namespace["main"].__globals__ + main_globals["_start_fixture_server"] = start_fixture_server + main_globals["_stop_fixture_server"] = stop_fixture_server + main_globals["_run_restart_trial"] = run_restart_trial + main_globals["_run_agent_task_trial"] = run_agent_task_trial + main_globals["_agent_task_surfaces_complete"] = lambda _trials: not fail_trials + main_globals["print"] = lambda *_args, **_kwargs: events.append("evidence") error: Exception | None = None with tempfile.TemporaryDirectory() as temp_dir: diff --git a/tests/test_mv3_page_diagnostic_redaction_contract.py b/tests/test_mv3_page_diagnostic_redaction_contract.py index 5be6e8663..2b461ac05 100644 --- a/tests/test_mv3_page_diagnostic_redaction_contract.py +++ b/tests/test_mv3_page_diagnostic_redaction_contract.py @@ -260,6 +260,76 @@ def close(self) -> None: self.assertNotIn("javascript error", str(captured.exception)) self.assertNotIn(HOSTILE_PAGE_VALUE, str(captured.exception)) + def test_session_creation_failure_has_a_bounded_type(self) -> None: + """A session-not-created response keeps its category without remote text.""" + + namespace = runpy.run_path(str(RUNNER), run_name="webdriver_session_start_contract") + json_request = namespace["_json_request"] + + class FakeResponse: + status = 200 + + def read(self, _limit: int) -> bytes: + return ( + '{"value":{"error":"session not created","message":"' + + HOSTILE_PAGE_VALUE + + '"}}' + ).encode() + + class FakeConnection: + def request(self, *_args: object, **_kwargs: object) -> None: + return None + + def getresponse(self) -> FakeResponse: + return FakeResponse() + + def close(self) -> None: + return None + + http_client = json_request.__globals__["http"].client + with patch.object(http_client, "HTTPConnection", return_value=FakeConnection()), self.assertRaises( + RuntimeError + ) as captured: + json_request(9515, "POST", "/session", {}) + + self.assertEqual(type(captured.exception).__name__, "WebDriverSessionNotCreatedError") + self.assertNotIn(HOSTILE_PAGE_VALUE, str(captured.exception)) + + def test_http_session_creation_failure_has_a_bounded_type(self) -> None: + """A W3C HTTP 500 session failure retains only its standard category.""" + + namespace = runpy.run_path(str(RUNNER), run_name="webdriver_http_session_start_contract") + json_request = namespace["_json_request"] + + class FakeResponse: + status = 500 + + def read(self, _limit: int) -> bytes: + return ( + '{"value":{"error":"session not created","message":"' + + HOSTILE_PAGE_VALUE + + '"}}' + ).encode() + + class FakeConnection: + def request(self, *_args: object, **_kwargs: object) -> None: + return None + + def getresponse(self) -> FakeResponse: + return FakeResponse() + + def close(self) -> None: + return None + + http_client = json_request.__globals__["http"].client + with patch.object(http_client, "HTTPConnection", return_value=FakeConnection()), self.assertRaises( + RuntimeError + ) as captured: + json_request(9515, "POST", "/session", {}) + + self.assertEqual(type(captured.exception).__name__, "WebDriverSessionNotCreatedError") + self.assertNotIn(HOSTILE_PAGE_VALUE, str(captured.exception)) + def test_driver_readiness_timeout_does_not_echo_last_exception(self) -> None: """Startup timeout must not serialize the last remote diagnostic into CI text.""" diff --git a/tests/test_presentation_probe_baseline_transition_contract.py b/tests/test_presentation_probe_baseline_transition_contract.py new file mode 100644 index 000000000..e9efd4174 --- /dev/null +++ b/tests/test_presentation_probe_baseline_transition_contract.py @@ -0,0 +1,40 @@ +"""Contract for causal presentation-probe baseline evidence.""" + +from __future__ import annotations + +import pathlib +import runpy +import unittest + +ROOT = pathlib.Path(__file__).resolve().parents[1] +RUNNER = ROOT / "scripts" / "ci" / "run_mv3_compatibility.py" + + +class PresentationProbeBaselineTransitionContractTests(unittest.TestCase): + """Require every claimed presentation surface to transition from ambient state.""" + + def test_baseline_rejects_any_surface_already_matching_the_fixed_target(self) -> None: + """A matching ambient surface cannot prove that its override took effect.""" + + namespace = runpy.run_path(str(RUNNER), run_name="presentation_baseline_contract") + validate = namespace["_validate_presentation_probe_baseline"] + target = namespace["_presentation_probe_target"]() + ambient = { + "viewport": "800x600", + "device_pixel_ratio": "1", + "timezone": "UTC", + } + + validate(ambient) + for key, target_value in target.items(): + with self.subTest(surface=key), self.assertRaisesRegex( + RuntimeError, + r"^presentation probe baseline already matched target$", + ): + matching = dict(ambient) + matching[key] = target_value + validate(matching) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_presentation_probe_hidden_output_contract.py b/tests/test_presentation_probe_hidden_output_contract.py new file mode 100644 index 000000000..7950397ef --- /dev/null +++ b/tests/test_presentation_probe_hidden_output_contract.py @@ -0,0 +1,39 @@ +"""Contract for hidden presentation-probe observations in pinned Chromium.""" + +from __future__ import annotations + +import inspect +import pathlib +import runpy +import unittest + +ROOT = pathlib.Path(__file__).resolve().parents[1] +RUNNER = ROOT / "scripts" / "ci" / "run_mv3_compatibility.py" +FIXTURE = ROOT / "tests" / "fixtures" / "agent_task_basic" / "index.html" + + +class PresentationProbeHiddenOutputContractTests(unittest.TestCase): + """Keep hidden evidence values observable without script execution.""" + + def test_hidden_probe_outputs_use_non_rendered_webdriver_property_reads(self) -> None: + """Hidden fixture values must use textContent, not rendered Get Element Text.""" + + namespace = runpy.run_path(str(RUNNER), run_name="presentation_hidden_output_contract") + reader = inspect.getsource(namespace["_read_presentation_probe"]) + fixture = FIXTURE.read_text(encoding="utf-8") + + for element_id in ( + "presentation-viewport", + "presentation-device-pixel-ratio", + "presentation-timezone", + ): + with self.subTest(element_id=element_id): + self.assertIn(f'id="{element_id}" hidden', fixture) + + self.assertIn('"/property/textContent"', reader) + self.assertNotIn('"/text"', reader) + self.assertNotIn('"/execute/sync"', reader) + + +if __name__ == "__main__": + unittest.main()