diff --git a/CHANGELOG.md b/CHANGELOG.md index 5fabc854d..213ceb189 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ All notable changes to OriginWeave are documented in this file. The format follo - Record truncated WebDriver trial responses as bounded failed-trial evidence instead of aborting the complete MV3 compatibility run. - Close the first pinned-Chrome fixture server when startup of the second server fails, and attempt both shutdowns when one cleanup fails, preventing partial compatibility runs from leaking loopback server threads. - The fixture-shutdown contract now exercises successful MV3 and Agent Task trial paths before asserting reverse-order server cleanup. +- The resource-evidence shutdown fixture now supplies complete semantic and measured resource surfaces before exercising reverse-order cleanup. - The fixture-shutdown success double now includes both browser-computed semantic verification surfaces, so cleanup failures cannot mask incomplete Agent Task evidence. ### Added - Corrected the 2026-08-26 product-gap snapshot with current #229 presentation-identity evidence, stacked-only #205 integration evidence, current base/head pairs, the 126-PR queue count, explicit root-versus-child merge ordering, and the active GitHub counted-approval gate. @@ -43,6 +44,7 @@ All notable changes to OriginWeave are documented in this file. The format follo - Cumulative interactive-first RAM, VRAM, batch, local-model, admission, pause, and compositor-pressure mitigation plans, including active-consumer reduction at exact hard limits. - Real pinned-Chrome WebDriver evidence for the controlled Agent Task fixture: the CI lane uses an isolated profile, disables extensions, types and submits synthetic text, observes the same-document post-condition, and proves profile cleanup; this does not claim a shipped OriginWeave browser adapter. - Active pinned-Chrome Agent Task evidence verifies browser-computed role/name for controlled input and submit targets before action; this remains test-harness semantic evidence and does not claim a product semantic observer or authority. +- Active pinned-Chrome Agent Task evidence records browser-process RSS, semantic-observation bytes, action latency, and task duration from bounded trusted adapter inputs; this remains test evidence and does not claim process-set attribution or product resource telemetry. - Universally value-redacted network evidence with explicit path, metadata, and provenance bounds; ambiguous path rejection; validated source URLs; lowercase SHA-256 identifiers; and verification state. - Versioned schema-bound extraction contracts with bounded identifiers and field counts, typed value/cardinality metadata, explicit duplicate-free reviewed source channels, fail-closed schema validation, and deterministic `Display`/`std::error::Error` contracts for public schema failures. - Rust 1.97.1 build contract, strict Clippy and rustdoc gates, and exact production function, line, region, and branch coverage enforcement. diff --git a/docs/DOCUMENTATION_FITNESS.md b/docs/DOCUMENTATION_FITNESS.md index e313077b0..f36d6c945 100644 --- a/docs/DOCUMENTATION_FITNESS.md +++ b/docs/DOCUMENTATION_FITNESS.md @@ -164,9 +164,9 @@ Active #65 supplies a deterministic synthetic local web fixture with a labelled ### 3.20 Bounded browser process-set resource evidence -Active #51→#66 establishes two distinct layers: #51 owns single explicitly supplied Linux PID sampling and the bounded telemetry value boundary; #66 owns bounded duplicate-safe aggregation/sampling over an exact caller-owned PID set. #66's exact current contract rejects empty, zero-PID, duplicate, oversized and overflow states and fails closed if any member cannot be sampled. +Active #51→#66 establishes two distinct layers: #51 owns single explicitly supplied Linux PID sampling and the bounded telemetry value boundary; #66 owns bounded duplicate-safe aggregation/sampling over an exact caller-owned PID set. #66's exact current contract rejects empty, zero-PID, duplicate, oversized and overflow states and fails closed if any member cannot be sampled. Active PR #72 records browser-process RSS, semantic-observation bytes, action latency, and task duration for the controlled pinned-Chrome fixture from bounded trusted adapter inputs. -**Resolution:** aggregate resource measurement must not silently undercount a known caller-owned process set, but process membership remains an external attribution responsibility. The implementation does not discover Chromium PIDs, prove process ancestry/task ownership, walk cgroups, sample GPU/VRAM or create a durable telemetry store. +**Resolution:** aggregate resource measurement must not silently undercount a known caller-owned process set, but process membership remains an external attribution responsibility. PR #72 is bounded resource evidence for test repeatability; it does not discover Chromium PIDs, prove process ancestry/task ownership, walk cgroups, sample GPU/VRAM or create a durable telemetry store, and does not turn the fixture into a product resource adapter. ## 4. Durable product decisions captured by the canonical graph diff --git a/docs/TEST_STRATEGY.md b/docs/TEST_STRATEGY.md index faf7f54b3..a2e898ff1 100644 --- a/docs/TEST_STRATEGY.md +++ b/docs/TEST_STRATEGY.md @@ -90,6 +90,12 @@ controlled input and submit target before the real WebDriver action. CSS remains a fixture-harness locator; this does not establish OriginWeave node authority, semantic provenance or policy dispatch. +Active PR #72 additionally records bounded browser-process RSS, +semantic-observation bytes, action latency and task duration for the same +controlled fixture. These are test-harness resource evidence from trusted +adapter inputs; they do not establish Chromium process-set attribution, +GPU/VRAM telemetry or a product resource adapter. + ### 3.5 Buyer acceptance Versioned task packs measure repeatable product outcomes rather than one lucky agent run. The benchmark artifact records browser build, OriginWeave version, model/provider/reasoning configuration, seed where supported, policy profile, hardware profile and source fixtures. diff --git a/docs/traceability/action-postcondition-evidence.md b/docs/traceability/action-postcondition-evidence.md index 602c3ebe8..0350a4c9a 100644 --- a/docs/traceability/action-postcondition-evidence.md +++ b/docs/traceability/action-postcondition-evidence.md @@ -71,6 +71,14 @@ PR #71 extends the pinned-Chrome fixture lane by reading WebDriver's browser-com This is bounded browser-computed evidence for a synthetic test target, not the OriginWeave semantic observation adapter. CSS locators remain test-harness selectors, and the lane does not create OriginWeave node handles, source-channel provenance, policy authority, or permission to execute page-advertised actions. +### PR #72 — bounded Agent Task resource evidence + +**Capability maturity:** `IMPLEMENTED_ON_ACTIVE_PR` + +PR #72 records browser-process RSS, semantic-observation bytes, action latency, and total task duration while the pinned-Chrome fixture runs. The measurements are bounded, positive observations from the trusted ChromeDriver process identifier and the controlled semantic payload; its shutdown regression double supplies every required semantic and resource surface before cleanup failures are exercised, so incomplete evidence cannot pass incidentally. This makes the real fixture's resource and timing evidence inspectable without introducing a new telemetry subsystem. + +This is resource evidence for the active test harness, not process-set attribution or a product resource adapter. It does not discover Chromium children, prove task ownership or ancestry, walk cgroups, sample GPU/VRAM, or export durable product telemetry. + ## 4. Non-transitive success semantics The intended first-slice chain is: @@ -96,7 +104,7 @@ VerifiedActionOutcomeEvidence type existence -/> proof of real Chromium executio controlled fixture success -/> proof of an OriginWeave product browser runtime ``` -PR #64 now rejects a caller-supplied observation timestamp that predates caller-supplied dispatch time, but the type cannot independently prove the clock source, that a real browser actually dispatched the action, that the supplied provenance belongs to the claimed browser target/node, or that the observed state was caused by that action. PR #70 proves real Chromium execution against the controlled fixture and PR #71 adds browser-computed role/name evidence, but their test-harness CSS locators and direct WebDriver calls are not the OriginWeave adapter/runtime composition required under issue #28. +PR #64 now rejects a caller-supplied observation timestamp that predates caller-supplied dispatch time, but the type cannot independently prove the clock source, that a real browser actually dispatched the action, that the supplied provenance belongs to the claimed browser target/node, or that the observed state was caused by that action. PR #70 proves real Chromium execution against the controlled fixture, PR #71 adds browser-computed role/name evidence, and PR #72 adds bounded resource evidence, but their test-harness CSS locators, direct WebDriver calls, and fixture-scoped measurements are not the OriginWeave adapter/runtime composition required under issue #28. ## 5. Active prerequisite graph for issue #28 @@ -111,9 +119,10 @@ The first real Chromium vertical slice remains distributed across bounded active - PR #64 — verified and caller-timestamp-ordered post-condition action-outcome evidence; and - PR #65 — controlled hostile local Agent Task workflow fixture; and - PR #70 — real WebDriver execution of that fixture on pinned Chrome, without claiming a product browser adapter; and -- PR #71 — browser-computed role/name evidence before controlled action, without claiming a product semantic observer. +- PR #71 — browser-computed role/name evidence before controlled action, without claiming a product semantic observer; and +- PR #72 — bounded browser-process RSS, semantic-observation byte, latency, and task-duration resource evidence, without claiming process-set attribution or a product resource adapter. -These active PRs are non-shipped evidence. PR #70/#71 prove bounded browser-level and semantic evidence, but the active set does not itself compose WebDriver BiDi/CDP transport, OriginWeave authority translation, trusted Chromium process attribution, policy-authorized real input dispatch, causal post-condition observation, or deterministic end-to-end teardown/recovery into one protected-main runtime. +These active PRs are non-shipped evidence. PR #70/#71/#72 prove bounded browser-level, semantic, and resource evidence, but the active set does not itself compose WebDriver BiDi/CDP transport, OriginWeave authority translation, trusted Chromium process attribution, policy-authorized real input dispatch, causal post-condition observation, or deterministic end-to-end teardown/recovery into one protected-main runtime. ## 6. Remaining issue #28 boundary diff --git a/scripts/ci/run_mv3_compatibility.py b/scripts/ci/run_mv3_compatibility.py index e49dd7357..9ea8bb765 100755 --- a/scripts/ci/run_mv3_compatibility.py +++ b/scripts/ci/run_mv3_compatibility.py @@ -9,8 +9,9 @@ restart-persistence behavior. It also executes the controlled Agent Task fixture with extensions disabled in a fresh profile, verifies browser-computed role/name for the controlled action targets, performs real WebDriver input and click -operations, verifies the observable post-condition, and proves profile cleanup -without treating page content as instruction or authority. +operations, verifies the observable post-condition, proves the controlled action +preserves its loaded URL, and records bounded runtime resource evidence without +treating page content as instruction or authority. """ from __future__ import annotations @@ -41,6 +42,8 @@ STARTUP_TIMEOUT_SECONDS = 20.0 FIXTURE_TIMEOUT_SECONDS = 20.0 MAX_WEBDRIVER_RESPONSE_BYTES = 1_048_576 +MAX_PROC_STATUS_CHARACTERS = 65_536 +MAX_U64 = (1 << 64) - 1 W3C_ELEMENT_KEY = "element-6066-11e4-a52e-4f735466cecf" PATH_TOKEN_CHARACTERS = frozenset(string.ascii_letters + string.digits + "-_.") @@ -222,6 +225,43 @@ def _get_element_semantics( return role, label +def _parse_linux_proc_status_rss_bytes(status_text: str) -> int: + """Parse exactly one positive Linux ``VmRSS`` kB field into bounded bytes.""" + + rss_values: list[int] = [] + for line in status_text.splitlines(): + if not line.startswith("VmRSS:"): + continue + fields = line.split() + if len(fields) != 3 or fields[0] != "VmRSS:" or fields[2] != "kB": + raise ValueError("malformed Linux VmRSS field") + raw_kibibytes = fields[1] + if not raw_kibibytes.isascii() or not raw_kibibytes.isdigit(): + raise ValueError("malformed Linux VmRSS value") + kibibytes = int(raw_kibibytes, 10) + if kibibytes <= 0: + raise ValueError("Linux VmRSS must be positive") + if kibibytes > MAX_U64 // 1024: + raise OverflowError("Linux VmRSS exceeds u64 byte range") + rss_values.append(kibibytes * 1024) + if len(rss_values) != 1: + raise ValueError("Linux proc status must contain exactly one VmRSS field") + return rss_values[0] + + +def _sample_linux_process_rss_bytes(process_id: int) -> int: + """Read one attributed Linux process RSS through a bounded ``/proc`` status file.""" + + if isinstance(process_id, bool) or not isinstance(process_id, int) or process_id <= 0: + raise ValueError("invalid Linux process identifier") + status_path = pathlib.Path("/proc") / str(process_id) / "status" + with status_path.open("r", encoding="utf-8", errors="strict") as status_file: + status_text = status_file.read(MAX_PROC_STATUS_CHARACTERS + 1) + if len(status_text) > MAX_PROC_STATUS_CHARACTERS: + raise RuntimeError("Linux proc status exceeded the bounded text limit") + return _parse_linux_proc_status_rss_bytes(status_text) + + def _cleanup_browser_session(driver_port: int, session_id: str) -> None: """Delete one WebDriver session through the fixed loopback authority.""" @@ -539,7 +579,7 @@ def _run_agent_task_browser_pass( fixture_url: str, profile_dir: str, ) -> dict[str, Any]: - """Execute one synthetic Agent Task through real WebDriver input in pinned Chrome.""" + """Execute one synthetic Agent Task and measure bounded real-browser evidence.""" started = time.monotonic() driver_port = _free_loopback_port() @@ -584,15 +624,22 @@ def _run_agent_task_browser_pass( capabilities = session.get("capabilities", {}) if not isinstance(raw_session_id, str): raise RuntimeError("ChromeDriver did not return an Agent Task session id") + if not isinstance(capabilities, dict): + raise RuntimeError("ChromeDriver Agent Task capabilities are malformed") session_id = _path_token(raw_session_id, "session identifier") - browser_version = ( - capabilities.get("browserVersion") if isinstance(capabilities, dict) else None - ) + browser_version = capabilities.get("browserVersion") + browser_process_id = capabilities.get("goog:processID") if browser_version != PINNED_CHROME_VERSION: raise RuntimeError( f"unexpected Agent Task Chrome version: expected {PINNED_CHROME_VERSION}, " f"got {browser_version!r}" ) + if ( + isinstance(browser_process_id, bool) + or not isinstance(browser_process_id, int) + or browser_process_id <= 0 + ): + raise RuntimeError("ChromeDriver did not return a valid browser process id") _json_request( driver_port, @@ -606,9 +653,8 @@ def _run_agent_task_browser_pass( _webdriver_path(session_id, "/url"), ).get("value") if initial_url != fixture_url: - raise RuntimeError( - f"Agent Task initial URL mismatch: expected {fixture_url!r}, got {initial_url!r}" - ) + raise RuntimeError("Agent Task did not load the requested fixture URL") + input_element = _find_element(driver_port, session_id, "#task-text") input_role, input_name = _get_element_semantics( driver_port, @@ -617,18 +663,6 @@ def _run_agent_task_browser_pass( ) if input_role != "textbox" or input_name != "Task text": raise RuntimeError("Agent Task input semantic evidence mismatch") - _json_request( - driver_port, - "POST", - _element_command_path(session_id, input_element, "/clear"), - {}, - ) - _json_request( - driver_port, - "POST", - _element_command_path(session_id, input_element, "/value"), - {"text": AGENT_TASK_INPUT_VALUE, "value": list(AGENT_TASK_INPUT_VALUE)}, - ) submit_element = _find_element( driver_port, session_id, @@ -641,12 +675,44 @@ def _run_agent_task_browser_pass( ) if submit_role != "button" or submit_name != "Submit task": raise RuntimeError("Agent Task submit semantic evidence mismatch") + semantic_observation = { + "input": {"role": input_role, "name": input_name}, + "submit": {"role": submit_role, "name": submit_name}, + } + semantic_observation_bytes = len( + json.dumps( + semantic_observation, + ensure_ascii=False, + separators=(",", ":"), + sort_keys=True, + ).encode("utf-8") + ) + if semantic_observation_bytes <= 0: + raise RuntimeError("Agent Task semantic observation was empty") + + action_started = time.monotonic() + _json_request( + driver_port, + "POST", + _element_command_path(session_id, input_element, "/clear"), + {}, + ) + _json_request( + driver_port, + "POST", + _element_command_path(session_id, input_element, "/value"), + {"text": AGENT_TASK_INPUT_VALUE, "value": list(AGENT_TASK_INPUT_VALUE)}, + ) _json_request( driver_port, "POST", _element_command_path(session_id, submit_element, "/click"), {}, ) + action_latency_ms = round((time.monotonic() - action_started) * 1000, 3) + if action_latency_ms <= 0: + raise RuntimeError("Agent Task measured a non-positive action latency") + post_submit_url = _json_request( driver_port, "GET", @@ -655,6 +721,7 @@ def _run_agent_task_browser_pass( url_unchanged = post_submit_url == initial_url if not url_unchanged: raise RuntimeError("Agent Task URL changed during submission") + result_element = _find_element(driver_port, session_id, "#task-result") state = _json_request( driver_port, @@ -669,6 +736,10 @@ def _run_agent_task_browser_pass( _validate_agent_task_submitted_state(state) if text != AGENT_TASK_INPUT_VALUE: raise RuntimeError("Agent Task result did not match the synthetic typed value") + browser_process_rss_bytes = _sample_linux_process_rss_bytes(browser_process_id) + task_duration_ms = round((time.monotonic() - started) * 1000, 3) + if task_duration_ms <= 0: + raise RuntimeError("Agent Task measured a non-positive task duration") return { "browser_version": browser_version, "post_condition": True, @@ -677,7 +748,11 @@ def _run_agent_task_browser_pass( "input_semantics_verified": True, "submit_semantics_verified": True, "extensions_disabled": True, - "duration_ms": round((time.monotonic() - started) * 1000), + "browser_process_rss_bytes": browser_process_rss_bytes, + "semantic_observation_bytes": semantic_observation_bytes, + "action_latency_ms": action_latency_ms, + "task_duration_ms": task_duration_ms, + "duration_ms": round(task_duration_ms), } finally: primary_error = sys.exc_info()[1] @@ -745,6 +820,10 @@ def _run_agent_task_trial( "input_semantics_verified": result["input_semantics_verified"], "submit_semantics_verified": result["submit_semantics_verified"], "extensions_disabled": result["extensions_disabled"], + "browser_process_rss_bytes": result["browser_process_rss_bytes"], + "semantic_observation_bytes": result["semantic_observation_bytes"], + "action_latency_ms": result["action_latency_ms"], + "task_duration_ms": result["task_duration_ms"], "profile_cleaned": profile_cleaned, "duration_ms": round((time.monotonic() - trial_started) * 1000), } @@ -764,6 +843,14 @@ def _agent_task_surfaces_complete(agent_task_trials: list[dict[str, Any]]) -> bo and trial.get("submit_semantics_verified") is True and trial.get("extensions_disabled") is True and trial.get("profile_cleaned") is True + and isinstance(trial.get("browser_process_rss_bytes"), int) + and trial["browser_process_rss_bytes"] > 0 + and isinstance(trial.get("semantic_observation_bytes"), int) + and trial["semantic_observation_bytes"] > 0 + and isinstance(trial.get("action_latency_ms"), (int, float)) + and trial["action_latency_ms"] > 0 + and isinstance(trial.get("task_duration_ms"), (int, float)) + and trial["task_duration_ms"] >= trial["action_latency_ms"] for trial in agent_task_trials ) diff --git a/tests/test_agent_task_pinned_chrome_contract.py b/tests/test_agent_task_pinned_chrome_contract.py index fd9dc397a..d1d726635 100644 --- a/tests/test_agent_task_pinned_chrome_contract.py +++ b/tests/test_agent_task_pinned_chrome_contract.py @@ -210,6 +210,10 @@ def test_agent_task_surface_completeness_is_non_vacuous(self) -> None: "submit_semantics_verified": True, "extensions_disabled": True, "profile_cleaned": True, + "browser_process_rss_bytes": 1, + "semantic_observation_bytes": 1, + "action_latency_ms": 1, + "task_duration_ms": 2, } ] ) @@ -310,6 +314,10 @@ def successful_agent_task_trial(*_args: object, **_kwargs: object) -> dict[str, "input_semantics_verified": True, "submit_semantics_verified": True, "extensions_disabled": True, + "browser_process_rss_bytes": 1, + "semantic_observation_bytes": 1, + "action_latency_ms": 1, + "task_duration_ms": 2, "profile_cleaned": True, } @@ -365,6 +373,45 @@ def test_agent_task_observes_computed_role_and_name_before_action(self) -> None: with self.subTest(expected=expected): self.assertIn(expected, runner) + def test_agent_task_records_real_bounded_resource_evidence(self) -> None: + """The real task must report measured browser/runtime resource evidence.""" + + namespace = runpy.run_path(str(RUNNER), run_name="agent_task_resource_contract") + runner = RUNNER.read_text(encoding="utf-8") + for expected in ( + "_parse_linux_proc_status_rss_bytes", + "_sample_linux_process_rss_bytes", + ): + with self.subTest(expected=expected): + self.assertIn(expected, namespace) + for expected in ( + '"goog:processID"', + '"browser_process_rss_bytes"', + '"semantic_observation_bytes"', + '"action_latency_ms"', + '"task_duration_ms"', + ): + with self.subTest(expected=expected): + self.assertIn(expected, runner) + + def test_linux_rss_parser_is_strict_and_overflow_safe(self) -> None: + """Runner-side RSS evidence must not accept ambiguous proc status input.""" + + namespace = runpy.run_path(str(RUNNER), run_name="agent_task_rss_contract") + parser = namespace["_parse_linux_proc_status_rss_bytes"] + self.assertEqual(parser("Name:\tchrome\nVmRSS:\t123 kB\n"), 123 * 1024) + for malformed in ( + "Name:\tchrome\n", + "VmRSS:\t0 kB\n", + "VmRSS:\t123 MB\n", + "VmRSS:\t123 kB extra\n", + "VmRSS:\t123 kB\nVmRSS:\t124 kB\n", + "VmRSS:\t18446744073709551616 kB\n", + ): + with self.subTest(malformed=malformed): + with self.assertRaises((ValueError, OverflowError)): + parser(malformed) + def test_agent_task_fixture_runs_under_the_existing_pinned_chrome_job(self) -> None: """No floating browser or second workflow may be introduced for this slice.""" @@ -396,6 +443,10 @@ def test_documentation_separates_active_browser_evidence_from_product_runtime(se self.assertIn("PR #71", traceability) self.assertIn("computed role/name", traceability) self.assertIn("computed role/name", fitness) + self.assertIn("browser-process RSS", changelog) + self.assertIn("PR #72", traceability) + self.assertIn("resource evidence", traceability) + self.assertIn("resource evidence", fitness) if __name__ == "__main__":