Skip to content

chore: isolate make validate and tests from agent-server env (#75) - #76

Merged
geeen-jp merged 1 commit into
mainfrom
issue-75
Oct 8, 2026
Merged

geeen-jp merged 1 commit into
mainfrom
issue-75

Conversation

@geeen-jp

@geeen-jp geeen-jp commented Oct 8, 2026

Copy link
Copy Markdown
Owner

AGENT:
Implemented through the multi-agent pipeline (Claude Sonnet 5.5 medium design/review/triage, OpenAI GPT-6 Luna max implementation, OpenAI GPT-6.1 Sol high audit). On this host, with dummy sentinel files only: make validate rc 0 in 15m59s (dummy auth/secrets/config/persistence checksums unchanged, sentinel absent from the log, git status --porcelain empty); the same make validate from a fresh clone with no .venv rc 0 in 16m02s (under 31 min, porcelain empty, dummy checksums unchanged). sdk_validate with CLAUDE_CODE_OAUTH_TOKEN and CLAUDE_AUTH_CODE unset: 2550 passed, 13 deselected.

Why

Builders run make validate inside an agent-server shell. That shell exports OPENHANDS_CODEX_AUTH_SOURCE, OPENHANDS_AGENT_SERVER_CONFIG_PATH, OH_PERSISTENCE_DIR, OH_SECRETS_DIR, CLOUD_AGENT_INJECTED_SECRET_NAMES, and sometimes OH_INTERNAL_SERVER_URL. Tests then read the host Codex auth file and write fixture records into the real secrets store. make validate previously unset only the two Claude OAuth variables, and a direct uv run pytest unset none of them.

Summary

  • make validate unsets those agent-server variables together with CLAUDE_CODE_OAUTH_TOKEN and CLAUDE_AUTH_CODE, via VALIDATE_UNSET_ENV. The suites, pyright, pre-commit, diff checks, and the Actions job name validate are unchanged.
  • tests/conftest.py strips the six agent-server variables in pytest_configure, before collection and before xdist workers start, so a direct pytest run does not inherit them. Tests that need a value still set it with monkeypatch.
  • tests/sdk/test_env_isolation.py checks the strip helper, a subprocess pytest that inherits dummy values, and the make -n validate unset list.

Issue Number

Closes #75

How to Test

Export the six variables at temp dummy files (codex auth.json, secrets secrets.json, a persistence dir, a config file, OH_INTERNAL_SERVER_URL=http://127.0.0.1:9) plus fake CLAUDE_CODE_OAUTH_TOKEN and CLAUDE_AUTH_CODE. Then:

make validate

On this host that was rc 0 in 15m59s. sha256 of the dummy files matched before and after, the sentinel string was absent from the log, and porcelain was empty. A fresh clone of this branch with no .venv, same dummy env, finished in 16m02s with porcelain empty.

Direct pytest of tests/agent_server and tests/sdk in that dummy env left the dummy files unchanged. sdk_validate with the two OAuth variables unset passed (2550 passed, 13 deselected).

Type

  • Bug fix
  • Feature
  • Refactor
  • Docs / chore

Notes

~/.openhands as a whole is not a stable snapshot on this box while the agent server is running: the diffs were settings fields the live server rewrites and extension/skill git caches. Host secrets.json and profile directories were not among the changed paths. Dummy files were never pointed at /home/box/.codex or the runtime state dir.

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

📁 PR Artifacts Notice

This PR contains a .pr/ directory with temporary PR-specific documents. The directory will be automatically removed when the PR is approved.

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

Python API breakage checks — ✅ PASSED

Result: ✅ PASSED

Action log

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

Endpoint audit

⚠️ 18 actionable Agent Server contract divergence(s) · report-only

Contract: pinned release artifact

Category Count
Actionable client-only calls 0
Actionable server-only operations 18
Documented non-divergences 13
Agent Server contract operations 129
Audited handwritten client endpoints 119

Actionable client-only calls (0)

none

Actionable server-only operations (18)

  • DELETE /api/canvas-extensions/installed/{}
  • DELETE /api/llm/provider-connections/{}
  • GET /api/canvas-extensions/installed
  • GET /api/canvas-extensions/installed/{}
  • GET /api/canvas-extensions/installed/{}/bundle
  • GET /api/conversations/{}/events
  • GET /api/file/archive
  • GET /api/git/commits
  • GET /api/git/commits/{}/changes
  • GET /api/init
  • GET /api/llm/provider-connections
  • PATCH /api/canvas-extensions/installed/{}
  • PATCH /api/llm/provider-connections/{}
  • POST /api/canvas-extensions/install
  • POST /api/conversations/{}/load_plugin
  • POST /api/file/create_directory
  • POST /api/init
  • POST /api/llm/provider-connections
Documented non-divergences (13)

Client calls intentionally absent from the filtered contract (11)

  • GET /
  • GET /alive
  • GET /health
  • GET /ready
  • GET /server_info

Reason: Operational Agent Server endpoints intentionally excluded from the filtered public release artifact.
Owner: OpenHands runtime maintainers

  • DELETE /api/meta-profiles/{}
  • GET /api/meta-profiles
  • GET /api/meta-profiles/{}
  • POST /api/meta-profiles/{}
  • POST /api/meta-profiles/{}/activate

Reason: Client-ahead API stacked on the pending Agent Server meta-profiles implementation.
Owner: OpenHands SDK maintainers
Tracking: OpenHands#3744

  • POST /api/profiles/{}/validate

Reason: Client-ahead API stacked on the pending Agent Server pre-flight LLM validation endpoint.
Owner: OpenHands TypeScript client maintainers
Tracking: OpenHands#4422

Server operations covered by an exposed browser URL (2)

  • GET /api/conversations/{}/workspace
  • GET /api/conversations/{}/workspace/{}

Reason: RemoteWorkspace.startWorkspaceSession exposes these authenticated URLs for browser iframe and file requests; they are not HttpClient method calls.
Owner: OpenHands TypeScript client maintainers

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

REST API breakage checks (OpenAPI) — ✅ PASSED

Result: ✅ PASSED

Action log

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

Coverage

Coverage Report •
FileStmtsMissCoverMissing
TOTAL428461181672% 
report-only-changed-files is enabled. No files were changed during this commit :)

@geeen-jp
geeen-jp merged commit 78e2853 into main Oct 8, 2026
39 checks passed
@geeen-jp
geeen-jp deleted the issue-75 branch October 8, 2026 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Chore: isolate make validate and tests from the agent-server environment (OH_*/OPENHANDS_* env)

1 participant