[celve]

Layer P17 of the CUA upstreaming stack: the Harbor computer-1 binding plus generic agent and verifier hooks, with no model classes. The runnable CUA example (tasks and a launcher) is deferred. It is built and reviewed in celve, and goes upstream with the whole stack once every layer is finished. Base: LIN-2522/session-images-on-4586 (#46, c1265b51a6, approved), which stacks on P15 (#33) and dzorlu's radixark/miles radixark#4586, which itself stacks on Kangrui Du's (Rockdu) radixark#4272. It depends on #46; #38 was dropped. Issue: LIN-2524, hub LIN-2504.

Problem. Harbor's computer-1 screenshot agent could not train through harbor_agent_function.run. It had no binding, so it got the default one. That one sets OPENAI_API_BASE/OPENAI_API_KEY env vars, which computer-1 never reads, so it went to OpenAI. It sent its own temperature (0.7), which the session server refuses for training, and none of Miles's other sampling arguments. It passed a max_seq_len that computer-1's constructor rejects. And there was no way to build an agent class other than Harbor's registered one, or to pass it settings the binding does not know about.

Fix.

• examples/experimental/harbor/harbor_agent_function.py gains a computer-1 binding (_computer_1_kwargs). It points LiteLLM at the session URL. Miles's temperature goes on the constructor. The other request arguments, plus the key, go in llm_kwargs, which LiteLLM sends on every call, so the compactor's and the final-answer fallback's calls carry them too. It turns images on explicitly (provider=litellm, enable_images=True) and leaves out max_seq_len (HarnessBinding.accepts_max_seq_len=False). HARBOR_AGENT_MAX_ITERATIONS sets the turn cap.
• build_trial_config: HARBOR_AGENT_IMPORT_PATH names the agent class Harbor builds, and the config then sets name=None, because Harbor builds the registered class whenever the name is a registered one. HARBOR_AGENT_KWARGS adds JSON kwargs and refuses any key the binding sets. HARBOR_VERIFIER_IMPORT_PATH/HARBOR_VERIFIER_KWARGS run a Harbor verifier class in place of a task's tests/. No model is named anywhere: a model's needs, such as its coordinate convention or the desktop size, go through these hooks as configuration.
• launch_common.agentic_train_args(session_images=True) emits --use-session-server v2 --multimodal-token-space-processing, the flag that carries media through the session server in feat(session): screenshots through the TITO session server on token space (stacked on #4272) radixark/miles#4586. v2 is selected because Check token-space image turns elementwise, compare media in the retry picker, keep session media under the run #46's image-aware retry picker is a v2 picker. harbor_env_vars forwards the four new env vars. The README says how a Qwen3-VL computer-1 run is wired.

Stopgap for Qwen3-VL's 0–999 clicks (second commit, Linyu: "I prefer the monkey patch first"). Harbor's generic JSON computer provider always prompts for desktop pixels and marks its actions NATIVE_PRESCALED. Harbor already rescales actions marked CoordinateSpace.NORMALIZED_0_999 (normalize_completion_action, which its Gemini provider relies on), but the generic provider has no option to choose that space. With HARBOR_CUA_COORDINATE_SPACE=normalized_0_999, harbor_agent_function.apply_coordinate_space() patches GenericJsonProvider.from_agent to prompt for a 1000×1000 screen and GenericJsonProvider.parse to mark each action on the grid. Harbor's own runtime then rescales the clicks to the desktop. The patch is applied once per process, on the first run(), because Harbor is imported lazily there. Unset, nothing is patched; any other value stops the run. It names no model and has no per-model branch. launch_common forwards the variable. This patch is a stopgap. A Harbor PR adding a coordinate_space option to the generic provider, on harbor-framework's harbor-miles-v0.20.0, would replace it; that PR is proposed and not started. The patch touches only public Harbor names (GenericJsonProvider.from_agent/parse, ModelStep.action.source, CoordinateSpace), but it is process-wide: every generic-provider agent in that process gets the grid.

Rework (Linyu: "rework 17"). 546b864717 drops HARBOR_CUA_DESKTOP_WIDTH/HEIGHT (the desktop size now goes in HARBOR_AGENT_KWARGS), names no model class in the code, tests or README, and makes the pass-through test cover coordinate_space. After it, the binding commit is +351/−6 over #46. With the stopgap, the layer is +441/−6, of which source is +114/−6. The stopgap commit alone is +93/−3 (source +43/−3).

Tests (EC2 CPU box, Miles venv; the contract tests use harbor-miles-v0.20.0 installed into a private target dir):

• pytest tests/fast/examples/experimental/harbor: 59 passed, 1 skipped (the contract module, with no Harbor in the venv). With examples/experimental/harbor reverted to the base, 21 of the new tests fail, including test_run_hands_computer_1_the_session_and_miles_sampling, which drives run().
• pytest tests/fast/examples/experimental/harbor/test_harbor_contract.py, with real Harbor: 24 passed. Two new tests drive Harbor's own AgentFactory and Computer1. One builds the class from the import path with max_seq_len in the metadata. The other runs the agent loop, both compactor summaries and the text fallback against a local chat server, and checks that every request goes to the session path with the dummy key and Miles's sampling arguments. It also checks that the screenshot request carries exactly one image that the session server's collect_media_refs (feat(session): screenshots through the TITO session server on token space (stacked on #4272) radixark/miles#4586) accepts. With the fix reverted, both fail.
• pre-commit run on the changed files: passed.
• Rebased onto Check token-space image turns elementwise, compare media in the retry picker, keep session media under the run #46 e1d0f02e5a, with no conflicts; the head is 4af2ea3da1. The rebase changed the session flags as above, and the contract test now checks the screenshot with collect_media_refs, because Carry images through the v2 session server on SGLang's token-space path #38's intern_images is gone. On it: pytest tests/fast/examples/experimental/harbor 59 passed, 1 skipped; the real-Harbor contract module 24 passed; pre-commit passed.
• Rebased again onto Check token-space image turns elementwise, compare media in the retry picker, keep session media under the run #46 b9fdaed38c, which added Check token-space image turns elementwise, compare media in the retry picker, keep session media under the run #46's four simplification commits; no conflicts, and P17's own diff is unchanged. The head is f5792150a1. On it, pytest tests/fast/examples/experimental/harbor (including the real-Harbor contract module) gives 83 passed.
• Rebased onto Check token-space image turns elementwise, compare media in the retry picker, keep session media under the run #46 c1265b51a6, which adds the startup refusal without --use-miles-router. The head was 0dd7bf90dd, with no conflicts.
• On the reworked head 546b864717 (fresh EC2 CPU box): pytest tests/fast/examples/experimental/harbor, including the real-Harbor contract module, gives 83 passed. With examples/experimental/harbor reverted to Check token-space image turns elementwise, compare media in the retry picker, keep session media under the run #46's head, 5 pass-through tests fail, all among the 21 above: test_agent_kwargs_join_the_bindings_own (it sets coordinate_space, desktop_width and desktop_height in HARBOR_AGENT_KWARGS and checks they reach AgentConfig.kwargs), the three test_agent_kwargs_cannot_replace_what_the_binding_sets cases, and test_agent_and_verifier_classes_are_forwarded_with_their_kwargs. pre-commit passes.
• Stopgap, on 6f19707331 (fresh EC2 CPU box, real Harbor harbor-miles-v0.20.0): pytest tests/fast/examples/experimental/harbor gives 86 passed. Three new contract tests run on a 1920×1080 computer-1 built through Harbor's AgentFactory, and parse a click through Harbor's real provider and normalize_completion_action. With the variable set, the prompt says 1000×1000 and (999, 500) lands on pixel (1919, 540). Unset, the prompt says 1920×1080 and (999, 500) stays (999, 500). An unknown value raises. With harbor_agent_function.py reverted to the binding commit, all three fail. pre-commit passes.

Measured. This is the binding LIN-2399's Harbor learning run used: held-out reward went from 0.160 to 0.273 by round 24. It was verified live on Megatron in LIN-2151.

Limits.

• No SandboxLost discard. The integration branch also discards a computer-1 trial whose own in-sandbox command fails after the agent has started (RuntimeRequestError) as InfraAbort("SandboxLost"). That depends on P9's InfraAbort contract (Discard agentic episodes that infrastructure failures broke #34, based on main), which is not in this base. It is added once P9 and this layer sit in one stack.
• Qwen3-VL's clicks rely on the stopgap patch above. Qwen3-VL clicks on a 0–999 grid, while computer-1's generic JSON provider prompts for pixels and stamps actions NATIVE_PRESCALED. Harbor already rescales CoordinateSpace.NORMALIZED_0_999 actions (its Gemini provider uses it), but the generic provider has no option to choose it. A ~15-line Harbor PR adding a coordinate_space kwarg is proposed, which recipes would then set with HARBOR_AGENT_KWARGS={"coordinate_space": "normalized_0_999"}. Until that exists, a Qwen3-VL run sets HARBOR_CUA_COORDINATE_SPACE=normalized_0_999; our fork's Qwen3VLComputer1 subclass is no longer needed for it. Running on an image's own desktop works the same way, with the fork's DesktopQwen3VLComputer1. The CUA example tasks and launcher are deferred.
• session_images=True emits only the session-server flags. The launcher still passes --sglang-enable-token-space-processor (P15 refuses token space without it), --shared-media-dir, and --use-miles-router. That last flag is needed because the default sgl-router drops mm_token_expansion_start_len until feat(protocol): forward mm_token_expansion_start_len on chat radixark/sgl-router-for-miles#28 lands; Check token-space image turns elementwise, compare media in the retry picker, keep session media under the run #46 refuses a run without it at startup.

Related upstream work. radixark/miles radixark#2806 (Tao Lin, nblintao, merged) is the in-process Harbor agent function that this layer extends with a binding for one more harness. No open upstream PR binds computer-1.

Origin commits: #8 (a007d0c78d) and #9 (09564ac8e2) on LIN-1985/investigate-existing-recipes-for, examples/experimental/harbor/* and tests/fast/examples/experimental/harbor/* only.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-11-01T11:44:08.224727Z	4b47409	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.