feat(resources_servers): add interactive browser RL environment (local Playwright + remote CDP backends)

[waple0820]

Summary

Addresses #644 (integrate Browser Gym), which asks for BrowserGym-style interactive web-task environments — an agent navigating and acting on a live page — in the Gym ecosystem. This PR delivers one natively as interactive_browser: one isolated browser per rollout, driven by tool calls, scored by verify(). Existing web envs (google_search, browsecomp) only read pages; this one operates them.

Where the browser runs is a config choice. The remote backend runs it off the training node, so browser RAM does not scale on the GPU node.

Updated per review feedback. The environment is now vendor-neutral: InteractiveBrowserResourcesServer + LocalPlaywrightBackend + RemoteCDPBackend, with hosted browser services plugged in as session providers. The Lexmount integration ships as an example provider in providers/lexmount/, which holds its code, its config and its README — the environment directory contains no vendor-specific file outside providers/.

What it is

• A stateful Resources Server: browser tools (browser_navigate / browser_click / browser_type / browser_observe / browser_finish) + verify(), with one isolated browser context per session_id.
• Reuses the standard simple_agent harness — the model's function calls are dispatched by name to the tool endpoints; no custom harness needed.
• Two shipped backends behind a small BrowserBackend contract (open / goto / click / type / observe / current_url / text / close):

backend:	Browser runs	Notes
local_playwright (default)	A Chromium launched on this machine, one per rollout	No account, no quota, no network. Grades the bundled offline site/ tasks in CI.
remote_cdp	Anywhere reachable over CDP, supplied per rollout by a session provider	Keeps browsers off the training node.

• A session provider answers one question for remote_cdp — where is this rollout's browser, and how do I give it back?

class BrowserSessionProvider(Protocol):
    name: str
    async def acquire(self, spec: BrowserSessionSpec) -> BrowserSessionHandle: ...   # a live cdp_url
    async def release(self, handle: BrowserSessionHandle) -> None: ...               # idempotent

Provider	What it does
static_cdp (built in)	Uses an endpoint that already exists — chromium --remote-debugging-port=9222, a browser container, another host. No third-party dependency, and what CI tests remote_cdp against.
lexmount (example)	Creates one isolated cloud browser session per rollout and deletes it afterwards. SDK is operator-installed and optional; the environment itself depends on playwright and nothing else.

Selection mirrors nemo_gym.sandbox: single-key {name: {kwargs}} mappings, providers imported only when selected, and out-of-tree providers publishable through the nemo_gym.browser_session_providers entry-point group (no fork needed).

Key result — browser off-node

Browser RAM on the training node, measured (PSS), by concurrency model:

Backend	node RAM @ 1000 concurrent rollouts
Local Chromium, one process per rollout	~40–50 GB
Local Chromium, one browser + N contexts	~16 GB
Remote over CDP (measured with the lexmount provider)	~0.14 GB — flat in concurrency

The browser engine runs off the node, so training-node RAM stays flat as concurrency grows and is free for vLLM / the model.

(Measured on a single host with a simple-page workload, characterizing each backend's browser engine — PSS of the chrome* process tree for the local backend vs node-side client PSS for the remote one.)

Session lifetime

A browser is released when the rollout is scored (verify) or when the same session_id is re-seeded. There is no independent episode TTL and no client-side cap on concurrent sessions, so a metered provider needs its account quota sized above the rollout concurrency (documented in the env README and the provider README).

Every acquired provider session is released exactly once — including when the CDP connect fails after the session was created, and however often close() is called. Both are covered by tests.

Integration path

• A (this PR): native SimpleResourcesServer with custom tool endpoints; upgradeable to MCPResourcesServer (@gym_tool on /mcp) if an MCP-native harness is preferred.
• B (follow-up): wrap the environment as an OpenEnv Environment so it runs through the existing openenv adapter (OpenEnvResourcesServer, which discovers tools at startup via ListToolsAction). This is the config-selected, backend-agnostic path for the third-party-environment integration tracked in [epic] 3rd Party Environments → Gym Integration #1701.
• Reward is settled once in verify() (not per-step accumulation), which sidesteps the TODO(ahmadki) cumulative-reward limitation in the OpenEnv adapter's step-wise accumulated_reward summation.

Status

• Environment (app.py, browser/, configs, offline test site, generate_data.py + example tasks, tests)
• Runs end-to-end via a gym env start + rollout (local backend, offline site): reward 1.0 on the navigate task
• One contract test suite run against both backends — remote_cdp against a Chromium the suite starts with --remote-debugging-port, so the remote path is covered in CI with no third-party service (22 tests, no GPU, no serving stack, no account)
• Off-node concurrency benchmark (per-backend browser-engine RAM)
• lexmount example provider implemented + smoke-validated on a live cloud session
• Remote path validated at training scale
• Pre-generated example_rollouts.jsonl
• GRPO training-signal run + reward curve on a web benchmark

Feedback welcome on the interface (A vs B), and on where you would like the example provider to live — it currently sits in-tree as a self-contained providers/lexmount/ (code + config + docs), mirroring how named sandbox providers ship under nemo_gym/sandbox/providers/. Loaded by path:

gym env start --config resources_servers/interactive_browser/providers/lexmount/configs/lexmount.yaml ...

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[cmunley1]

looks good! some comments

[cmunley1]

Should goto(initial_url) go in the try/except above? if navigation fails here close() never runs and browser process leaks, and on remote_cdp the provider session is never released. The test in test_registry_and_sessions.py only exercises connect-time failure.

[waple0820]

Done

[cmunley1]

The docstring at line 48 says every rollout gets its own context, but with with static_cdp they share context for all rollouts. So do cookies and storage carry across rollouts, and the page from open() is never closed (page.py:162 skips teardown when _owns_context is False), so tabs accumulate. new_context() might be better?

[waple0820]

Done.

Measured before fixing: it was not just tabs — a value written to localStorage by one
rollout was readable by the next, so an episode inherited the previous one's state. It survived because test_isolated_state_between_episodes did not test isolation: two tabs in one context also have different URLs. Added a test that writes storage in one episode and asserts the next cannot read it.

[cmunley1]

should the grader here be answer_equals ALPHA-42 instead

[waple0820]

Done

[cmunley1]

done isn't read by simple_agent. Its while True at app.py:106 exits only on incomplete_details (163), an assistant message with no tool calls (171), or max_steps (228). The flag does reach the model as text in the serialized tool output, so it works as a hint, but nothing enforces it. The model can finish and keep navigating, and _score reads the live page at verify, so a rollout that solves the task and then goes back to double-check scores 0.0.

Simplest change that stays inside this env - capture current_url() and the page text here on the first finish, store them on _SessionState, and have _score grade the snapshot, falling back to the live page only if the model never finished. The episode still runs to max_steps, but the reward stops depending on what happens after the model commits. Making the loop actually stop would mean implementing done in simple_agent which is used by many environments, or creating a new agent server.

[waple0820]

Done, implemented as you described — snapshot at the first finish, _score grades the snapshot, live page only if the model never finished. Agreed on leaving simple_agent alone.

[cmunley1]

You could set setting mask_sample alongside failure_reason here. Nothing in Gym reads failure_reason, so an unmeasurable rollout reaches training as a plain 0.0, indistinguishable from a policy that never solved the task, which is what the _ScoringUnavailable path exists to prevent. NeMo-RL's GRPO path reads instance_config["mask_sample"] off the verify result and zeroes loss_multiplier for flagged samples, so declaring a response subclass with instance_config: {"mask_sample": False} the way conversational_tool_use_simulation does (app.py:435) should exclude from gradient in training. Same at 260-264.

[waple0820]

You are right that nothing reads failure_reason today — it is human-readable only.

#2611 is exactly this gap: it puts mask_sample on BaseVerifyResponse, and after review there it is consumed centrally in compute_aggregate_metrics() rather than per environment. That is the version I would rather ship here, since the argument that PR makes is that this fact should stop being re-invented privately — and unlike swe_agents, where #2611 mirrors a field that already exists, adding one here would be creating a new one.

If #2611 stalls and this PR is otherwise ready, I will add the instance_config field to unblock, and drop it when the contract field lands. Happy to go either way.

[waple0820]

Actually, correcting myself — there is a path that works on main today, at least for eval. /verify is wrapped in judge_failsafe, so raising JudgeError puts the row in the failures sidecar and aggregation never sees it.

I didn't go that way though. It'd label a dead browser as judge_failed, which seems wrong while #2750 is trying to settle on failure names, and it only helps eval anyway — training reads the verify response directly and never goes through that path.

Which is really the case for #2611: right now the same fact needs _ng_failure_class for eval and instance_config["mask_sample"] for training, depending on who's reading it.