[opensource-elearning]

Fixes #12274

Summary

Replaces the full Chromium/Cloakbrowser browser engine with Obscura (a ~30MB Rust headless browser) as the primary engine for the shared browser pool and the Cloudflare Playground executor. Obscura speaks the Chrome DevTools Protocol, so Playwright's chromium.connectOverCDP drives it as a drop-in replacement — no API changes, no executor rewrites. Full Chromium remains as an automatic fallback when Obscura is not installed.

---

Problem

The current browser pool launches one full Chromium process (~150–400MB resident) per provider that needs browser-grade operations:

Provider path	Chromium launched	Memory cost
browserPool.ts → launchBrowser()	Shared Chromium, kept warm for 5 min	~150–400MB
cloudflare-playground.ts → PlaywrightCfTransport.start()	Fresh Chromium per session	~150–400MB each
Total (worst case)	2 Chromium processes	300–800MB

On a VPS with limited RAM, holding 2 Chromium processes alongside the Node.js server and database causes OOM kills and session terminations. This is the memory profile before this change.

---

Solution

New shared module: open-sse/services/obscura.ts

One obscura serve process is spawned lazily on first use and shared for the server's lifetime. Callers receive a fresh Playwright Browser (connected over CDP) on demand; closing the connection does not stop the shared server.

Lifecycle management:

• Spawns obscura serve --port <random> --host 127.0.0.1 once, then caches the process
• Readiness is checked by probing /json/version (Obscura never completes a response for a bare GET /)
• OBSCURA_CDP_ENDPOINT env var: use an already-running Obscura externally (the module does not own the process)
• OBSCURA_BIN env var: point to a non-standard binary location
• OMNIROUTE_BROWSER_POOL=off: disables Obscura entirely, same as disabling the pool
• killSharedObscuraServer(): terminates the shared process (process-lifetime cleanup)

browserPool.ts changes

launchBrowser() now tries Obscura first via connectObscuraBrowser(), then falls back to Cloakbrowser, then plain Chromium — the same fallback chain as before, just with Obscura at the front:

Obscura (~30MB, CDP) → Cloakbrowser → Plain Chromium (~150–400MB)

Other improvements in browserPool.ts:

• engine field tracked in PoolState and surfaced via getBrowserPoolStatus() / getBrowserPoolMetrics() so operators can see which engine is active
• pendingContexts map now stores { promise, createdAt } with a 5-minute TTL: stale pending entries (from failed context creations) are evicted, preventing silent leaks
• isStealth detection includes Obscura: state.engine === "obscura" || state.cloakLaunch !== null
• shutdownPool() does NOT kill the shared Obscura server — that is owned by obscura.ts and reused by executors

cloudflare-playground.ts changes

PlaywrightCfTransport.start() calls connectObscuraBrowser() first. If available, the executor attaches to the shared Obscura server. If not, it falls back to launching its own Chromium as before.

---

Installing Obscura

Obscura is an optional system binary — not an npm dependency. When absent, the browser pool falls back to the existing Chromium/Cloakbrowser path with no change in behavior. To get the memory savings, install the binary and place it on PATH.

Quick install (Linux x86_64):

curl -LO https://github.com/h4ckf0r0day/obscura/releases/latest/download/obscura-x86_64-linux.tar.gz
tar xzf obscura-x86_64-linux.tar.gz
mv obscura ~/.local/bin/
# Verify: obscura --version

All supported platforms (from the Obscura releases):

Platform	Archive
Linux x86_64	obscura-x86_64-linux.tar.gz
Linux ARM64	obscura-aarch64-linux.tar.gz
macOS Apple Silicon	obscura-aarch64-macos.tar.gz
macOS Intel	obscura-x86_64-macos.tar.gz

Alternative methods:

• Docker: docker run -d --name obscura -p 127.0.0.1:9222:9222 h4ckf0r0day/obscura
• Build from source: git clone https://github.com/h4ckf0r0day/obscura && cd obscura && cargo build --release -p obscura-cli --bins --no-default-features (requires Rust 1.75+)
• Arch Linux AUR: yay -S obscura-browser
• NixOS: nix-env -iA nixpkgs.obscura

No action required if you do not install Obscura — everything works the same as before.

---

Documentation update

docs/reference/ENVIRONMENT.md has been updated to document the three new env vars (OBSCURA_BIN, OBSCURA_CDP_ENDPOINT, OBSCURA_PORT) and the updated OMNIROUTE_BROWSER_POOL description noting Obscura as the primary engine.

---

Memory comparison (verified)

Metric	Before	After (Obscura installed)	After (Obscura absent)
Browser engine launched	Chromium (~150–400MB)	Obscura (~30MB)	Chromium (same as before)
Processes held	1–2 Chromium	1 Obscura (shared)	1–2 Chromium
Estimated peak memory	300–800MB	~30MB	300–800MB

The ~30MB figure is Obscura's reported resident memory (RSS) measured with ps after spawning obscura serve — verified on this deployment target (x86_64 Linux, AMD64).

---

Scope — what this deliberately does NOT change

• No API or provider-facing behavior changes. Every caller that previously got a Chromium Browser object still gets one. The only difference is which process backs it.
• No DuckDuckGo IP rotation. IP bans from DDG (the ERR_BN_LIMIT 418 responses) are a separate problem tracked in feat(providers): adopt Obscura as primary headless browser engine with Chromium/Chrome fallback #12274 and require a proxy rotation hook, not a browser engine change. That work is intentionally excluded from this PR.
• No changes to the browserPool API. acquireBrowserContext, releaseBrowserContext, getBrowserPoolStatus — all unchanged in shape.
• No changes to obscura serve command-line interface. The module uses the documented --port and --host flags only.

---

Test plan

Command	Result
node --import tsx/esm --test tests/unit/obscura-integration.test.ts (Obscura binary installed)	5/5 pass — env-toggle (2), null-binary path, round-trip page, external-endpoint
tsc -p tsconfig.json --noEmit	Clean (1 pre-existing error in unrelated test file)
node --import tsx/esm --test tests/unit/obscura-integration.test.ts (Obscura binary absent)	2 skipped (live tests), 3 pass (null path, env toggles)

What the round-trip test covers

Spawns the shared obscura serve, connects Playwright via chromium.connectOverCDP, creates a new context, navigates to https://example.com, asserts the page title is "Example Domain", then cleans up — confirming the full end-to-end path works without any Chromium process present.

---

Verification

• All Obscura integration tests pass (5/5)
• browserPool.ts typechecks cleanly
• obscura.ts typechecks cleanly
• cloudflare-playground.ts typechecks cleanly
• getBrowserPoolStatus() returns the new engine field
• No Co-Authored-By footer in commits
• No email addresses in the PR body
• killSharedObscuraServer() does not run on pool shutdown
• pendingContexts TTL eviction prevents silent stale-entry leaks