feat(world): add browser automation runtime - #12752
Conversation
b0a195e to
3ef423f
Compare
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Incremental review of Files Reviewed (0 files this round)
Previous Review Summaries (9 snapshots, latest commit fa916c0)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit fa916c0)Status: No Issues Found | Recommendation: Merge Incremental review of Files Reviewed (0 files this round)
Previous review (commit 8ceefc9)Status: No Issues Found | Recommendation: Merge Incremental review of
The new fingerprint test exercises the real implementation against a synthetic monorepo fixture (no mocks) and asserts both directions: unrelated lock changes keep the fingerprint stable, playwright bumps invalidate it. Files Reviewed (3 changed files this round)
Assumptions: Reviewed statically in read-only mode — no install, build, or test run. The Previous review (commit 6634812)Status: 2 Issues Found | Recommendation: Address before merge Overview
Incremental review of Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (4 changed files this round)
Assumptions: Reviewed statically in read-only mode — no install, build, test run, or browser launch. The bun.lock text-parsing assumptions in Fix these issues in Kilo Cloud Previous review (commit b17830b)Status: 3 Issues Found | Recommendation: Address before merge Overview
Incremental review of Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (4 changed files this round)
Assumptions: Reviewed statically in read-only mode — no install, build, test run, or browser launch. The catalog resolution path was verified against the root Fix these issues in Kilo Cloud Previous review (commit 87df327)Status: No Issues Found | Recommendation: Merge OverviewIncremental review of Since no PR-authored line changed in this round, there is nothing new to review and no previously reported finding sits on a changed line. Existing inline comments from earlier rounds are unchanged and still apply where GitHub still anchors them. Files Reviewed (0 changed files this round)No PR-owned file changed between Assumptions: Reviewed statically in read-only mode — no install, build, test run, or browser launch. Equivalence of the two commits was established by a path-scoped Previous review (commit c746b6e)Status: No Issues Found | Recommendation: Merge OverviewIncremental review of Previously reported finding that is now resolved:
No new issues in the changed lines. Inline comments previously posted on files that were not touched in this round are unchanged and still apply. Files Reviewed (1 changed file this round)
Assumptions: Reviewed statically in read-only mode — no install, build, test run, or browser launch. Previous review (commit 1197d81)Status: 6 Issues Found | Recommendation: Address before merge Overview
OverviewIncremental review of What's left is mostly the edges of those new mechanisms: the timeout still only requests termination (no Previously reported findings that are now resolved:
Still open from the previous round on a changed file:
Other open inline comments on files not touched this round are unchanged. Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (7 changed files this round)
Assumptions: Reviewed statically in read-only mode — no install, build, test run, or browser launch. Fix these issues in Kilo Cloud Previous review (commit 0246c18)Status: 9 Issues Found | Recommendation: Address before merge Overview
OverviewIncremental review of The remaining findings cluster around the promote-and-stamp sequence (the stamp lands before the binary it describes, so a mid-rename kill can mark a stale daemon current), the lock's stale detection (an unwritten lock reads as pid Previous findings on changed lines:
Other open inline comments on files not touched this round are unchanged. Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (6 changed files this round)
Assumptions: Reviewed statically in read-only mode — no install, build, test run, or browser launch, so the lock/stamp races, the orphaned smoke daemon, and the unhandled-rejection path are reasoned from Fix these issues in Kilo Cloud Previous review (commit 1349e4d)Status: 4 Issues Found | Recommendation: Address before merge Overview
OverviewIncremental review of The new lazy-build path is where the new findings sit: it inherits the host process's stdout/stderr, has no cross-process locking against Previous findings on changed lines:
Other open inline comments on files not touched this round are unchanged. Issue Details (click to expand)WARNING
SUGGESTION
Files Reviewed (5 changed files this round)
[Snapshot truncated.] Additional previous summary content was truncated to keep this comment within platform limits. Reviewed by kimi-k3 · Input: 33.5K · Output: 3.9K · Cached: 235.4K Review guidance: REVIEW.md from base branch |
3ef423f to
e9292d9
Compare
e9292d9 to
b87da72
Compare
b87da72 to
4954a30
Compare
4954a30 to
0b4b8d5
Compare
0b4b8d5 to
49a9e8a
Compare
49a9e8a to
7787e8a
Compare
7787e8a to
1349e4d
Compare
1349e4d to
0246c18
Compare
0246c18 to
1197d81
Compare
c746b6e to
87df327
Compare
|
Hey! This PR is finally done. I know it is quite large, and hard to test by itself, so if you have any questions, feel free to ask me. I am fully open to any changes. Note that this PR only adds the World implementation, adding it to the CLI is covered in #12753 and adding it to VS Code is covered in #12754. I hope to add JetBrains support soon, and I'll work on Computer Use as well (Windows and Linux only at first). I might also be able to share benchmarks and comparisons of with and without World. |
87df327 to
b17830b
Compare
b17830b to
6634812
Compare
8ceefc9 to
fa916c0
Compare
fa916c0 to
3da9630
Compare
|
No issues found, check are all green! |
BenchmarksTransparency first: the code I used to benchmark is available at https://gist.github.com/IamCoder18/ad7c646cd43326325bd0316b3e052954. Prompt: Create a dashboard with lots of charts and graphs populated with sample data. Add a button on the page that toggles all the graphs into a 3D space where the user can rotate the view with the mouse, zoom in/out with the scroll wheel, and pan/move around the scene with click-and-drag. Constraint: do not use any chart/graph library — write the visualization code yourself (e.g. SVG, Canvas, or raw DOM). You may use a 3D library such as three.js to render the 3D view. Results
No Playwright, No World (MiniMax M3, $0.0368, ~3m13s)
With Playwright (MiniMax M3, $0.0357 (<- highest cost out of the runs), ~4m12s)
Note: Some 3D controls were inverted, rotating caused text selection on cards, and there were a few visibility and contrast issues. With World (MiniMax M3, $0.0305, ~3m36s)
Notes: Cards always face the camera is a great touch. Winner: WorldPrompt: Turn Results
No Playwright, No World (MiniMax M3, $0.0245, 2m32s)
With Playwright (MiniMax M3, $0.0669, 3m11s)
With World (MiniMax M3, $0.0331, 2m16s) (half the cost of Playwright, fastest run)
Winner: WorldThough this run was close, World wins because it has better visuals to show data flow (an explicit part of the prompt) and has nicer 3D control. ResultsBoth Playwright and World drastically improve agent performance, however, World improves the quality of agents' work faster and cheaper than Playwright. Through earlier (undocumented) testing, World has a larger improvement on free models than smarter models, and is a valuable asset to users running free models for webdev. |
|
Comment to prevent this PR from automatically closing due to 30 days of inactivity. |














Context
Kilo’s existing browser automation relies on an extension-managed Playwright MCP server. That makes browser use client-specific, adds a separate registration and lifecycle path, and requires an external tool before the agent can verify its own UI work.
This PR introduces a native browser runtime shared by every Kilo client. Persistent, isolated sessions let the agent perform multi-step browser work without rebuilding context on every call, while direct screenshots and a compact action grammar make verification faster and more reliable than the previous MCP integration.
Most importantly, this makes the agent more proactive and improves the quality of visual work: the eventual World tool will instruct the agent to always verify visual changes instead of treating browser validation as an optional follow-up. The runtime’s action, session, and daemon boundaries also lay the foundation for future computer-use capabilities beyond the browser.
Closes #12751
Review and merge order: this PR → #12753 → #12754.
Implementation
I, a human, have manually reviewed every change in this PR.
Add a new
@kilocode/worldpackage with a quote-aware browser script parser and commands for navigation, snapshots, interaction, JavaScript evaluation, tabs, cookies, and screenshots. Browser state lives in a persistent per-session Node daemon, with private authenticated handshakes and isolated processes so sessions cannot read or reconfigure one another.The runtime supports headless and headed execution, system Chrome detection with Chromium fallback, abortable actions, and platform-aware expansion of home and environment-variable paths before file access. The daemon is bundled as a companion artifact and has a Node smoke test so later clients can ship the same runtime beside the Kilo binary.
Screenshots / Video
N/A — this PR introduces the headless runtime and does not change a user interface.
How to Test
Manual/local verification
Reviewer test steps
packages/kilo-world, runbun run typecheck.bun test --timeout 30000.bun run build:daemonand confirmdist/world-daemon.cjsis produced.Checklist
Get in Touch
Discord: @IamCoder18