feat(extension): workflow params, honest dry runs, and actionable errors - #5099
Merged
Conversation
- Title and explainer on the card (save vs update intent is explicit) - Show the workflow description, the one human-readable field - Wrap script lines instead of clipping them at the card edge - Cap card height; script scrolls, approve/reject stay visible - Plain labels: Runs on / Starts at
Workflows declare params (name, description, example, required).
- Script contract: async ({ page, state, input }); input is re-injected
on every page, so runs never lose inputs across navigations
- run_workflow validates required params and returns an actionable
error naming each missing value with description and example
- Non-object input fails fast with the expected shape
- Params surface in the workflows index, search_workflows results,
and the approval card (Inputs section)
- Manual runs prompt for params with a small form; required values
gate the Run button
- page.waitFor(selector, timeoutMs) polls for dynamic content;
dry runs record the wait instead of polling
- Settings rows show the workflow description; empty state tells
users they can ask Kilo to create a workflow
Every workflow error now says what happened and what to do next: - Out-of-scope run: names the tab URL and the scope, suggests navigating or setting a startUrl - Unapproved script: says to re-save so the user can approve - Invalid script return: echoes the returned value and both valid shapes; navigate-without-state gets its own message - Bad tool arguments: zod field-level details instead of a bare "Invalid arguments" - Workflow not found: points at search_workflows - Safe-mode gate: names the exact toggle - Page limit: names the limit and the likely navigation loop search_workflows with a query now searches all sites (in-scope results first) and returns inScope/startUrl per result, so agents can find and run a workflow saved for another site.
Covers the manual run form (required-value gating), input reaching the script, page.waitFor bridging async results, and the missing-required-input error text an agent sees.
- Deleting a workflow now takes two clicks: the first arms a red confirm state (reset on blur), the second deletes - vitest include missed entrypoints/**/*.test.tsx, so the workflow-settings and conversation-events suites (25 tests) never ran; include them, add the jsdom docblock the events suite was missing, and register testing-library cleanup (globals are off, so RTL cannot self-register and renders leaked across tests) - Update the revived suites for the current copy and the delete confirmation
…fusion Live probes with kilo-auto/efficient surfaced two failure modes: - Scripts read results a dry run never renders (clicks are recorded, not performed) and fall through returning undefined. The invalid-value error now explains dry-run semantics and the fix. - Models write workflow page helpers (page.click) inside eval. The eval description now says helpers exist only in workflow scripts.
…scripts
A dry run records clicks and fills instead of performing them, so any
workflow that reads content its own actions produce hit a hard error
and looked broken. Live probes showed agents then re-saving and
re-running the same correct workflow in a loop.
- A selector miss after the first recorded action reports success with
the recorded actions and a note; a miss before any action stays a
real failure, so wrong selectors are still caught
- Same for a dry-run script that returns nothing after recording
actions; a real run returning nothing still fails
- run_workflow's description states this and says not to edit a
workflow because a dry run stopped there
- A relative startUrl ("/", "/search") resolves against the scope
instead of being rejected; the rejection message now shows the
expected form and the received value
A run result carried only pagesVisited and result, so the transcript header showed a raw tab id and the model had to remember which workflow it ran. Successful results now carry workflowName, and failures read: Workflow "X" failed: <reason>.
…liases The settings file was at the import and line caps, which forced two obscure type aliases (Awaited<ReturnType<...>>) in place of a plain type import. The row is self-contained, so it moves to its own file and owns its delete confirmation and param form. WorkflowRunPrompt now takes a name and params instead of a whole workflow.
Contributor
Author
Final validation on the shipped buildThree more live
The dry-run fix is the largest single improvement measured. Same create-a-workflow scenario, N=4, before and after:
Before the fix, agents read a dry-run stop as a broken workflow and re-saved the same correct script up to five times. |
Input is embedded in the injected page code on every page, but only navigation state was size-checked. Oversized or unserializable input now fails before any navigation, with the limit named.
![kilo-code-bot[bot]](https://avatars.githubusercontent.com/in/2193792?s=60&v=4){kind=small}
Contributor
Code Review SummaryStatus: No Issues Found | Recommendation: Merge Executive SummaryIncremental review of the new commit (2 files, 61 changed lines): the previously flagged empty-search message is fully resolved — Files Reviewed (2 files)
Previous Review Summary (commit 8dee466)Current summary above is authoritative. Previous snapshots are kept for context only. Previous review (commit 8dee466)Status: 1 Issue Found | Recommendation: Address before merge Executive SummaryOne misleading empty-result message in Overview
Fix these issues in Kilo Cloud Issue Details (click to expand)WARNING
Files Reviewed (25 files)
Reviewed by kimi-k3 · Input: 45.2K · Output: 5.6K · Cached: 369.7K Review guidance: REVIEW.md from base branch |
…missed An empty search told the model to "search with a query to find them" even when it had just searched with one, inviting the same call again. A query already covers every site, so that branch now says the search was exhaustive and offers the next useful move. The no-query branch keeps the query suggestion, and neither branch claims the misses belong to other sites.
Contributor
Author
|
Verified the review fix with the model, not just unit tests: three live Each run now settles in at most two |
jeanduplessis
approved these changes
Aug 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
UX and reliability pass: extension workflows
Workflows get first-class parameters, a readable approval card, a manual run form, actionable errors, and honest dry runs. Every behavioral decision was checked against live
kilo-auto/efficientruns — the weakest model we intend to support — plus Playwright e2e.Headline: on "run my flight workflow for ZRH to NRT",
mainneeded a median of 6 tool calls and never succeeded on the firstrun_workflow(0/4); this branch one-shots it 4/4 at 1 call. Creating a reusable, parameterized workflow went from impossible (no params support) to 4/4.Why the old flow failed weak models
Two design traps, both found by watching real runs:
run_workflowaccepted aninputobject, but nothing declared what a workflow accepted, andinputonly reached the script asstate.inputon the first page — a navigation silently dropped it. Agents read the whole script throughget_workflowto guess key names, then guessed wrong.save_workflowcalls for one workflow.What changed
Parameters
params:name,description,example,required.async ({ page, state, input }).inputis re-injected on every page, so navigations no longer lose it.state.inputstill mirrors it on page one for existing scripts.run_workflowrefuses to start when a required param is missing, naming each missing value with its description and example, and echoing the exactinputobject to retry with.search_workflowsresults, and on the approval card.Honest dry runs
page.waitFor(selector, timeoutMs?)handles dynamic content; dry runs record the wait rather than polling.Actionable errors
Every workflow error now says what happened and what to do next: out-of-scope names the tab URL and the scope, unapproved says to re-save for approval, invalid returns echo the value and both valid shapes, bad tool arguments carry zod field details instead of a bare "Invalid arguments", the safe-mode gate names the exact toggle, and failures are prefixed with the workflow name.
Discovery
search_workflowswith a query now searches every site (in-scope results first) and returnsinScopeandstartUrlper result, so "run my flights workflow" works from any tab. A relativestartUrl(/,/search) resolves against the scope instead of being rejected.Fixes found along the way
run_workflowinput is now size-bounded like navigation state. It is embedded in the injected page code on every page, but only state was checked.Awaited<ReturnType<...>>, added only to dodge the import-count lint cap) become a plain type import.vitest.config.tsnever matchedentrypoints/**/*.test.tsx, so the workflow-settings and conversation-events suites — 25 tests — existed but never ran. They run now, with the jsdom docblock and testing-library cleanup they needed.Data: live A/B on
kilo-auto/efficientN=4 per scenario per arm, against a local flights fixture whose results render 1.2 s after the search click (so a script must wait). Real gateway, real model, fresh browser profile per run, approval cards auto-approved.
main)run_workflowDry-run fix measured separately, same scenario, before and after:
save_workflowcallsWhat
mainactually did on "run my workflow for ZRH→NRT": callget_workflow, read the whole script to guess the input keys, run it, get empty results because nothing waits for async content, then re-run or scrape the page witheval. This branch reads the params from the index and one-shots it.Screenshots
Approval card, before and after:
Manual run parameter form, and a failed run:
Verification
pnpm --filter kilo-extension verify— typecheck, lint, 947 unit testspnpm --filter kilo-extension buildandbuild:firefoxpnpm --filter kilo-extension e2e:chrome— 117 passed, including a new parameterized-workflow scenario covering the run form,waitFor, and the missing-input errorpnpm --filter kilo-extension e2e:firefox— 35/35 (first attempt hit the documentednewSessionload flake; retry clean)The live-probe harness is not committed: it needs a personal gateway token and would run in CI. Its findings are encoded in the tests and error strings above.