Add trajectory recording to cua do CLI - #1110
Conversation
Every cua do action is now automatically recorded to a replayable
trajectory at ~/.cua/trajectories/{machine}/{session}/. Viewing opens
cua.ai/trajectory-viewer via a local CORS-enabled file server.
New files:
- trajectory_recorder.py: session management, turn writing, zip, clean
- trajectory.py: cua trajectory ls/view/stop/clean commands
Modified:
- do.py: --no-record flag, post-action screenshot recording in all handlers,
session reset on switch
- main.py, __init__.py: register trajectory command
- SKILL.md: document trajectory recording
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
@sarinali is attempting to deploy a commit to the Cua Team on Vercel. A member of the Team first needs to authorize it. |
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one) (1)
Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
cb7852b
into
trycua:ddupont/cua-cli-improvements
* add agent-computer style usage to cua-cli, refactor pyautogui-like handlers from computer-server into its own SDK for reuse by our various SDKs * address CR comments, add auto-focus when zooming to windows on the host * Add cua-auto to pypi workflow * Bump cua-cli requirements * default `cua do ls` to listing all sandboxes * Fix linting error * fix linting * Add trajectory recording to cua do CLI (#1110) Every cua do action is now automatically recorded to a replayable trajectory at ~/.cua/trajectories/{machine}/{session}/. Viewing opens cua.ai/trajectory-viewer via a local CORS-enabled file server. New files: - trajectory_recorder.py: session management, turn writing, zip, clean - trajectory.py: cua trajectory ls/view/stop/clean commands Modified: - do.py: --no-record flag, post-action screenshot recording in all handlers, session reset on switch - main.py, __init__.py: register trajectory command - SKILL.md: document trajectory recording Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: ddupont <3820588+ddupont808@users.noreply.github.com> --------- Co-authored-by: Sarina Li <sarinajin.li@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…mputer-server, computer, and the cua CLI (#1114) * add agent-computer style usage to cua-cli, refactor pyautogui-like handlers from computer-server into its own SDK for reuse by our various SDKs * address CR comments, add auto-focus when zooming to windows on the host * Add cua-auto to pypi workflow * Bump cua-cli requirements * default `cua do ls` to listing all sandboxes * Fix linting error * fix linting * Add trajectory recording to cua do CLI (#1110) Every cua do action is now automatically recorded to a replayable trajectory at ~/.cua/trajectories/{machine}/{session}/. Viewing opens cua.ai/trajectory-viewer via a local CORS-enabled file server. New files: - trajectory_recorder.py: session management, turn writing, zip, clean - trajectory.py: cua trajectory ls/view/stop/clean commands Modified: - do.py: --no-record flag, post-action screenshot recording in all handlers, session reset on switch - main.py, __init__.py: register trajectory command - SKILL.md: document trajectory recording Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: ddupont <3820588+ddupont808@users.noreply.github.com> * add interactive terminal support * add try/except around imports that require X display server * address CR comments * lint & isort * bump cua-core dep * add tests for windows * fix cua do shell unknown command error * reorder imports * add interactive shell to docs * update computer sdk reference * lint windows tests * fix external link checking lychee workflow checking internal links * attempt to fix lychee workflow again * fix external links --------- Co-authored-by: Sarina Li <sarinajin.li@gmail.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Summary
cua doaction is now automatically recorded to~/.cua/trajectories/{machine}/{session}/in a format compatible with the TrajectoryViewer atcua.ai/trajectory-viewercua trajectory viewzips the session, starts a background CORS-enabled file server, and openscua.ai/trajectory-viewer?zip=localhost:PORT/file.zipin the browsercua do --no-recordNew files
cua_cli/utils/trajectory_recorder.py— session management, turn writing, zip, cleancua_cli/commands/trajectory.py—cua trajectory ls/view/stop/cleancommandsModified files
cua_cli/commands/do.py—--no-recordflag, post-action screenshot recording in all 11 handlers, session reset onswitchcua_cli/main.py,cua_cli/commands/__init__.py— register trajectory commandskills/cua-do-cli/SKILL.md— document trajectory recordingTest plan
cua do switch docker test && cua do screenshot && cua do click 100 200, verify~/.cua/trajectories/test/hasturn_001/andturn_002/with JSON + screenshotscua do --no-record click 100 200, verify no new turncua do switch docker other, verify new session directorycua trajectory ls, verify outputcua trajectory view, verify zip served and browser openscua.ai/trajectory-viewercua trajectory stop, verify server killedcua trajectory clean -y, verify sessions deleted🤖 Generated with Claude Code