Skip to content

Conversation

@ryanthemanuel
Copy link
Collaborator

@ryanthemanuel ryanthemanuel commented May 20, 2025

Additional details

We've implemented a new experimental command: cy.prompt() (that is still gated behind a feature flag). It will let you write tests using natural language prompts, like:

cy.prompt([
  `Visit https://aicotravel.com`,
  `Type Paris in "where do you want to go"`,
  `Click on the first result`,
  `Click on the number of day picker`,
  `Click on 4 days`,
  `Press the Create Itinerary Button`
])

Cypress will translate this into test steps, execute them, and adapt as the application under test changes (similar to self-healing behavior).

Steps to test

How has the user experience changed?

PR Tasks


Note

Adds a feature-flagged experimental cy.prompt command powered by a cloud-delivered bundle, with app/reporter UI, server/browser integration, a new experimentalPromptCommand config flag, and comprehensive tests.

  • Experimental Command:
    • Introduces cy.prompt (feature-flagged) with new experimentalPromptCommand config; reserves command name; adds error handling and types.
    • Driver: implements command, cloud bundle init/download timeout/SSL errors, cross-origin backend request handling, preserveRunState helper.
  • App/Reporter UI:
    • New Vue modals (PromptGetCodeModal, PromptMoreInfoNeededModal) and Pinia prompt-store.
    • Reporter shows “Get code” action for prompt logs; supports custom docs URL titles; minor log group defaults.
  • Server/Cloud:
    • Adds CyPromptLifecycleManager/CyPromptManager, routes, socket listeners, bundle download/verification, error reporting, and connection via CDP.
    • New cloud APIs: create session, get bundle, report errors; non-retriable cert error codes.
  • Browsers:
    • Chrome/Electron: connectCyPromptToBrowser; CRI client clones/manages additional target; reset/close updates.
  • Core/Data:
    • Data context actions to track current run/instance; HTML serve tweaks for simulated open mode.
  • Types/Tooling:
    • New types for prompt (app/driver/server); gulp task to download prompt types; config/tests updated; changelog entry.
  • CI/Infra:
    • CircleCI filters/branches tweaked; snapshot/hash updates; system/unit/e2e tests added.

Written by Cursor Bugbot for commit 3b81d4b. This will update automatically on new commits. Configure here.

@ryanthemanuel ryanthemanuel self-assigned this May 20, 2025
@cypress
Copy link

cypress bot commented May 20, 2025

cypress    Run #66187

Run Properties:  status check passed Passed #66187  •  git commit 3b81d4bef6: fix build
Project cypress
Branch Review feat/cy-prompt
Run status status check passed Passed #66187
Run duration 17m 52s
Commit git commit 3b81d4bef6: fix build
Committer Ryan Manuel
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 2
Tests that did not run due to a developer annotating a test with .skip  Pending 132
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 5590
View all changes introduced in this branch ↗︎

* feat: cy prompt infrastructure

* refactor and add tests

* refactor

* rename experimental config

* prompt

* fix test

* Update cy-prompt-development.md

* Update cy-prompt-development.md

* PR comments

* Update packages/server/lib/cloud/api/cy-prompt/get_cy_prompt_bundle.ts

Co-authored-by: Copilot <[email protected]>

* PR comments

---------

Co-authored-by: Copilot <[email protected]>
ryanthemanuel and others added 13 commits May 29, 2025 21:51
* chore: add cdp connection to cy prompt

* minor fix

* fix type build

* try to fix build

* Update packages/server/lib/browsers/browser-cri-client.ts

Co-authored-by: Bill Glesias <[email protected]>

* Update packages/server/lib/browsers/browser-cri-client.ts

Co-authored-by: Bill Glesias <[email protected]>

* do not support prompt in firefox or webkit

* rework timing of lifecycle

* refactor

* fix tests

* troubleshooting

* troubleshooting

* fix tests

* additional troubleshooting

* additional troubleshooting

* additional troubleshooting

* attempt to fix build

* add back

* debugging

* debugging

* debugging

* debugging

* clean up

* fix unit tests

* rework

---------

Co-authored-by: Bill Glesias <[email protected]>
…#31803)

* chore: add promptBackend as an additional Cypress-attached function

* Update packages/app/src/runner/event-manager.ts

* update types

* fix types

* fix spacing

* refactor

* additional refactor

* fix type build

* fix build

* refactor

* reword messages

* fix

* debugging

* undo debugging

* PR comment

* fix tests

* fix tests

* fix tests

* fix test
* chore: add watcher for cy-prompt development

* test caching

* fix types
…31835)

* internal: (cy.prompt) handle errors better in the command definition

* internal: (cy.prompt) add timeout and handle loading errors more cleanly

* add process environment variable

* clean up test

* update JSDoc
* chore: handle errors

* Fix ts, add test

* Fix error title

* Fix ts

* Fix ts
ryanthemanuel and others added 9 commits June 13, 2025 13:31
* share error utils with cloud

* additional rework

* Fix command, add isOpenMode

* Add / fix test

* fix ts

---------

Co-authored-by: Ryan Manuel <[email protected]>
…31904)

* chore: (cy.prompt) add infrastructure to support a Get Code modal

* fix tests

* fix code paths

* Update eject button styles

* handle errors

* update types

* Update packages/server/lib/socket-base.ts

* Fix cy test

* update readme

---------

Co-authored-by: estrada9166 <[email protected]>
* chore: (cy.prompt) refactor getTestsState to take a runnable id

* fix tests

* minor tweak
…31922)

* chore: (cy.prompt) add manifest for all of the cloud delivered files

* fix tests and remove environment variables

* update strategy

* fix build

* rework

* require manifest

* clean up

* refactor

* refactor

* Update packages/server/lib/cloud/cy-prompt/CyPromptLifecycleManager.ts

Co-authored-by: Matt Schile <[email protected]>

* fix test

---------

Co-authored-by: Matt Schile <[email protected]>
@jennifer-shehane jennifer-shehane changed the title misc: cy.prompt experimental command [Feature Flagged] (WIP - do not merge) feat: cy.prompt experimental command [Feature Flagged] (WIP - do not merge) Oct 3, 2025
@ryanthemanuel ryanthemanuel changed the title feat: cy.prompt experimental command [Feature Flagged] (WIP - do not merge) feat: cy.prompt experimental command [Feature Flagged] Oct 3, 2025
@ryanthemanuel
Copy link
Collaborator Author

The code owner approval was here: #31752 (comment)

@ryanthemanuel ryanthemanuel merged commit df047b0 into develop Oct 4, 2025
141 of 143 checks passed
@ryanthemanuel ryanthemanuel deleted the feat/cy-prompt branch October 4, 2025 14:01
@cypress-bot
Copy link
Contributor

cypress-bot bot commented Oct 7, 2025

Released in 15.4.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v15.4.0, please open a new issue.

@cypress-bot cypress-bot bot locked as resolved and limited conversation to collaborators Oct 7, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prompt: cy.prompt – Ability to generate Cypress tests from natural language

5 participants