'Run all specs' removal in Cypress 10

[jennifer-shehane]

We’re excited to share that Cypress 10.0 will soon be released. It’s packed with new features, like first-class support for Component Testing.

The ability to 'Run all specs' or 'Run filtered specs' will be removed in Cypress 10.0. We have long had plans to improve the experience of 'Run all specs' as detailed in this issue: #1586

The motivation behind these changes have been many, but we want to bring parity with running specs during cypress run and also make considerations for the architecture and performance of the new Cypress App in 10.0 to run all end-to-end or component tests.
We’d like to hear your feedback on the usefulness of this feature to your workflow. What is the value for your team to 'Run all specs' or 'Run filtered specs' locally?

Please share your thoughts below. Your feedback will help us prioritize features to make writing Cypress tests even easier.

Update (11/21/22)

Updates provided by https://github.com/brian-mann

We have an experimental flag coming out in our next release tomorrow 11.2.0 introducing the experimentalRunAllSpecs flag that will restore the previous behavior seen in versions < 10.

Many of the core issues as documented and described below still need to be solved for, but this new experimental flag will provide an escape hatch for users who desired this behavior and weren't experiencing the tradeoffs/downsides.

We're still working towards an official GA release that incorporates the changes mentioned below and makes this behavior officially supported without any downsides or problems.

Update (6/21/22)

Updates provided by https://github.com/brian-mann

Hello, everyone! Thank you for all the feedback. While you may have seen some of our team members engaging in the discussion, we know that many of you are awaiting a more official response on what's to come. I'm updating Jennifer's original thread to float this update to the top of the discussion. I'll shed some light on the reasoning behind these changes, touch on possible workarounds, and share some plans for the near future.

The first two weeks since the release of Cypress 10.0 has been exciting, insightful, and humbling. We've kept busy with a handful of patch releases that we hope resolve some of the issues our users have experienced after migrating to v10. We know it's not everything everyone wants, and we understand that can be frustrating.

Regarding the 'Run all specs' capability, we've certainly been listening and learning about the various use-cases you all have. While we believe that the functionality as it existed was flawed and misleading, it's become clear that it provided value to several of you; albeit in an imperfect state.

Why did we remove this?

• We have a significant amount of users who utilize Cypress in many different patterns and use cases
• For many of our users, the existing implementation "was broken" for them because of how we originally architectured supporting this feature. For as much feedback as we’ve received regarding the removal of this feature - we had far more feedback about it not working correctly. Our recommendation to those people was to simply avoid using the feature.
• As a testing framework, we have an implicit guarantee to our users that tests will run consistently and deterministically. This feature broke that guarantee which was irresponsible to publish and expect people to use.

Why was the feature broken?

• “Run all specs” or “run filtered specs” used a naive and cheap “hack” to achieve this behind the scenes. It worked by combining multiple specs together into a single bundle and included a single “supportFile” to run with many specs in that bundle

• However, running an individual spec in open mode cypress open or utilizing the normal run all mode with cypress run would build each spec in complete isolation, and serve the supportFile once for each isolated spec even when passing a glob option to --spec **/*.cy.js

• This lead to specs running asymmetrically in a way that was problematic; for instance if you had a root level hook such as a before, beforeEach, afterEach, or after defined outside of a describe or context block, then that hook in a single spec would run for all specs that were served in the bundle.

• Besides hooks, any code that operated “globally” could leak into other specs and affect them. Reproducing these issues outside of the regular open mode or run mode was impossible because they only happened when running “all specs” or “filtered specs”. See issues here:

  • Inconsistent beforeEach behavior when 'Run all specs' executed. #3323
  • Cypress.config not scoped to spec when running all tests in the test runner #3693
  • Global before() and after() hooks are executed multiple times when running tests with Command Line but only one time when running tests with Test Runner #5160
  • GUI and CLI Treat Top-Level beforeEach Differently #7313

• Additionally while in open mode, Cypress adds specific debuggability features that can lead to excessive amounts of memory being allocated as well as performance degradation based on how many tests you have running at the same time. Serving multiple specs together lead to these situations being regularly aggravated.

• Serving multiple specs together in a single bundle also masked the individual specs being run. As you can tell from our UI - the individual spec header above the tests (called the reporter) indicates which spec you’re in and allows you to click on it from within the UI to open the spec directly in your IDE. By combining these, and not offering another visual indicator meant that tests were all merged into the same root tree without being able to visibly discern them from each other.

• See issues:

  • Screenshots saved under 'All Specs' directory when running all specs #2319
  • "run" is much slower than "open" #2912

What it take to bring this back?

Being able to run arbitrary, multi specs at the same time (correctly) is an incredibly powerful feature that we plan to bring back; and build it correctly. It is a nontrivial change however that requires at least 5 major changes to the UI and architecture to do so.

Changes Necessary

1. The specs list currently uses “a single select” link which runs the spec you’ve just clicked. This needs to become a multi-select variant.

2. The secondary specs list panel currently uses “a single select” link which runs the spec you’ve just clicked. This needs to become a multi-select variant.

3. The reporter has to become “multi spec” aware. Currently we use a single top level divider to indicate the spec. This needs to be upgraded from a “has one” to “has many” specs. We’d need a divider for each spec and there is additional navigation complexity with collapsing or traversing the reporter across many specs.

4. The individual specs even in multi-spec mode must utilize the same bundle/architectural guarantees as “cypress run” and cannot be combined into a single bundle. This would prevent state leaking from one spec to another.

5. We’d need to move some of (or most of) the debuggability features of Cypress in open mode out of the browser (to avoid hefty and potentially exhaustive memory allocation) and store them in indexdb in the node process. This would additionally not only help achieve multi-spec mode, but it would also improve the performance of single spec runs with many tests. The config option numTestsKeptInMemory would then be obviated and could be removed. However - the challenge with this is that while many of the debuggable snapshots we create could be serialized to indexDB, not all things could be. For instance using cy.stub(...) or cy.spy(...) would lose references to individual functions which are helpful to console.log(...) when clicking on their command logs. Those would need to stay in memory or we’d lose that feature and functionality.

What new workflows would this unlock?

• The above set of changes is a considerable amount of work, which is why it was cut from the original v10 scope

• However, creating a proper “run all specs” mode would unlock new powerful features and workflows that isn’t possible to achieve today

• We’d be able to move from watching a single spec to watching many specs and utilize the dependency graph available especially for component testing to bidirectionally watch specs and their source code. This would enable you to make a change in a src file and us automatically run all the specs that are importing and testing that src file. This would potentially have the power to negate the need to move around the UI manually and instead let your source code changes automatically drive “switching” to the specs that are affected.

Will we bring this back?

• We're internally creating a product + technical requirements doc in Q3 that investigates the amount of work to bring this feature back

Are there any current workarounds?

• Yes. If you are impacted by the omission of this feature, it is possible to achieve the same level of parity as 9.x with a workaround Gleb Bahmutov explains here: https://glebbahmutov.com/blog/run-all-specs-cypress-v10/
• This will still inherit the same problems as the previous implementation (which is why it was removed) but it will work in certain cases where the previous implementation was not problematic for your use case

[JasonFairchild]

While I wouldn't say it was particularly important, I would use the run by a filtered set fairly regularly. Usually when I updated a test and wanted to run other tests for a given area just as a small sanity check. We tend to break out tests into more spec files in my current setup, as opposed to more 'it' blocks in a file, so that is another reason this is likely more useful.

I'm also wondering if the change for this is partly from concerns that running many tests at once can cause memory issues with all the DOM snapshots and perhaps other data that gets saved in a run? Mostly curious. And I guess I would just emphasize again that I'm much more concerned with running some subset of tests as opposed to running 'all tests'. I still like a lot of the suggestions in issue #1586 :)

[jfairchild-aumni]

I wanted to add that I think this feature would be even more useful for component testing. At least since I've been working in the component test world much more for the past few months, I've missed this feature even more. Maybe in particular since we utilize VRT quite a lot for components, and we want to maintain that all the tests pass in both headed and headless browsers (it hasn't been easy, but we've been able to manage it so far). It also just seems more fitting for component tests, since they are supposed to be fast and easy to run compared to E2E and so run more frequently and locally.

[SimonDevon]

Agree with the above comment - having this for component as well as e2e tests would be a great addition!

[lmiller1990]

Did you try experimentalRunAllSpecs that was released recently for E2E? What do you think of it? Any feedback on the UI?

Implementing this for Component Testing is one of our (Component Testing team) deliverables for next quarter (Jan-March).

[lmiller1990]

https://docs.cypress.io/guides/references/experiments#End-to-End-Testing I thought we did a blog post, I guess we didn't. We should.

[murillowelsi]

From my perspective, it's something that affects the workflow, it was a nice feature that helped when I wanted to run my entire regression test suite (visually tracking the execution).

[josbelluna]

Part of our team workflow is testing using the Run filtered specs to filter some application modules. For example, if we modify something in the accounts module of the app, an engineer typically runs all the specs in the accounts folder. Running one by one is an option, but it is not a pleasant experience as the previous one, and maybe it would be a blocker for us to migrate to version 10.

[neilpoulin]

I'm a little confused - i read the ticket linked in the description #1586 but it didn't clear up the reason that "Run all specs" was removed. You mentioned brining parity with running specs during cypress run... what does that mean?

I often run all of my tests locally so I can get a visual indication on what tests failed and have the results immediately available to look at & interact with. Granted, i have a small test suite so I haven't run into any performance issues that may be affecting larger projects. One reason I may want to run all specs interactively is that I may have made a change to my codebase that would affect multiple test suites, and i'm using cypress actively while I develop. It's nice to just hit "run all tests" to make sure i didn't break anything.

Version 10 is looking really cool, and I hope that I can get the component testing working with my Next.js project soon.

[neilpoulin]

Thank you for the thoughtful and thorough explanation.

I get what you're saying about the difference between the two runners. That's certainly not what I would have assumed. However! I am personally OK knowing that the Ui (open) may run all tests slightly differently than run, particularly since the UI tests run so much faster. Maybe if the button was called something like "Quick Run All" (or something better) it could help communicate that the behavior is different while giving all the people on this thread the productivity boost we need.

That being said, it just seems very unexpected that a beautiful test runner software wouldn't have a button at all to run all tests. I'd honestly be (mostly) satisfied if there was a run all button that kicked off cypress run in the background and presented nicely formatted results with links to go to the failed test suite.

[pjg]

Our biggest goal at Cypress is to ensure determinism of test runs

@jennifer-shehane This is understandable and it is the correct approach to js tests. As such I know why we no longer have the "Run all specs" in the GUI as you're concerned about leaking state (I did see state leaking from spec to spec in my specs when using this option prior to version 10).

However, this results in a very long feedback loop from running tests to seeing results (and correcting code/specs for any failures). Some (most?) of us want to have a closer/faster feedback loop, and having "Run all specs" in the GUI was just that. I don't care about seeing specs being run in the browser nearly as much as I care about speed.

What do you think about adding the --unsafe-fast (or similar) command line option to the cypress run, which would 1) disable video recording/screenshots (why isn't there a separate CLI option to do that?) and 2) run all specs in a single browser window with all of its implications. This would be at least something for running all specs in a timely manner, and having the unsafe monicker there would clearly mark it as something one should use sparingly.

[granite]

Our biggest goal at Cypress is to ensure determinism of test runs

Disabling interactive features doesn't improve the determinism of CI test runs.

A person's productivity writing and running tests is also very important. If the tool is too hard to use, people won't use it, or switch to a more user friendly tool.

Removing an existing useful interactive feature (which btw is much faster than the non-interactive cypress run), is harming people's productivity.

It's rare to see really bone-headed decisions made on an open source project, but for this decision to have been made, cypress devs must be out-of-touch with how people are using the software.

I think there could also be some competing motivations at play here since the company is making money selling subs to Cypress Dashboard which has advanced features like parallelization, run failed tests first, flaky test detection, etc. Alot of those features could have been added to the open source test runner also, but obviously counter productive for the company. Maybe that's why these decisions are being made; The open source test runner is low priority, and by removing features, maybe more companies will upgrade to Cypress Dashboard.

[lmiller1990]

As the (at the time of this comment) 53 comments suggests Run All Tests is something sorely missed - the feedback here is loud and clear, and we are definitely hearing it, but I think your last point warrants a more direct response. Removing this feature has nothing to do with our paid dashboard - the OSS runner is a separate product and not changes there are only ever about improving the local experience, never about pushing people to the paid solution.

To clarify, features like parallelisation take advantage of multiple machines on CI environments (not something you can do locally on a single machine). Features like flaky test detection and prioritising historically failed specs require a database with the history of all the runs from all your developers, which also doesn't really make sense as part of the core runner on a single developer's machine. In contrast, the Run All Specs feature is only really useful locally - you don't often run a specific subset of specs on CI, you just run them all.

They are all optional and additional adds on that make sense from a paid SaaS perspective (capturing CI artifacts and persisting thousands of screenshots and videos costs a lot).

I hope this gives some more perspective; the main point I'm trying to convey is, while removing this particular feature (temporarily, until we figure out all the kinks and bring it back with a better UX) is exactly that - the goal is not, and never will be, to force people into a paid solution by downgrading the core OSS offering.

[darrelfrancis]

@granite - At first I had the same suspicion as you, that it was to nudge people into the paid version, and alleged this in my Medium post offering a Community-proposed workaround. However, there is actually no direct equivalent of this in the paid version. I believe the Cypress maintainers, that it was really to improve the fidelity of the tests, by preventing data leakage between test files. The maintainers have made useful suggestions in making our Community workaround more reliable, which is evidence against any monetization motive. I have therefore corrected my Medium post.

In any breaking change, the angry people (like you and me) will initially vastly outnumber the happy people, since our workflows are immediately destroyed, while the happy people aren't actually happy yet since they are not aware of the new possibilities the breaking change has created: their gains will be gradual as they notice what they might be able to do.

Since Cypress are helping people optimize the Community workaround, let's give them space to work out how to go forward. You and I might be working alone or in small groups, so we can basically just decide what to do and then do it, but I bet the Cypress people have to have meetings about meetings, discuss "going forwards", "mindshare", "learnings", "thinking outside the box", before even beginning to change the software.

Businesses, like people, can make mistakes. It takes them much longer to reverse out of them, though. Let's give them space.

[logan5259]

Why have you implemented this horrible idea to remove filtered specs and run all specs ?
What about cases when test creators/devs want:

• run only group of API tests locally
• want to check if new test works with other related tests
• run only group of e2e tests locally
  Removing this option encourage me to think about Cypress negatively, I was using run filtered specs all the time, same as run all specs. In my opinion it was great feature to have an ability to see how filtered group of test works right inside the Cy.

[darrelfrancis]

Workaround: https://medium.com/@darrelfrancis2013/cypress-10-how-to-run-all-tests-despite-the-option-being-removed-725199f4e720

[lmiller1990]

Neat idea... I expanded on it a little with a plugin + UI: https://github.com/lmiller1990/cypress-explorer

[matrunchyk]

First of all, thank you for Cypress 10! This is crazy cool!

But I ask you to please revert "Run all specs" back. How can I run my 200+ tests with cypress open now? Clicking one by one is not the best idea of course.
Running cypress run works, but I don't use cypress run on my local machine (but CI only), because I want to see and examine my failed tests.

Thanks again.

[lmiller1990]

You can do yarn cypress run --headed --browser chrome - maybe this helps? Maybe not exactly what you are looking for, but wanted to point this out, just in case you didn't know about the --headed flag with cypress run.

[Fari-cloudflight]

almost but if a test fails it does not stop nor does it show any useful output...

[samnittala]

Yeah! we need this back - be able to run a single/set of tests.

[mirobo]

I think removing this features is a huge loss. My workflow is that I start Cypress.exe in global mode so I'm able to use just one instance of Cypress over a variety of Cypress projects (and switch between them very easily). I don't see the point of starting a new instance with "cypress run" and fiddling around with "cypress run --spec ...". Most of the time I just need to analyze and run a single test (i.e. if it's flaky) or a group of tests within the same folder. With the possibility to just type a partial name of the test in the input field, which filters the specs, it is extremely handy and quick to do so.

Alltough there is an issue when filtering specs with for example "my test" Cypress can't run the filtered tests using "Run X integration specs" (probably because of different filtering mechanics in the UI and with the "specs" option?).. but at least you can start each spec on it's own.

I rarely use "cypress run" locally; I'd rather run all the specs via "cypress open" so I can analyze errors directly in the command log. Screenshots and videos are often not enough; it just gives an idea what you probably have to fix. But to get the full picture I always rerun the test via "cypress open". I only use "cypress run" when running the tests in CI.

Please bring this feature back ❤

[Zwimber]

This functionality was simple & great and it being removed is making me regret upgrading, even though I love many of the changes in 10.0.0.

Also cypress run is still slower than cypress open + run all specs. With the added benefit that the latter will also give you something that's fun to watch and easier to debug when finished. Only when cypress run is improved to run much faster than cypress open this function can been removed.

I'd love it if this functionality can be added again so we can upgrade our other projects 🔥

[KuroiRaven]

In my opinion, this feature is a must have in most project.
It was really handy to be able to just filter some specs that were affecting a patch/dev and run only these to ensure we didn't break this part of the app.

Now we either have to run a cypress run on a manually filtered set of spec or run these one by one. That's pretty tedious to be honest.
I'd love to see that feature come back :)

Thanks alot for all the effort you guys put in the new version though ! It is pretty impressive !

[JamShady]

I think this was a terrible decision to remove a feature without offering any alternative, as though no-one used that feature at all.

I heavily relied on being able to filter and run specific spec files when I'm changing functionality that'll affect a group of tests. I can hit one button and run everything I need. Now, I have to run each one individually... what a waste of time, and two steps backward.

[jennifer-shehane]

This was a hard call to make on our side as we knew this was a valuable feature for many. There was not an assumption that no one used it. Thanks for sharing your feedback - we intend to use your feedback to drive our prioritization of this feature in the future.

[JamShady]

Surely an easier call would have been to just leave it alone? If you're making this "hard call", then one has to wonder why you thought it was more beneficial overall than just leaving it in? I'd love to know the reasoning, because from where I (and presumably everyone else) are sitting, it's brought no benefit.

[Deop]

I'd love to see this feature being added back. Being able to filter test for a specific subset and run them all together was very useful.

[Vlasterx]

Bring back "Run all tests", I don't want to click on every spec separately. This is tedious and terrible UX! 👎

[sitefinitysteve]

I'm so annoyed by this. We have a series of ordered test (first being a primer kind of test to get the CMS pages warmed up), then like 30 subsequent tests. We don't have any CI, the tests are just manually kicked off when there might be a major breaking change (CMS\ORM upgrade).

Why would you remove such a useful obvious feature because some people might not use it, very clearly lots of us relied on that (a more manual) workflow.

[tho-masn]

"Run filtered tests" was a pretty important feature for us too. We have a large amount of spec files. Combined with the search this is really, really powerful. Would be great to have it back - the button is neither annoying, nor does it take much space...

[HendrikJan]

I am considering migrating from Cypress v9 to V10, but I need this feature at least in the terminal.
Does anybody know if this is also affecting terminal tests, or only in-browser tests?

[dmolin]

@HendrikJan A few ways have been suggested to "patch" this until Cypress provides an official fix. This is how I solved it (and it works just fine for my own use case): #21628 (reply in thread)

[yktoo]

This only affects Cypress IDE (the visual mode), the console mode hasn't changed.

[dmolin]

true, you can still run console tests as before

[HendrikJan]

Thank you yktoo and dmolin!
With console tests and a crude workaround that's good for now.

[raymondshiner]

Please bring back this feature. For smaller projects (like personal projects) or just in general this is a great feature to have. One example is i like to run my tests locally before pushing them to PR as i dont want to have to wait for all the CI tests (BE and FE) to run just to check the part i'm responsible for not breaking (FE).

[lmiller1990]

We are working on this right now, see #21628 (reply in thread)

[brian-mann]

I've posted an update in the opening thread (above) documenting the new experimentalRunAllSpecs flag which is releasing tomorrow in 11.2.0. It restores the old functionality prior to version 10 but within the new UI, specs list, sidebar, etc.

[gilsa-2022]

Thanks. Working just fine. Great job.

[melibe23]

thank you for bringing it back! Can't wait to migrate now :) Xmas has really arrived earlier this season!

[sitefinitysteve]

"scripts": {
    "cypress": "cypress open --experimentalRunAllSpecs"
  }

@gilsa-2022 If you don't mind, how did you run it with the flag? I just get error: unknown option: --experimentalRunAllSpecs

Cypress package version: 11.2.0
Cypress binary version: 11.2.0
Electron version: 21.0.0
Bundled Node version:
16.16.0

[emilyrohrbough]

@sitefinitysteve this should be cypress open --config experimentalRunAllSpecs=true

[sitefinitysteve]

Perfect! Thanks @emilyrohrbough

[mirobo]

Just tried it out, the search/filtering algorithm seems to be partially broken.. (just get the old code.. this one worked fine :-P) it's behaving the same in the runner UI. I was expecting to be able to click "run all" on the first search UI, but this is minor inconvenience. I think the big, white search UI is superior (bigger, easier to read.. more screen estate is used compared to the tiny search area in the runner UI)

Update
Sorry, this has nothing to do with 11.2.0 update.. this bug is also present in 11.1.0.. can't tell when it was introduced. I pretty sure it worked correctly in 9.7.0.

[lmiller1990]

Seems like fuzzy searching isn't working as ideally as you'd expect, definitely a bug but not one caused by Run All Specs (just made more obvious).

[jfairchild-aumni]

Who else would like this feature or something much like it for component testing?

[lmiller1990]

I would, we will work on this on early 2023.

[ahaurat]

Will there be a v12 release that includes the experimentalRunAllSpecs option?

[lmiller1990]

It's already included in v11 and v12 @ahaurat, you just need to enable it: https://docs.cypress.io/guides/references/experiments#End-to-End-Testing

[ahaurat]

Wonderful, thanks for all the work on this @lmiller1990 !

[joshribakoff-sm]

You are now forced to click each spec one by one

Please register my displeasure :/

[joshribakoff-sm]

Note after setting experimentalRunAllSpecs in the wrong place, you show me this message via hot module replacement / reload

After moving it to the correct place, the error doesn't go away. I have to kill and restart Cypress. Then the "run all specs" does show up.

[lmiller1990]

You should be able to hit "Try again" button to reload the config.

Once you got it working, how was it? Does it work as you expected/any issues/feedback?

[joshribakoff-sm]

Thanks, I must have missed that. I guess I expected it to just "hot module replacement" the page away :)

Yeah the "run all specs" works now like I wanted!

I had to read this thread several times before I understood your concerns that beget it's removal. In the future I would consider leaving a feature in (even if you feel it was flawed), and marked it deprecated / added warnings surrounding it's usage or callouts to the docs. I think from an outside perspective, it seemed capricious to take it away the button to "run all specs", like when Apple takes away physical ports on the phone and you just sort of 🤷 😆

[TheJaredWilcurt]

Last update is from 6/21/2022. Can we get an update on prioritization for adding a "Run all tests" option back in.

Version 12 still requires the hacky workaround.

---

Edit: As of versions 11 and 12 you can manually re-enable it.

In /cypress.config.js:

const { defineConfig } = require('cypress');
module.exports = defineConfig({
  e2e: {
    experimentalRunAllSpecs: true
  }
});

This too, should be documented in the main post on this page so people don't need to randomly stumble upon it in the weeds of the dozens of replies this has.

[emilyrohrbough]

@TheJaredWilcurt The work to provide support for Run All Specs in Component Testing is on our radar. You can follow its progress on the Cypress App Priorities board.

[TheJaredWilcurt]

That link is a 404, here is the actual link:

https://github.com/orgs/cypress-io/projects/13/views/1?pane=issue&itemId=19257105

[justinsako]

The experimental "Run X Specs" seems to be working great for me with the addition of the setting in cypress.config.ts: e2e: { ... experimentalRunAllSpecs: true ... } ...in Cypress 12.3.0 and 12.4.0, at least. Thank you for this feature, by the way! :-)

…

On Feb 1, 2023, at 11:51 AM, The Jared Wilcurt ***@***.***> wrote: Last update is from 6/21/2022. Can we get an update on prioritization for adding a "Run all tests" option back in. Version 12 still requires the hacky workaround. — Reply to this email directly, view it on GitHub <#21628 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAUHRNUWBDZTOKWHAGBQ3X3WVKWCHANCNFSM5W5VG4QQ>. You are receiving this because you commented.

[mirobo]

I'm not quite sure why they added the tiniest of "run all"-buttons in the Runner UI and not also on the first (bigger) UI (in white and with blue buttons) that initially had the "run all" button. The runner UI is so tiny and thus hard to do any comfortable filtering within. I'm missing the button on the "big" UI (the launcher?). The runner UI takes a third of it's screen space when spec filtering is opened... (the spec filter and the side bar (with not much to click on anyway) takes the same space as the command log)

[lmiller1990]

I think the new UI is okay, but if you have a recommendation can you file and issue and (ideally) make a recommendation (or attach a screenshot and use paint to demonstrate)?

I don't think adding more features to the electron app (where you choose your testing type and browser) is likely, but what is more likely is improving the runner UI. It sounds your goal is "I want to open Cypress and run all the specs as fast as possible"?

[mirobo]

I finally just had an idea.. :D

Drop this view completely:

Instead open directly the runner but don't start any tests.

• Move some navigation buttons from the left side to the bottom (rarely used buttons/icons like settings, keyboard shortcuts, Cypress logo)
• Move navigation buttons like Debug, Specs, Runs to the top (above the search specs)
• Add some more buttons for filtering the specs

Advantages:

• More horizontal screen space :)

[lmiller1990]

You can actually implement a proof of concept right now - in your supportFile, you can write any CSS/JS you want - you could rework the UI by adding/removing CSS classes. There are probably some limitations, but I suspect you can easily do

• auto navigate to runner page
• move sidebar to the bottom

Without much effort.

[Fari-cloudflight]

https://dev.to/jprealini/extended-run-all-specs-feature-for-cypress-10-303c
the fact that someone had to come up with that shows how poor the decision was

[lmiller1990]

@Fari-cloudflight this was re-introduced with a bunch of improvements: https://www.cypress.io/blog/2023/01/12/check-out-our-experimental-release-of-run-all-specs/

You need to enable it for now, since it's still experimental (has same bugs as the v9 implementation had, but works as it used to).

[c32hedge]

We depend on the ability to run filtered specs (and will have to use Gleb's workaround as we're upgrading to the latest Cypress) because of #17940. If y'all fix that bug, I'd love to get away from having to split our tests that switch back and forth between superdomains into groups of similarly-named files. Until then, it's a bit of a pain to have to maintain multiple files and the extra file needed for each group as part of the workaround instead of just being able to run filtered specs like we could prior to Cypress 10.

[lmiller1990]

@c32hedge we added Run All Specs back behind a experimental flag, see this blog post. You'll need to opt in to use it, but it works like the old one did. Let me know if you have any problems or run into any edge cases that aren't solved by this.

[c32hedge]

@lmiller1990 Ah, I knew there was a flag added for Run All Specs but based on the name it wasn’t obvious that this allows the old behavior of being able to run a filtered subset of tests as well. Thanks! (Though I’d still love to see that bug get some attention 😉)

[h2oearth]

Please bring it back!

[mirobo]

You can already use it with an experimental flag: https://www.cypress.io/blog/2023/01/12/check-out-our-experimental-release-of-run-all-specs/#:~:text=To%20use%20Run%20All%20Specs,when%20hovering%20over%20a%20directory.

[albanx]

the testIsolaion mess and this now, I am thinknig to seriosly switch to Playwright

[vinh-point-me]

Please bring this feature back...

[lmiller1990]

This is available via https://www.cypress.io/blog/2023/01/12/check-out-our-experimental-release-of-run-all-specs

[BennyHudson]

Is there still a plan to roll out 'Run All Specs' for component testing? I've got a suite of ~250 that I would really like to just be able to run through quickly, saw this was on the roadmap for Q1 2023, but all seems to have gone quiet since.