Cypress job do not exit on fail (Gitlab CI)

[AlexLo33]

Hi everyone ! I don't know if it's a Cypress bug or a Gitlab bug but i'm stuck with my pipeline...

Current behavior:

I run my tests in a job with Gitlab.

When all tests are OK (no fails), the job end correctly and the next job is launched

BUT

When one test fail, the job is still running altough the Cypress process is trying to exit with code 1

gitlab-ci.yml

tests_e2e:
  stage: tests_e2e 🖥️
  script:
    - yarn cy:test

package.json :

"cy:run": "DEBUG=cypress:* cypress run",
"cy:test": "wait-on -t 600000 http://localhost:9000 && yarn cy:run"

Desired behavior:

My job will "successfully" fail after a test fail

With Debug mode, when all tests passed :

With debug mode when a test fail :

Steps to reproduce: (app code and test code)

• Launch a Gitlab job with successfull tests
• See the job pass
• Retry with a failed test
• See the job stuck

My test code :

Success :

it('Go to login page', () => {
    cy.visit('/signin');
    cy.get('input[name=login]').should('be.visible');
  });

Fail : (input with name login123 does not exist)

it('Go to login page', () => {
    cy.visit('/signin');
    cy.get('input[name=login123]').should('be.visible');
  });

Versions

Cypress 3.7.0

[AlexLo33]

FYI, Maybe i'm not the only one with this issue :
https://stackoverflow.com/questions/58880355/gitlab-ci-runner-hangs-after-cypress-tests

[jennifer-shehane]

@AlexLo33 Can you try running the tests in CI passing --browser chrome? This will launch the tests within a Chrome browser detected on your machine. I'd like to narrow does if this is an issue isolated to the Electron browser.

Also, there is an issue with hanging in CI here that we're working to fix: #5475 It may be related, but I haven't seen this happen in the same way your describing - only when a test fails.

You may want to try downgrading to 3.4.1 regardless, see if this is an issue isolated to a recent release in the past 6 weeks.

[AlexLo33]

I already have tried to downgrade to v3.4.1, same issue...
I'll try with a Chrome :)

[AlexLo33]

I have found the issue, it was not with Cypress but with Gitlab...

For informations :

I started my server in background with /myapp/bin/start & (so with &) and at the end of my tests Gitlab didn't want to go to the next script because my application's process was still running...

So I had to, in the same script, launch my application, run my tests, save the exit code out by Cypress, kill my application and return the saved exit code.

And, it works!

This issue can be closed

[bahmutov]

@AlexLo33 we recommend using something like https://github.com/bahmutov/start-server-and-test to start and shutdown server after the tests complete, you can find a few examples in our https://on.cypress.io/ci guide as well