Integration With CucumberJs

[bhreinb]

Are you requesting a feature or reporting a bug?

Feature

What is the suggested behavior?

TestCafe should be able to run tests written using the Cucumber syntax out of the box. This includes correct error handling and support for all TestCafe features (multiple browsers, screenshots, videos, etc).

Additional Details

Current behavior

How to reproduce

I have attached a sample program to help with the above...which can be found here

testcafe-cucumber devExpress.zip

npm install 
npm run e2e-test

Test code

const errorHandling = require('./ErrorHandler');
const testControllerHolder = require('./ControllerHolder');

fixture('fixture');

test('test', testControllerHolder.capture.bind(testControllerHolder))
  .after(t => errorHandling.ifErrorTakeScreenShot.bind(errorHandling)(t));

Results

1. UnhandledPromiseRejectionWarning on all TestCafe versions when failed assertion occurs within Role context
2. Warning: Was unable to take a screenshot due to an error rejects the promise as discussed here Screenshot Error on some Browsers - Unable to locate the page area in the browser window screenshot #2444 (this happens with v0.20.0 to the latest thus causing the flow within cucumberjs to be broke)

System information

• operating system: macOS High SIerra 10.13.6
• testcafe version: v0.22.0
• node.js version: v8.11.3

Original inquiry

Hello there,

I have been using testcafe for about 2 months. It is super tool but it's shame it's not officially supported with cucumberjs as it would open the tool to a bigger audience imho. That said I found this edition

https://github.com/rquellh/testcafe-cucumber

I made some further tweaks to that and it pretty much does everything I want it do do.

That said, I'm seeing one or two minor issues mentioned above when I'm testing with the latest edition, namely unhandled exceptions. I'm wondering can I add anything to the the testcafe script so that I can handle those aforementioned unhandled exceptions above and exit gracefully so the flow continues naturally.

I know cucumberjs isn't officially supported but it is a requirement for the project I'm working on atm so I'd appreciate any guidance on any of the above. Many thanks in advance. Any ideas when the official support is coming btw?

[bhreinb]

Update to this thread

I have updated testcafe to "0.19.2" and item 1) is no longer an issue for me. However that said item 2) is a significant blocker as I can't move past version "0.19.2" because of that.

I did some updates to the sample program used to reproduce this behaviour. It is attached. Let me know if you have any questions.

testcafe-cucumber 10Sept2018.zip

[AndreyBelym]

Hi @bhreinb! What browsers do you use for testing? If you test with Chrome and Firefox, you shouldn't have problems with the page area warning on [email protected].

Also, please note that you don't have to call t.takeScreenshot manually for taking screenshots when an error happens in your tests. I see that you already use Screenshot on Fails feature (in the cucumber/hooks.js file), but if you need to customize the path where these screenshots will be saved, you can specify a screenshot path pattern as the third argument of the .screenshots method.

[bhreinb]

Hi @AndreyBelym

Thanks for your response. I test against a wide range of browsers, for example chrome, firefox, safari, edge & ie. I also wanted to test with opera but that seems a bit unstable for now and not a high priority compared to the others I mentioned.

In terms of that warning "Was unable to take a screenshot due to an error rejects the promise" I have narrowed down the problem to an issue on mac.

Attached is my setup on mac which is connected to 2 monitors. When the runner launches the browser on the mac display it runs to completion every time. However, when the runner launches chrome for example on either of the secondary monitors it throws that unhandled exception error every time. As mentioned above this seems to only occur on Mac as I tested this with a colleagues setup (which is windows). Bizarre I know 👎. Hope this helps with that. Let me know if you need more detail.

When you say I shouldn't be taking screenshots manually, how do I accomplish this if I move way from that? The workflow currently is

1. Create the runner and capture the testController instance at the {Before} hook stage
2. Run testCafe commands bound to the testController instance at the {Given,When,Then} steps stage
3. Check for testCafe errors, take a screenshot, attach image & close runner instance at the {After} hook stage

Screenshots weren't been generated for me unless I explicitly have this line

.after(t => errorHandling.ifErrorTakeScreenShot.bind(errorHandling)(t));

in the testCafe script. Appreciate any guidance on the above.

[bhreinb]

Hi @AndreyBelym

Any thoughts on the above?

Ideally, I'd like to move away from I'm doing in step 1), that been creating the runner & capturing in the Before step but rather do this in the BeforeAll Step. Reason been there is a significant performance hit doing this especially when integrated with Saucelabs or Browserstack. Moving to the BeforeAll stage however introduces something else as detailed in this thread

https://testcafe-discuss.devexpress.com/t/testcafe-and-cucumber-js/207/24

so if you have any suggestions to allow the test run to continue when a test scenario fails (i.e. an ability to reset the assertion on the fly via an API call I'd really really appreciate this).

Btw any idea when official support for cucumberjs is coming?

[miherlosev]

https://stackoverflow.com/questions/57214388/debug-configuration-in-webstorm-for-cucumberjs-and-testcafe-based-project

[sachin-funde]

please implement cucumber integration with test cafe as soon as possible . it will definitely double the test cafe users.. just token of appreciation . u guys have done really wonderful work.

[Nofit]

Can you give an indication around what time this feature will be implemented? I am a fan of testcafe and I really love your work :). I am currently doing a project and they would like to have cucumber layer

[aleks-pro]

Hello @Nofit ,

Thank you for the feedback, but I cannot give you any estimates or time frames as to when this feature will be implemented. We still have plans to introduce it though. We will update this thread once we have any news regarding this.

[remram]

Any news regarding the integration of BDD - CucumberJs? We are starving and we would love to use testcafe in combination with CucumberJs. Please please please prioritize this higher

[AlexSkorkin]

We will update this thread once we have any news regarding this.

[Marketionist]

You can already use TestCafe with CucumberJs - through the plugins (see the Cucumber Support section in https://github.com/DevExpress/testcafe#plugins):

• gherkin-testcafe (by @kiwigrid) - run your Cucumber tests with TestCafe as a backend. Requires CucumberJS.
• testcafe-cucumber-steps (by @Marketionist) - provides predefined Cucumber steps for gherkin-testcafe.

[Nofit]

@Marketionist I think most people are aware that there are third party plugins for this feature, that is not an issue. But people would prefer a Cucumber integration with Testcafe that guarantees all features provided by Testcafe are still there, working, and not lost. I experimented with third party plugins for Cucumber integration in the past for a project and each plugin had flaws: lost features and/or information. Not losing functionality, is a fundamental requirement for an integration in my opinion.
This is what the original poster mentioned:
"TestCafe should be able to run tests written using the Cucumber syntax out of the box. This includes correct error handling and support for all TestCafe features (multiple browsers, screenshots, videos, etc)."
In my case, if the integration does not respect these requirements, then I won't use it. It's simple as that. I use a lot of features from Testcafe and I need them ALL, I can't afford to lose them for an integration. So for now, we cope without CucumberJs. I am willing to use third-party plugins, if they satisfy the conditions and the conditions are somewhat stated in the original poster's comment.

[Marketionist]

@Nofit I understand your point and I would also love to see Cucumber native support in TestCafe.

Just as a suggestion - gherkin-testcafe can be worth trying as it uses the same .testcaferc.json config file as TestCafe itself, so pretty much all options are there and can be used. It supports all TestCafe features (like multiple browsers, screenshots, videos, etc).

I agree that it's not "Cucumber syntax out of the box", but I'd say that it's the closest to "Cucumber syntax with 1 additional package".

[Nofit]

@Marketionist I remember I tried this plugin before and it is the best plugin between other third-party integration plugins in my opinion, but there was still something missing in it for me. Maybe it was the report format that I didn't like or the gherkin syntax, I am not sure. But I'll give it a shot again with your interesting plugin (which seems to make the syntax more complete), thank you for the suggestion! I hope there is also a step verification so that each cucumber-step is evaluated with a pass or fail in the report. That would be nice, but anyway, I'll try it out first.

[dskochubey]

I hope there is also a step verification so that each cucumber-step is evaluated with a pass or fail in the report.

@Nofit Have you had any luck with cucumber-steps in the report?

[miherlosev]

https://stackoverflow.com/questions/65190893/running-two-user-stories-in-the-same-scenario-in-cucumber-gherkin

[miherlosev]

https://stackoverflow.com/questions/69269832/any-official-support-for-testcafe-cucumber-integration

[miherlosev]

https://stackoverflow.com/questions/69935750/typeerror-err-invalid-arg-type-the-from-argument-must-be-of-type-string

[miherlosev]

Hi folks,

We decided to focus on core TestCafe features. We do not plan to implement the built-in integration with Cucumber.

TestCafe already has integrations with Cucumber: https://www.npmjs.com/search?q=testcafe%20cucumber. Some of them are quite popular and actively developed. So, we see that the community is successfully developing this direction on its own, and our decision will not lead to its stagnation.