E2E tests not executing

[jloveland]

after running gulp protractor

Error: Angular could not be found on the page http://localhost:3000/authentication/signin : retries looking for angular exceeded

from the angular docs:

Caveats

Protractor does not work out-of-the-box with apps that bootstrap manually using angular.bootstrap. You must use the ng-app directive.

It looks like we manually bootstrap here
Do we need to add np-app somewhere?

[ilanbiala]

@jloveland we should probably have an ng-app on either the HTML or body, though I usually prefer HTML. https://github.com/meanjs/mean/blob/master/modules/core/server/views/layout.server.view.html

[jloveland]

That didn't work. Need someone to take a stab at this who has more Angular experience.

When was the last time someone tries running e2e tests with gulp protractor?

[ilanbiala]

@jloveland have you tried running protractor through Grunt?

[jloveland]

@ilanbiala I found that when running grunt protractor or gulp protractor you need to have the app running beforehand using grunt or gulp. Now that I can get the e2e tests running, I see that the following e2e test fails:

Signin Validation Should report missing credentials

I will fix this when I add e2e tests to #910

[ilanbiala]

@jloveland is there any way to start the server before we start Karma? I thought there was.

[mleanos]

I guess the grunt configuration is missing the protractor task... we can add it with this...

grunt.registerTask('test:e2e', ['env:test', 'lint', 'server', 'protractor']);

If this is added to the grunt file the tests should run... Although the User e2e is incorrect, due to the how we're handling client validation with ng-messages.
https://github.com/meanjs/mean/blob/master/modules/users/tests/e2e/users.e2e.tests.js#L8

The only additional steps for a developer using the e2e tests is to make sure selenium web driver is installed; I also installed protractor globally (not sure if it's needed though)
https://www.npmjs.com/package/grunt-protractor-runner#faq

One thing I noticed is that the protractor tests are pointing to port 3000. Is this correct? Shouldn't the tests be looking for the test environment port 3001?
https://github.com/meanjs/mean/blob/master/modules/articles/tests/e2e/articles.e2e.tests.js#L6

[jloveland]

@mleanos you are correct, they should point to 3001 I am adding tests to #910 and can resolve this there.

[mleanos]

@jloveland Are you adding a grunt task as well?

[jloveland]

yes, I am adding both grunt and gulp

[mleanos]

@jloveland I'm wondering if the intention of the e2e tests is for them to be ran against a production/staging database (environment). Looking at the articles e2e test, it looks like it is expecting the count of articles to be zero because when this test was written, we didn't allow non authenticated users to view the articles list. I think the e2e may need/expect actual data. It's a little confusing because the test is also expecting "Missing Credentials". WDYT?

@lirantal @ilanbiala @codydaig Should the e2e tests be running against an empty test database, or a production/staging database that has actual data? I'm thinking the latter.

[mleanos]

This is a pretty good article that covers this subject...
http://blog.scottlogic.com/2014/08/06/angular-knockout-e2e-testing.html

What I gather from this article, is that we should be running it against our test environment, and writing our e2e tests with data population; just like we're doing in the server tests.

[jloveland]

Yup, I am writing tests that use the test database. I have a function that will drop the test database and then my sign up tests create a user by filling out the sign up form. Then I can test sign in, etc.

[ilanbiala]

@mleanos always test from scratch to avoid false logic in the code.

[mleanos]

@ilanbiala Thank you. That sounds about right to me. I just got thrown off because the articles e2e test is a bit confusing.