Setup for application tests and package testing

[gmsecrieru]

Should close #1044

[geekgonecrazy]

oh got it working! You sir rock!

Is the jasmine piece of the folder structure needed? If not maybe we could take it out? So we can make this as simple as possible.

[gmsecrieru]

I don't think it is needed, but should we add another testing framework to our stack it would be good to have those test cases organized IMHO.

[gmsecrieru]

@RocketChat/owners

Added Travis CI scripts and removing testing dependencies. This should be ready to merge and will help us setting up #1074

[gmsecrieru]

For some reason we are experiencing a timeout during travis-ci/pr build:

[samhatoum]

I think you guys are missing the latest chrome, so Karma can't start. Try this adding this to your travis yml:

addons:
  apt:
    sources:
      - google-chrome
    packages:
      - google-chrome-stable

[Sing-Li]

@samhatoum Thank you! The tests-after-build proceeded much more smoothly.

One question - the set-up time per packaged test is extremely long right now (for unit test where we might have thousands to run per build ... and if we have 10s of packages) - total build time can become quite a problem ... any ideas on how best to deal with it/structure our tests?

@gmsecrieru - please examine the logs - as there are multiple errors (one being web socket ECONNERR) in the logs but the final return code is 0 = green.

[samhatoum]

@Sing-Li no problem

regarding packages, there's a couple of things you can do.

1. cache the files downloaded by the package so that startup time becomes less
2. run the package tests in parallel on different travis nodes
3. combination of both!

[samhatoum]

you should consider using the velocity-cli npm package. See here: https://velocity.readme.io/docs/continuous-integration-ci

[Sing-Li]

@samhatoum Thanks again for the great suggestions!
@gmsecrieru @geekgonecrazy Are we ready to move on with this?

[leefaus]

thanks @samhatoum. so many things we can learn from him!

[samhatoum]

@leefaus no worries, happy to help

[Sing-Li]

@samhatoum - to avoid doing an extra build (takes up to 8 minutes - and we needed to do it twice already) just for testing, we included the following packages into our demo-server production build:

meteor add sanjo:jasmine velocity:console-reporter

Are they placebo when not activated? Will there be any side-effects? TIA

[samhatoum]

you should add

export VELOCITY=0

then they will be placebo, otherwise Jasmine will start mirrors

[Sing-Li]

@samhatoum We just merged a Cordova build to main branch, and it has a local copy of autoupdate 1.2.4.

The build breaks now when running velocity due to ISOBUILD constraints violation. Have you seen this before?

TIA

163.30s$ meteor add rocketchat:livechat rocketchat:hubot sanjo:jasmine velocity:console-reporter
rocketchat:hubot: updating npm dependencies -- coffee-script, hubot,
hubot-calculator, hubot-google-hangouts, hubot-google-images,
hubot-google-translate, hubot-maps, hubot-help, hubot-scripts, hubot-youtube...
builLivechat: updating npm dependencies -- shelljs...
WARNING: The output directory is under your source tree.

Changes to your project's package version selections:

package-version-parser            added, version 3.0.4
practicalmeteor:chai              added, version 2.1.0_1
practicalmeteor:loglevel          added, version 1.2.0_2
rocketchat:hubot                  added, version 0.0.1
rocketchat:livechat               added, version 0.0.1
sanjo:jasmine                     added, version 0.20.2
sanjo:karma                       added, version 3.0.2
sanjo:long-running-child-process  added, version 1.1.3
sanjo:meteor-files-helpers        added, version 1.2.0_1
sanjo:meteor-version              added, version 1.0.0
velocity:chokidar                 added, version 1.2.0_1
velocity:console-reporter         added, version 0.1.4
velocity:core                     added, version 0.10.8
velocity:meteor-internals         added, version 1.1.0_7
velocity:meteor-stubs             added, version 1.1.1
velocity:shim                     added, version 0.1.0
velocity:source-map-support       added, version 0.3.2_1

rocketchat:livechat: Livechat plugin for Rocket.Chat
rocketchat:hubot: Package hubot for Meteor server
sanjo:jasmine: Velocity integration of the Jasmine testing framework
velocity:console-reporter: A console reporter for Velocity
The command "meteor add rocketchat:livechat rocketchat:hubot sanjo:jasmine velocity:console-reporter" exited with 0.
$ ./node_modules/velocity-cli/bin/velocity test-packages --ci
[[[[[ Tests ]]]]]
=> Started proxy.
=> Started MongoDB.
=> Errors prevented startup:

   While selecting package versions:
   error: No version of autoupdate satisfies all constraints: @=1.2.4, @=1.2.3
   Constraints on package "autoupdate":
   * autoupdate@=1.2.4 <- top level
   * autoupdate@=1.2.3 <- top level

=> Your application has errors. Waiting for file change.
No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.
The build has been terminated

[samhatoum]

Does it work locally? Try zapping the versions file maybe?

[Sing-Li]

@samhatoum - same behavior locally. zapping the versions file made no difference. it added back autoupdate 1.2.4 -- seems like something in velocity start is insisting on 1.2.3 of autoupdate.