build: make ci test addons in test/addons

[bnoordhuis]

Make make test-addons part of the make run-ci target.

R=someone from @nodejs/build?

CI: https://jenkins-iojs.nodesource.com/job/node-test-pull-request/105/

[bnoordhuis]

I'm open to suggestions on whether this is the best place to add it.

[rmg]

Based on discussion in #2134 when the run-ci target was added, this seems like the right place to add it as long as the intention is to run these tests as part of every build done on CI.

[orangemocha]

Probably best to rewrite the test-ci rule instead, as follows:

test-ci: build-addons
    $(PYTHON) tools/test.py -p tap --logfile test.tap --mode=release message internet parallel sequential addons

[thefourtheye]

@orangemocha Ah, internet? Are we including that as well?

[orangemocha]

Sorry I copied that from my local changes which include #2424 and add internet. I wanted to suggest adding build-addons to the rule prerequisites and addons to the test runner arguments. So you can try:

test-ci: build-addons
    $(PYTHON) tools/test.py -p tap --logfile test.tap --mode=release message parallel sequential addons

[thefourtheye]

Cool then :-)

[rvagg]

I'd love this to be added, we did try it early on but there are weird make conflicts in play, check out the results of your CI run and you'll see that it's very not happy

[orangemocha]

Some of these rules might need refactoring. High level observations:

1. Ideally we'd add this to the test-ci rule, which is about running tests for ci assuming that configure and build have already been done.
2. Rules pre-requisites can run in parallel. To reproduce on your machine what we do on Jenkins, you can run make run-ci -j $(getconf _NPROCESSORS_ONLN)
3. Since test-addons depends on test-build and test-build depends on all, test-addons is trying to re-build node when regular tests start running.
4. Can this have a Windows counterpart?

[bnoordhuis]

A more elaborate approach using order-only prerequisites, PTAL. CI: https://jenkins-iojs.nodesource.com/job/node-test-pull-request/117/

OS X is probably going to fail again with this:

xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance

I'm not sure what's causing the xcodebuild error - or rather, why it happens with add-ons but not for the main build.

The xcode-related diff between tools/gyp and deps/npm/node_modules/node_gyp/gyp is small enough that it's probably something in node-gyp rather than gyp proper.

EDIT: Apparently it's not a fatal error, the add-ons are built and run, see https://jenkins-iojs.nodesource.com/job/node-test-commit-osx/198/nodes=osx1010/console.

[bnoordhuis]

@orangemocha I addressed points 1 to 3 but as to point 4:

4. Can this have a Windows counterpart?

I honestly wouldn't know where to start.

[jbergstroem]

this target should probably live in PHONY, right?

[bnoordhuis]

No, that would force a rebuild every time.

[jbergstroem]

ok

[orangemocha]

4. Can this have a Windows counterpart?

I honestly wouldn't know where to start.

Ok, we can catch up later.

[bnoordhuis]

@nodejs/build Can I get a LGTM?

[rvagg]

excellent work @bnoordhuis, I had to run another one just to double-check that you'd actually fixed this: https://jenkins-iojs.nodesource.com/job/node-test-commit/291/

also, TIL about | dependencies in make...

lgtm

[jbergstroem]

LGTM. Do we perhaps want to separate the build process into build-ci or similar?

[rvagg]

Do we perhaps want to separate the build process into build-ci or similar

well it's the building of the addons that's the test here so I'm fine leaving that in there