GPII-1731: Windows Installer should be integrated with our deployment…

[minogian]

… and CI infrastructure.

Added a build script for the vagrant provisioning stage that clones the installer repository (https://github.com/GPII/gpii-wix-installer) and builds an MSI installer for the project

[javihernandez]

Hey there,

I've been trying this and the provision failed when trying to build the installer. Check this out. Also I've got a question, probably I'm missing some information but, does the build process make sense at provision time? Ideally, we should have a grunt task that is able to perform such build inside the vagrant box, isn't it?

[minogian]

That's strange. I checked it again, set up a new VM from scratch and after "vagrant up" everything goes as expected and an installer is built (see http://pastebin.com/3PcHyQcj for the whole log).
Did you also try with a new VM from scratch or did you apply the changes to an existing one?

As for building an installer "on demand" and not as part of the CI build, you should be able to run the build-installer.bat script anytime. Just make sure the windows path is properly set before doing so (I have observed that the vagrant VM may need a restart before the path is properly persisted).

[javihernandez]

I ran a vagrant destroy -f and then vagrant up. After the vagrant up, I force a reboot when provision is finished but still won't build. Any ideas why I'm having such problem?

[minogian]

Can you open the verbose output of the "harvest" command?
Comment out lines 6-8 of C:\vagrant\provisioning\build-installer.bat (http://pastebin.com/TG3CeNzB) and add a "-v" option to line 69 of C:\installer\setup\setup.msbuild (http://pastebin.com/1qnpMQBt).

Use the existing VM, do not start from scratch.
Open an administrator command prompt inside the VM and run build-installer.bat

[gpii-bot]

Could one of the admins verify that these changes are reasonable to test? If so, please reply with "ok to test".

[minogian]

@javihernandez I managed to reproduce the issue you were having: I was using the (then) master branch and you were using the master branch plus the 35ccc28 commit.
Now that the commit is merged to the master branch I stumbled upon the same issue.

What happens is that the inclusion of the universal branch as an npm dependency creates unwieldy long paths that are problematic to handle under Windows. I had to use npm prune --production and npm dedupe to shorten the paths before building the installer (pruning and deduping is applied on a copy of the repository - NOT in C:\vagrant). I have rebased and updated the pull request with the aforementioned changes.

Still, the final installer can only be used to install GPII in "short" paths like C:\GPII and not longer ones (like C:\Program Files (x86)\GPII) because Windows won't allow it (see the attached screenshot).

Unfortunately there is not much I can do about it. That is a known problem with Windows and the way npm handles dependencies. You either have to refactor somehow the dependencies of the universal repository or accept that the installer can only be used for installing in short paths.

[amb26]

I can see only two forms of option -
i) To move our architecture over to npm 3. In theory this should cause few problems since I've been trying to make sure for a while that we have no hard-coded inter-module paths, but will take a while to shake down. This will also add an extra installation step since we will be installing a version of npm different from the one bundled with the current LTS node.

ii) Improve the installer so that it can use the special Windows filename format and API which allows for manipulation of long paths: http://superuser.com/a/760601 - it seems that this has been supported on Windows since Windows 7 SP1 as long as the filesystem is NTFS - https://support.microsoft.com/en-gb/kb/2891362

[javihernandez]

@minogian,

correct me if I'm wrong, but the problem is not during the "npm install" process but when trying to copy the already "npm-installed" files into a location. If so, I guess that "i" won't solve the problem but mitigate it a bit, so our best bet is going for "ii", isn't it?

[amb26]

@javihernandez - not correct. npm 3 adopts a different, flatter layout during install, which will enable them to be successfully copied later.

[minogian]

@javihernandez No, the problem is not while copying the repository. Robocopy handles long paths just fine. Running npm prune and npm dedupe handles the specific problem you were seeing (ie running the heat command) and an .msi file is built.
It's actually running the .msi file and choosing a path like C:\Program Files (x86)\GPII that won't work

[minogian]

@amb26 The installer produces a standard .msi file and these do not seem to like very long paths nor can they use the \\?\ syntax for overcoming the issue. Also, the registry hack in the superuser.com answer seems dangerous (and it doesn't work anyway in our case). Finally, the Windows patch mentioned in the other link seems irrelevant as I'm testing on the Windows 10 VM.

@amb26 @javihernandez The installer is build using the Windows Installer SDK and the semi-official/Microsoft sanctioned WiX toolset. There is no magic switch that I can turn to make the .msi cope with long paths. Unless we switch to an all custom install solution I can't do much about the problem.

By reading the long threads over at npm/npm#3697 and nodejs/node-v0.x-archive#6960, it seems to me that the only viable solution is switching to npm 3.
I can also confirm that upgrading npm to version 3.10.6 inside the vagrant VM before runningnpm install solves all installer problems (no npm prune and dedupeneeded). Obviously I did not test the whole platform afterwards - I can only confirm that the msi file is built and can be used without problems.

[amb26]

This needs to be updated so that the reference to the gpii-wix-installer can be somehow versioned, rather than just binding to master of the project. Without this, it will be impossible to test particular pull requests in our CI infrastructure. I don't know whether npm is a suitable option for this, but this dependency needs to be parameterised somehow.

[minogian]

@amb26 Would tagging work for versioning the gpii-wix-installer reference?
I could tag the current HEAD revision of gpii-wix-installer as v1.0.0 and then reference that inside the build-installer script.

[gloob]

As defined and decided in the several GPII Windows weekly meetings (https://pad.gpii.net/p/windows-summit-29-7-16-alr4n2a) we are moving to npm3 to fix this and other problems (like node-gyp problem in last windows-10-64bits (0.4.3) vagrant box).

Migration to npm3 (and consequently to node.js 6) is tracked in the jira issue GPII-1929 after successful migration we will fix the issue with the execution of the GPII Windows installer.

[minogian]

I updated the pull request to refer to a tagged commit (v1.0.0) of the gpii-wix-installer repository (see @amb26's comment about versioning references)

[gloob]

@minogian perfect, thanks!

[javihernandez]

ok to test

[gpii-bot]

CI job passed.

[minogian]

I updated the pull request but the current master does not build.
#88 should be merged first.

[gpii-bot]

CI job passed.

[javihernandez]

It looks like this is ready to go. For the record, this addresses the inclusion of the installer into our CI (GPII-1763), all the information, next steps and related issues can be found in https://issues.gpii.net/browse/GPII-1731.