Upgrade to OpenSSL-1.0.2

[indutny]

See https://www.openssl.org/docs/ssl/SSL_CTX_set_cert_cb.html

Basically, we will be able to eliminate our hello parser and use new APIs for async OCSP/SNI and stuff like that.

cc @bnoordhuis

[jbergstroem]

I started porting yesterday. The import is pretty simple but since I can't find any traces of google incorporating 1.0.2, the gyp-file needs work. I haven't even started looking at asm/yet..

[indutny]

@jbergstroem cool! Just FYI, this .gyp file is barely related to the one that was done by Google team. The comment above it is mostly a legacy thing, we rewrote it a couple of times.

[jbergstroem]

See #723 for work in progress by @shigeki.

[shigeki]

I missed this issue. Yes, I'm working on it. The current branch of openssl-1.0.2 supports only Linux and MacOS and I will do work on Windows next, but I don't have an arm environment for development and tests. I hope someone helps it.

[jbergstroem]

@shigeki just wanted people visiting that issue to check your work out. I'm doing some testing on FreeBSD at the moment.

[indutny]

@shigeki need any help on this? I'd like to ship it as soon as possible ;)

[indutny]

Mostly because of http://blog.chromium.org/2015/02/hello-http2-goodbye-spdy-http-is_9.html and https://www.openssl.org/docs/ssl/SSL_CTX_set_cert_cb.html

[shigeki]

@indutny asm was done for x64 in linux and macos. I will work windows today but I don't have arm and others. ALPN has already been done based on Node-0.11 as in shigeki@5d6c147 .

@chrisdickinson @rvagg How can I use jenkins-iojs.nodesource.com for build testing?

[rvagg]

@shigeki I'll try and get around to adding you today to CI, I have a list of people I need to add

[indutny]

@shigeki there is no need for ALPN patch, this feature is in OpenSSL-1.0.2 anyway.

[shigeki]

@indutny This patch uses ALPN API in OpenSSL-1.0.2 and add options.ALPNProtocols. Openssl-1.0.2 was applied to Node-v0.11 by git submodule without using gyp. I think this can be applied to io.js after upgrading openssl-1.0.2

[indutny]

@shigeki aaah, I see now. You was talking about node.js support, not about patching OpenSSL itself. Good job, man. Let's move it forward! ;)

[shigeki]

@indutny Upgrading to openssl-1.0.2 is almost done in https://github.com/shigeki/io.js/tree/WIP_upgrade_openssl102 except testing on arm. But there is a build issue of x86-win32-masm asm with using ml.exe as reported in http://rt.openssl.org/Ticket/Display.html?id=3650&user=guest&pass=guest . A patch to fix this was shown but not merged yet. And I also found in the issue tracker that openssl team strongly recommended to use nasm. There is no error when we use nasm.

Upgrading of openssl cannot be finished without solving this issue, so now we have 4 options to go as

1. Wait for the upstream to be fixed and released.
2. The patch is applied to iojs for a tentative fix and use masm(ml.exe).
3. Use no-asm for x86-win32.
4. Move to use [EDIT nasm] as strongly recommended by openssl team .

I prefer 2 or 3 for now. I've confirmed both is working well and the above my WIP branch was worked on 2. Then, we need to discuss if we move to use nasm in the future. Do you have any thoughts?

[indutny]

I'm -1 for 3 and +1 for 2.

[shigeki]

@indutny Thanks. I continue to work on 2. I'd like to have a directory to store this kind of private patches for a third party library somewhere in iojs repository so as not to miss it in the future. I've spent some time to fix TLS tests on Windows as your patch of ab71223 was missed to be applied.

@rvagg I'm waiting for my account for testing on arm. Did you already sent it to me?

[indutny]

@shigeki I usually just use git log deps/openssl to reapply all patches.

[shigeki]

@indutny Yes, that's the way we need now. I just think of some kinds of http://src.chromium.org/chrome/trunk/deps/third_party/openssl/patches.chromium/ but patches should be already applied as it is not a good idea to apply patches during build.

[indutny]

@shigeki I'm -1 for this, don't see any point in duplicating information that is already available in git tree.

[indutny]

@shigeki I wonder if we could actually use nasm on windows. @bnoordhuis what are your thoughts on this?

[indutny]

@shigeki may I ask you to borrow the assembly stuff from indutny/bud@be4af45 if it works for you? (I'm in process of testing builds on various platforms)

[indutny]

UPDATE: It works on ia32/x86_64 OSX. Testing it on ubuntu.

[indutny]

UPDATE: Fixed ubuntu-ia32: indutny/bud@29c5e01

[bnoordhuis]

I wonder if we could actually use nasm on windows.

If by 'use' you mean 'gyp-ify, bundle and build', I guess that could work. It's pretty easy to build and it's license compatible (BSD.)

[indutny]

Yes, exactly.

UPDATE: ia32/x86_64 Linux works.

[indutny]

UPDATE: indutny/bud@29c5e01

[indutny]

CI: https://jenkins-iojs.nodesource.com/job/iojs+any-pr+multi/183/

[shigeki]

@indutny I've changed asm/Makefile largely in 019616bc9d609d429f2caf7441a97fe50e2f1463 so as to generate exact same asm files as those generated by Configure of openssl in various platforms. there are no diffs as far as I've checked. I summarized my changes in Makefile into 5 point as written in the commit log below.

    deps: update asm Makefile for openssl-1.0.2

    This includes following changes,
     - Updated asm files for each platforms which are required in
     openssl-1.0.2.
     - Some perl files need CC and ASM envs. Added a check if these envs
     exist. Followed asm files are to be generated with CC=gcc and
     ASM=nasm on Linux.
     - Added new 32bit targets/rules with a sse2 flag (OPENSSL_IA32_SSE2)
     to generate asm to use SSE2.
     - Generating sha512 asm files in x86_64 need output filename which
     has 512. Added a new rule and target directory of ~~asm/sha256~~ [Edit: fixed to] asm/sha512 to store
     them.
     - PERLASM_SCHEME of linux-armv4 is `void` as defined in openssl
     Configure. Changed its target/rule and all directories are moved from
     arm-elf-gas to arm-void-gas.

asm/sha512/ directory needs from the 4th point in above as https://github.com/shigeki/io.js/blob/upgrade_openssl102/deps/openssl/openssl/crypto/sha/asm/sha512-x86_64.pl#L137 . The new rule generates new asm files of sha512 by without using stdout but its filename args.

I got a Rasp Pi2 last night and can work asm target today. I can finish it soon so let's finish #723 before upgrading.