-
Notifications
You must be signed in to change notification settings - Fork 29.7k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
This document is intended to describe the procedure to upgrade openssl from 1.0.1m to 1.0.2a in io.js. Fixes: #589 PR-URL: #1389 Reviewed-By: Fedor Indutny <[email protected]> Reviewed-By: Ben Noordhuis <[email protected]>
- Loading branch information
Shigeki Ohtsu
committed
Apr 14, 2015
1 parent
deb9d23
commit 4830b4b
Showing
2 changed files
with
191 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,191 @@ | ||
| ## How to upgrade openssl library in io.js | ||
|
|
||
| This document describes the procedure to upgrade openssl from 1.0.1m | ||
| to 1.0.2a in io.js. | ||
|
|
||
|
|
||
| ### Build System and Upgrading Overview | ||
| The openssl build system is based on the `Configure` perl script in | ||
| `deps/openssl/openssl`. For example, running `Configure linux_x86-64` | ||
| in the openssl repository generates `Makefile` and `opensslconf.h` for | ||
| the linux_x86_64 target architecture. | ||
|
|
||
| The `Makefile` contains the list of asm files which are generated by | ||
| perl scripts during build so that we can get the most of use of the | ||
| hardware performance according to the type of cpus. | ||
|
|
||
| `Configure TABLE` shows various build parameters that depend on each | ||
| os and arch. | ||
|
|
||
| In io.js, build target is defined as `--dest-os` and `--dest-cpu` in | ||
| configure options which are different from the one that is defined in | ||
| openssl and it's build system is gyp that is based on python, | ||
| therefore we cannot use the openssl build system directly. | ||
|
|
||
| In order to build openssl with gyp in iojs, files of opensslconf.h and | ||
| asm are generated in advance for several supported platforms. | ||
|
|
||
| Here is a map table to show conf(opensslconf.h) and asm between | ||
| the openssl target and configuration parameters of os and cpu in iojs. | ||
| The tested platform in CI are also listed. | ||
|
|
||
| | --dest-os | --dest-cpu | conf | asm | openssl target | CI | | ||
| |---------- |----------- |----- |----- |------------------- |--- | | ||
| | linux | ia32 | o | o |linux-elf | o | | ||
| | linux | x32 | o | x(*2)|linux-x32 | x | | ||
| | linux | x64 | o | o |linux-x86_64 | o | | ||
| | linux | arm | o | o |linux-arm | o | | ||
| | linux | arm64 | o | o |linux-aarch64 | o | | ||
| | mac | ia32 | o | o |darwin-i386-cc | - | | ||
| | mac | x64 | o | o |darwin64-x86_64-cc | o | | ||
| | win | ia32 | o | o(*3)|VC-WIN32 | x | | ||
| | win | x64 | o | o |VC-WIN64A | o | | ||
| | solaris | ia32 | o | o |solaris-x86-gcc | o | | ||
| | solaris | x64 | o | o |solaris64-x86_64-gcc| o | | ||
| | freebsd | ia32 | o | o |BSD-x86 | o | | ||
| | freebsd | x64 | o | o |BSD-x86_64 | o | | ||
| | openbsd | ia32 | o | o |BSD-x86 | x | | ||
| | openbsd | x64 | o | o |BSD-x86_64 | x | | ||
| | others | ia32 | x(*1)| o | - | x | | ||
| | others | x64 | x(*1)| o | - | x | | ||
| | others | arm | x(*1)| o | - | x | | ||
| | others | arm64 | x(*1)| o | - | x | | ||
| | others | others | x(*1)| x(*2)| - | x | | ||
|
|
||
| - (*1) use linux-elf as a fallback configuration | ||
| - (*2) no-asm used | ||
| - (*3) currently masm (Microsoft Macro Assembler) is used but it's no | ||
| longer supported in openssl. We need to move to use nasm or yasm. | ||
|
|
||
| All parameters such as sources, defines, cflags and others generated | ||
| in openssl Makefile are written down into `deps/openssl/openssl.gypi`. | ||
|
|
||
| The header file of `deps/openssl/openssl/crypto/opensslconf.h` are | ||
| generated by `Configure` and varies on each os and arch so that we | ||
| made a new `deps/openssl/config/opensslconf.h`, where it includes each | ||
| conf file from `deps/openssl/config/archs/*/opensslconf.h` by using | ||
| pre-defined compiler macros. This procedure can be processed | ||
| automatically with `deps/openssl/config/Makefile` | ||
|
|
||
| Assembler support is one of the key features in openssl, but asm files | ||
| are dynamically generated with | ||
| `deps/openssl/openssl/crypto/*/asm/*.pl` by perl during | ||
| build. Furthermore, these perl scripts check the version of assembler | ||
| and generate asm files according to the supported instructions in each | ||
| compiler. | ||
|
|
||
| Since perl is not a build requirement in iojs, they all should be | ||
| generated in advance and statically stored in the repository. We | ||
| provide two sets of asm files, one is asm_latest(avx2 and addx | ||
| supported) in `deps/openssl/asm` and the other asm_obsolete(without | ||
| avx1/2 and addx) in `deps/openssl/asm_obsolute`, which depends on | ||
| supported features in assemblers. Each directory has a `Makefile` | ||
| to generate asm files with perl scripts in openssl sources. | ||
|
|
||
| `configure` and gyp check the version of assemblers such as gnu | ||
| as(gas), llvm and Visual Studio. `deps/openssl/openssl.gypi` | ||
| determines what asm files should be used, in which the asm_latest | ||
| needs the version of gas >= 2.23, llvm >= 3.3 or MSVS_VERSION>='2012' | ||
| (ml64 >= 12) as defined in | ||
| https://github.com/openssl/openssl/blob/OpenSSL_1_0_2-stable/crypto/sha/asm/sha512-x86_64.pl#L112-L129, | ||
| otherwise asm_obsolete are used. | ||
|
|
||
| The following is the detail instruction steps how to upgrade openssl | ||
| version from 1.0.1m to 1.0.2a in iojs. | ||
|
|
||
| ### 1. Replace openssl source in `deps/openssl/openssl` | ||
| Remove old openssl sources in `deps/openssl/openssl` . | ||
| Get original openssl sources from | ||
| https://www.openssl.org/source/openssl-1.0.2a.tar.gz and extract all | ||
| files into `deps/openssl/openssl` . | ||
|
|
||
| ### 2. Apply private patches | ||
| There are three kinds of private patches to be applied in openssl-1.0.2a. | ||
|
|
||
| - The two fixes of assembly error on ia32 win32. masm is no longer | ||
| supported in openssl. We should move to use nasm or yasm in future | ||
| version of iojs. | ||
|
|
||
| - The fix of openssl-cli built on win. Key press requirement of | ||
| openssl-cli in win causes timeout failures of several tests. | ||
|
|
||
| - Backport patches for alt cert feature from openssl-1.1.x. Root certs | ||
| of 1024bit RSA key length were deprecated in io.js. When a tls | ||
| server has a cross root cert, io.js client leads CERT_UNTRUSTED | ||
| error because openssl does not find alternate cert chains. This fix | ||
| supports its feature but was made the current master which is | ||
| openssl-1.1.x. We backported them privately into openssl-1.0.2 on | ||
| iojs. | ||
|
|
||
| ### 3. Replace openssl header files in `deps/openssl/openssl/include/openssl` | ||
| all header files in `deps/openssl/openssl/include/openssl/*.h` are | ||
| symbolic links in the distributed release tar.gz. They cause issues in | ||
| Windows. They are replaced into the files to include a real header | ||
| file such as | ||
| ```` | ||
| #include "../../crypto/aes/aes.h" | ||
| ```` | ||
| ### 4. Change `opensslconf.h` so as to fit each platform. | ||
| The opensslconf.h in each target was created in advance by typing | ||
| `deps/openssl/openssl/Configure {target}` and copied | ||
| into `deps/openssl/conf/archs/{target}/opensslconf.h`. | ||
| `deps/openssl/conf/openssconf.h` includes each file according to its | ||
| target by checking pre-defined compiler macros. These can be generated | ||
| by using `deps/openssl/conf/Makefile` | ||
|
|
||
| We should remove OPENSSL_CPUID_OBJ define in opensslconf.h because it | ||
| causes build error when --openss-no-asm option is specified. Instead, | ||
| the OPENSSL_CPUID_OBJ is defined in `deps/openssl/openssl.gypi` | ||
| according to the configure options. | ||
|
|
||
| One fix of opensslconf.h is needed in 64-bit MacOS. | ||
| The current openssl release does not use RC4 asm since it explicitly | ||
| specified as `$asm=~s/rc4\-[^:]+//;` in | ||
| https://github.com/openssl/openssl/blob/OpenSSL_1_0_1-stable/Configure#L584 | ||
| But iojs has used RC4 asm on MacOS for long time. Fix type of RC4_INT | ||
| into `unsigned int` in opensslconf.h of darwin64-x86_64-cc to work on | ||
| the RC4 asm. | ||
|
|
||
| ### 5. Update openssl.gyp and openssl.gypi | ||
| Sources, cflags and define parameters that depends on each target can | ||
| be obtained via `Configure TABLE`. Its list is put in the table of | ||
| [define and cflags changes in openssl-1.0.2a](openssl_define_list.pdf) | ||
|
|
||
| There is no way to verify all necessary sources automatically. We can | ||
| only carefully look at the source list and compiled objects in | ||
| Makefile of openssl and compare the compiled objects that stored | ||
| stored under `out/Release/obj.target/openssl/deps/openssl/' in iojs. | ||
|
|
||
| ### 6. ASM files for openssl | ||
| We provide two sets of asm files. One is for the latest assembler | ||
| and the other is the older one. | ||
|
|
||
| ### 6.1. asm files for the latest compiler | ||
| This was made in `deps/openssl/asm/Makefile` | ||
| - Updated asm files for each platforms which are required in | ||
| openssl-1.0.2a. | ||
| - Some perl files need CC and ASM envs. Added a check if these envs | ||
| exist. Followed asm files are to be generated with CC=gcc and | ||
| ASM=nasm on Linux. See | ||
| `deps/openssl/openssl/crypto/sha/asm/sha512-x86_64.pl` | ||
| - Added new 32bit targets/rules with a sse2 flag (OPENSSL_IA32_SSE2) | ||
| to generate asm for use SSE2. | ||
| - Generating sha512 asm files in x86_64 need output filename which | ||
| has 512. Added new rules so as not to use stdout for outputs. | ||
| - PERLASM_SCHEME of linux-armv4 is `void` as defined in openssl | ||
| Configure. Changed its target/rule and all directories are moved | ||
| from arm-elf-gas to arm-void-gas. | ||
| - add a new rule for armv8 asm generation | ||
|
|
||
| With export environments of CC=gcc and ASM=nasm, then type make | ||
| command and check if new asm files are generated. | ||
|
|
||
| ### 6.2.asm files for the older compiler | ||
| For older assembler, the version check of CC and ASM should be | ||
| skipped in generating asm file with perl scripts. | ||
| Copy files from `deps/openssl/asm` into | ||
| `deps/openssl/asm/asm_obsolete` and change rules to generate asm files | ||
| into this directories and remove the check of CC and ASM envs. | ||
|
|
||
| Without environments of CC and ASM, then type make command and check | ||
| if new asm files for older compilers are generated. |
Binary file not shown.