"configure" script for platform detection (#611)

* "configure" script for platform detection Move most of the platform detection logic out from the Makefile into a traditional "configure" script, Since the system is unlike to change, this allows to run (costly) platform detection things only once at the first build. Incremental builds during development become much faster. macOS Linux Before After Before After make clean in empty dir 3.196 2.858 0.890 1.166 make all from scratch 11.762 12.183 6.226 6.273 make all again with no changes 3.231 0.504 0.936 0.300 make all with one changed file 3.247 0.816 1.101 0.553 make clean after build 4.564 0.939 0.896 0.328 After a while you get *really* annoyed by every make operation taking usually around three seconds and up to 8 (eight, Carl!) seconds if caches are cold. Lowering it to sub-second range is more tolerable. Why so slow? ------------ The main culprit here is Homebrew that really takes its time to locate OpenSSL installation directory. Thankfully, it does not update the formula database now, but it did before. Some people complained [1] about this simple operation taking enormous amount of time but the maintainer said this is fine and not a bug. [1]: Homebrew/brew#3097 All other scripting languages combined can be treated as accomplices. Each individual version check looks cheap on its own, but doing them altogether takes a couple of seconds. Surprisingly, the "supported" calls to check C compiler flags are fast enough so I did not bother moving them out to ./configure (though it's a task traditionally performed by it). There's also some weird interplay with AFL there, so that's left for the future. On "./configure" ---------------- The script follows GNU conventions expected from it [2] though there is not much configuration involved. We also diverge from the usual approach to out-of-source builds. The traditional way is to call configure from the build directory with --srcdir, but we keep using BUILD_PATH to set the build directory and expect "make" to be call from tree root. [2]: https://www.gnu.org/prep/standards/html_node/Configuration.html Note that it is not necessary to explicitly call ./configure before the build. "make" will check if it has not been called and will configure the build if necessary. So for most of the users the happy path is still make sudo make install But you can use ./configure if, say, you want a different installation prefix: ./configure --prefix=/opt/themis make sudo make install The entire script is an exercise in portable POSIX shell scripting. Don't you dare utter "Au*****ls" in this house. What are those fancy comment sections? -------------------------------------- We're (slowly) unifying the makefile structure across projects. This is the new way™ of doing things. * Run configure after switching PHP versions Since GitHub Actions for PHP install all PHP versions and switch between them for tests, we need to run ./configure after every switch to apply the current default PHP version to the configuration.
cossacklabs · Mar 27, 2020 · f79cd23 · f79cd23
1 parent 3eeb737
commit f79cd23
Show file tree

Hide file tree

Showing 3 changed files with 325 additions and 99 deletions.
diff --git a/.github/workflows/test-php.yaml b/.github/workflows/test-php.yaml
@@ -59,6 +59,7 @@ jobs:
           sudo update-alternatives --set php-config /usr/bin/php-config5.6
           sudo update-alternatives --set phpize     /usr/bin/phpize5.6
           sudo git clean -fxd src
+          ./configure
           sudo make phpthemis_install
           make test_php
       - name: Run test suite (PHP 7.0)
@@ -68,6 +69,7 @@ jobs:
           sudo update-alternatives --set php-config /usr/bin/php-config7.0
           sudo update-alternatives --set phpize     /usr/bin/phpize7.0
           sudo git clean -fxd src
+          ./configure
           sudo make phpthemis_install
           make test_php
       - name: Run test suite (PHP 7.1)
@@ -77,6 +79,7 @@ jobs:
           sudo update-alternatives --set php-config /usr/bin/php-config7.1
           sudo update-alternatives --set phpize     /usr/bin/phpize7.1
           sudo git clean -fxd src
+          ./configure
           sudo make phpthemis_install
           make test_php
       - name: Run test suite (PHP 7.2)
@@ -86,6 +89,7 @@ jobs:
           sudo update-alternatives --set php-config /usr/bin/php-config7.2
           sudo update-alternatives --set phpize     /usr/bin/phpize7.2
           sudo git clean -fxd src
+          ./configure
           sudo make phpthemis_install
           make test_php
       - name: Run test suite (PHP 7.3)
@@ -95,6 +99,7 @@ jobs:
           sudo update-alternatives --set php-config /usr/bin/php-config7.3
           sudo update-alternatives --set phpize     /usr/bin/phpize7.3
           sudo git clean -fxd src
+          ./configure
           sudo make phpthemis_install
           make test_php
       - name: Run test suite (PHP 7.4)
@@ -104,6 +109,7 @@ jobs:
           sudo update-alternatives --set php-config /usr/bin/php-config7.4
           sudo update-alternatives --set phpize     /usr/bin/phpize7.4
           sudo git clean -fxd src
+          ./configure
           sudo make phpthemis_install
           make test_php
 

diff --git a/Makefile b/Makefile
@@ -14,7 +14,30 @@
 # limitations under the License.
 #
 
-########################################################################
+#===== Early setup =============================================================
+
+# Set default goal for "make"
+.DEFAULT_GOAL := all
+
+# Set shell for target commands
+SHELL = /bin/bash
+
+# Disable built-in rules
+MAKEFLAGS += --no-builtin-rules
+.SUFFIXES:
+
+## build directory
+BUILD_PATH ?= build
+
+# Include system configuration file, creating it if necessary
+-include $(BUILD_PATH)/configure.mk
+
+$(BUILD_PATH)/configure.mk:
+	@./configure
+
+#===== Variables ===============================================================
+
+#----- Versioning --------------------------------------------------------------
 
 # Increment VERSION when making a new release of Themis.
 #
@@ -23,42 +46,51 @@
 VERSION := $(shell test -d .git && git describe --tags || cat VERSION)
 LIBRARY_SO_VERSION = 0
 
-########################################################################
-#
-# Overridable default paths to applications and build/install directories
-#
+#----- Toolchain ---------------------------------------------------------------
 
-CMAKE = cmake
-SHELL = /bin/bash
+CMAKE ?= cmake
 
 CLANG_FORMAT ?= clang-format
 CLANG_TIDY   ?= clang-tidy
 
-INSTALL = install
-INSTALL_PROGRAM = $(INSTALL)
-INSTALL_DATA    = $(INSTALL) -m 644
+INSTALL         ?= install
+INSTALL_PROGRAM ?= $(INSTALL)
+INSTALL_DATA    ?= $(INSTALL) -m 644
 
-BUILD_PATH ?= build
+#----- Build directories -------------------------------------------------------
 
 SRC_PATH = src
 BIN_PATH = $(BUILD_PATH)
 OBJ_PATH = $(BIN_PATH)/obj
 AUD_PATH = $(BIN_PATH)/for_audit
+
 TEST_SRC_PATH = tests
 TEST_BIN_PATH = $(BIN_PATH)/tests
 
+#----- Installation paths ------------------------------------------------------
+
+## installation prefix
 PREFIX ?= /usr/local
 
-prefix       = $(PREFIX)
-exec_prefix  = $(prefix)
-bindir       = $(prefix)/bin
-includedir   = $(prefix)/include
-libdir       = $(exec_prefix)/lib
-jnidir       = $(libdir)
-pkgconfigdir = $(libdir)/pkgconfig
+# Advanced variables for fine-tuning installation paths
+prefix       ?= $(PREFIX)
+exec_prefix  ?= $(prefix)
+bindir       ?= $(prefix)/bin
+includedir   ?= $(prefix)/include
+libdir       ?= $(exec_prefix)/lib
+jnidir       ?= $(libdir)
+pkgconfigdir ?= $(libdir)/pkgconfig
+
+#----- Basic compiler flags ----------------------------------------------------
 
-CFLAGS += -I$(SRC_PATH) -I$(SRC_PATH)/wrappers/themis/ -fPIC
+# Add Themis source directory to search paths
+CFLAGS  += -I$(SRC_PATH) -I$(SRC_PATH)/wrappers/themis/
 LDFLAGS += -L$(BIN_PATH)
+# Not all platforms include /usr/local in default search path
+CFLAGS  += -I/usr/local/include
+LDFLAGS += -L/usr/local/lib
+# Build shared libraries
+CFLAGS  += -fPIC
 
 ########################################################################
 #
@@ -86,80 +118,6 @@ PRINT_WARNING_ = printf "$(WARN_STRING)\n" && printf "$(CMD)\n$$LOG\n"
 BUILD_CMD = LOG=$$($(CMD) 2>&1) ; if [ $$? -ne 0 ]; then $(PRINT_ERROR); elif [ "$$LOG" != "" ] ; then $(PRINT_WARNING); else $(PRINT_OK); fi;
 BUILD_CMD_ = LOG=$$($(CMD) 2>&1) ; if [ $$? -ne 0 ]; then $(PRINT_ERROR_); elif [ "$$LOG" != "" ] ; then $(PRINT_WARNING_); else $(PRINT_OK_); fi;
 
-########################################################################
-#
-# Platform detection macros and default tweaks
-#
-
-UNAME := $(shell uname)
-
-ifeq ($(UNAME),Darwin)
-	IS_MACOS := true
-else ifeq ($(UNAME),Linux)
-	IS_LINUX := true
-else ifeq ($(shell uname -o),Msys)
-	IS_MSYS := true
-endif
-
-ifneq ($(shell $(CC) --version 2>&1 | grep -oi "Emscripten"),)
-	IS_EMSCRIPTEN := true
-endif
-ifneq ($(shell $(CC) --version 2>&1 | grep -E -i -c "clang version"),0)
-	IS_CLANG_COMPILER := true
-endif
-
-SHARED_EXT = so
-
-ifdef IS_MACOS
-SHARED_EXT = dylib
-ifneq ($(SDK),)
-SDK_PLATFORM_VERSION=$(shell xcrun --sdk $(SDK) --show-sdk-platform-version)
-XCODE_BASE=$(shell xcode-select --print-path)
-CC=$(XCODE_BASE)/usr/bin/gcc
-BASE=$(shell xcrun --sdk $(SDK) --show-sdk-platform-path)
-SDK_BASE=$(shell xcrun --sdk $(SDK) --show-sdk-path)
-FRAMEWORKS=$(SDK_BASE)/System/Library/Frameworks/
-SDK_INCLUDES=$(SDK_BASE)/usr/include
-CFLAGS += -isysroot $(SDK_BASE)
-endif
-ifneq ($(ARCH),)
-CFLAGS += -arch $(ARCH)
-endif
-endif
-
-ifdef IS_MSYS
-SHARED_EXT = dll
-endif
-
-ifdef IS_EMSCRIPTEN
-# Recent versions of Emscripten toolchain provide new "emcmake" utility
-# specifically for CMake. Older versions rely on generic "emconfigure".
-ifeq ($(shell which emcmake >/dev/null 2>&1 && echo yes),yes)
-CMAKE = emcmake cmake
-else
-CMAKE = emconfigure cmake
-endif
-endif
-
-# Not all platforms include /usr/local into default search path
-CFLAGS  += -I/usr/local/include
-LDFLAGS += -L/usr/local/lib
-
-########################################################################
-#
-# Detecting installed language runtimes and their versions
-#
-
-PHP_VERSION := $(shell php -r "echo PHP_MAJOR_VERSION;" 2>/dev/null)
-RUBY_GEM_VERSION := $(shell gem --version 2>/dev/null)
-RUST_VERSION := $(shell rustc --version 2>/dev/null)
-GO_VERSION := $(shell which go >/dev/null 2>&1 && go version 2>&1)
-NODE_VERSION := $(shell node --version 2>/dev/null)
-NPM_VERSION := $(shell npm --version 2>/dev/null)
-PIP_VERSION := $(shell pip --version 2>/dev/null)
-PYTHON2_VERSION := $(shell which python2 >/dev/null 2>&1 && python2 --version 2>&1)
-PYTHON3_VERSION := $(shell python3 --version 2>/dev/null)
-
 ########################################################################
 #
 # Select and configure cryptographic engine
@@ -192,10 +150,9 @@ CFLAGS += $(CRYPTO_ENGINE_CFLAGS)
 # Homebrew's OpenSSL instead of the system one by default.
 ifdef IS_MACOS
 	ifeq ($(CRYPTO_ENGINE_PATH),openssl)
-		OPENSSL_PATH := $(shell brew --prefix openssl)
-		ifneq ($(OPENSSL_PATH),)
-			CRYPTO_ENGINE_INCLUDE_PATH = $(OPENSSL_PATH)/include
-			CRYPTO_ENGINE_LIB_PATH = $(OPENSSL_PATH)/lib
+		ifneq ($(HOMEBREW_OPENSSL_PATH),)
+			CRYPTO_ENGINE_INCLUDE_PATH = $(HOMEBREW_OPENSSL_PATH)/include
+			CRYPTO_ENGINE_LIB_PATH = $(HOMEBREW_OPENSSL_PATH)/lib
 		endif
 	endif
 endif
@@ -413,8 +370,6 @@ endif
 # Principal Makefile targets
 #
 
-.DEFAULT_GOAL := all
-
 all: themis_static soter_static themis_shared soter_shared themis_pkgconfig soter_pkgconfig
 	@echo $(VERSION)