"configure" script for platform detection #611
Conversation
Move most of the platform detection logic out from the Makefile into
a traditional "configure" script, Since the system is unlike to change,
this allows to run (costly) platform detection things only once at the
first build. Incremental builds during development become much faster.
macOS Linux
Before After Before After
make clean in empty dir 3.196 2.858 0.890 1.166
make all from scratch 11.762 12.183 6.226 6.273
make all again with no changes 3.231 0.504 0.936 0.300
make all with one changed file 3.247 0.816 1.101 0.553
make clean after build 4.564 0.939 0.896 0.328
After a while you get *really* annoyed by every make operation taking
usually around three seconds and up to 8 (eight, Carl!) seconds if
caches are cold. Lowering it to sub-second range is more tolerable.
Why so slow?
------------
The main culprit here is Homebrew that really takes its time to locate
OpenSSL installation directory. Thankfully, it does not update the
formula database now, but it did before. Some people complained [1]
about this simple operation taking enormous amount of time but the
maintainer said this is fine and not a bug.
[1]: Homebrew/brew#3097
All other scripting languages combined can be treated as accomplices.
Each individual version check looks cheap on its own, but doing them
altogether takes a couple of seconds.
Surprisingly, the "supported" calls to check C compiler flags are fast
enough so I did not bother moving them out to ./configure (though it's
a task traditionally performed by it). There's also some weird interplay
with AFL there, so that's left for the future.
On "./configure"
----------------
The script follows GNU conventions expected from it [2] though there is
not much configuration involved. We also diverge from the usual approach
to out-of-source builds. The traditional way is to call configure from
the build directory with --srcdir, but we keep using BUILD_PATH to set
the build directory and expect "make" to be call from tree root.
[2]: https://www.gnu.org/prep/standards/html_node/Configuration.html
Note that it is not necessary to explicitly call ./configure before the
build. "make" will check if it has not been called and will configure
the build if necessary. So for most of the users the happy path is still
make
sudo make install
But you can use ./configure if, say, you want a different installation
prefix:
./configure --prefix=/opt/themis
make
sudo make install
The entire script is an exercise in portable POSIX shell scripting.
Don't you dare utter "Au*****ls" in this house.
What are those fancy comment sections?
--------------------------------------
We're (slowly) unifying the makefile structure across projects.
This is the new way™ of doing things.
| --*=*) | ||
| var=${1#--} | ||
| value=${var#*=} | ||
| var=${var%%=*} |
There was a problem hiding this comment.
oh, that is why I hate bash and makefile languages... too much non-alpha symbols in syntax
There was a problem hiding this comment.
You'll enjoy APL then 😉
{↑1 ⍵∨.∧3 4=+/,¯1 0 1∘.⊖¯1 0 1∘.⌽⊂⍵}
|
Okay, no clue why PHP build broke, but it seems that something is wrong with PHP 5.6, not this changeset, and it's not some transient issue in GitHub Actions. |
| Linux) | ||
| set_variable IS_LINUX "true" | ||
| ;; | ||
| *) |
There was a problem hiding this comment.
I'd suggest:
- to specify explicitly pattern for MS:
MINGW*|CYGWIN*|MSYS*) - to throw an exception on unknown platforms
There was a problem hiding this comment.
specify explicitly pattern for MS
IIRC, this one is specifically for MSYS2, not a generic Windows platform. Windows support is in kinda "purgatory" state, since it's not tested automatically, and not regularly tested manually either. So I'd be wary to add new behavior without proper testing.
I'd suggest to keep the current behavior (detect only MSYS2) and add other platforms later, in a more systematic way.
throw an exception on unknown platforms
Hm... Seems like a good idea, but it may break some builds. Currently, if we cannot detect the platform we assume it to be generic UNIX system. This way make will probably work on various BSDs, etc. If we disallow unknown systems then we might break builds for some users that build Themis for platforms that we don't explicitly support yet they still work.
Are we okay with that?
There was a problem hiding this comment.
Sounds reasonable. Thanks.
Since GitHub Actions for PHP install all PHP versions and switch between them for tests, we need to run ./configure after every switch to apply the current default PHP version to the configuration.
|
I have realized why PHP build on GHA broke with PHP 5.6 while CircleCI build is fine. This is because with these changes we no longer detect environment changes on each GHA testing script installs all PHP versions and switches them during the test run. However, PHP 7 is the default one and that's what we detect on the first CircleCI runs PHP tests in isolated environments so PHP versions gets detected there correctly. Well, we can't do much about it as this will be the expected behavior with these changes. It should not affect non-developers as they are unlikely to change environments mid-build. However, we need to be more careful in the future. All hail the tests! |
Move most of the platform detection logic out from the Makefile into a traditional
configurescript, Since the system is unlike to change, this allows to run (costly) platform detection things only once at the first build. Incremental builds during development become much faster.make cleanin empty dirmake allfrom scratchmake allagain with no changesmake allwith one changed filemake cleanafter buildAfter a while you get really annoyed by every make operation taking usually around three seconds and up to 8 (eight, Carl!) seconds if caches are cold. Lowering it to sub-second range is more tolerable.
Why so slow?
The main culprit here is Homebrew that really takes its time to locate OpenSSL installation directory. Thankfully, it does not update the formula database now, but it did before. Some people complained about this simple operation taking enormous amount of time but the maintainer said this is fine and not a bug.
All other scripting languages combined can be treated as accomplices. Each individual version check looks cheap on its own, but doing them altogether takes a couple of seconds.
Surprisingly, the
supportedcalls to check C compiler flags are fast enough so I did not bother moving them out to ./configure (though it's a task traditionally performed by it). There's also some weird interplay with AFL there, so that's left for the future.On ./configure
The script follows GNU conventions expected from it though there is not much configuration involved. We also diverge from the usual approach to out-of-source builds. The traditional way is to call configure from the build directory with
--srcdir, but we keep using BUILD_PATH to setthe build directory and expect
maketo be call from tree root.Note that it is not necessary to explicitly call ./configure before the build. Make will check if it has not been called and will configure the build if necessary. So for most of the users the happy path is still
But you can use ./configure if, say, you want a different installation prefix:
The entire script is an exercise in portable POSIX shell scripting. Don't you dare utter "Au*****ls" in this house.
What are those fancy comment sections?
We're (slowly) unifying the makefile structure across projects. This is the new way™ of doing things.
Checklist