- Manual: Hand-made
- Binaries
- Build Configurations
- Build system: /Make/, Ninja, MSBuild, …
- Targets
- Recipes
- Dependencies
- Meta-build system: /GN/, CMake, meson, …
# GCC/Clang on Linux/macOS
$ g++ hello.cpp
$ ./a.out
$ g++ -std=c++20 -Wall hello.cpp -o hello
$ ./hello
# MSVC on Windows
> cl /EHsc /std:c++17 /W3 hello.cpp
> hello.exe
- Common Flags
- Input: bunch of files
- Output:
-o
- Language standard:
-std=c++14
optional - Warning level:
-Wall
best practise optional
g++
andcl
are justcompilertoolchain front-ends- Clang (
clang++
) copies GCC’s flags
- Clang (
A useful
wizardryskill even today.
#include <iostream> // preprocessor
float perimeter(float radius); // compiler
extern float PI; // linker
int main() { // linker
constexpr auto radius = 2.71828f; // compiler
std::cout << perimeter(radius); // compiler + linker
}
- Many tools come together in making a binary
- Preprocessor: glorified find-replace / copy-paste
- Compiler: generates code
- Linker: resolves dependencies
- more…
- Front-end → arguments → Back-end
- Directly or Indirectly
Add | Include Dir (Preprocessor) | Library Dir (Linker) |
---|---|---|
Directly | -I./inc | -L./lib |
Indirectly | -Xpreprocessor -I -Xpreprocessor ./inc | -Xlinker -L -Xlinker ./lib |
- Kinds
- Executable
- Static Library
- Shared Library
- Build lingo: artefacts
$ g++ -std=c++17 -c hello.cpp
$ ls
hello.cpp hello.o
$ nm hello.o
0000000000000000 T greet
> cl /EHsc /c hello.cpp
> dir
hello.cpp hello.obj
> dumpbin /symbols hello.obj
- C++ →
bytenative code; runs on actual CPU- Not interpreted by VM¹
- Java: JVM, Python: CPython, C#: CLR, …
- Not interpreted by VM¹
- Non-portable binary; depends on
- Architecture (arm64, x64, MIPS, PowerPC, …) EXECUTE
- OS (Linux, Windows, macOS, …) LOAD SYSCALLS
- Basic unit of compiled C, C++ and Obj-C
- GCC/Clang:
.o
- MSVC:
.obj
- GCC/Clang:
1: Don’t conflate with entire machine VMs like Hyper-V, VirtualBox, …
# How do I find the OS/architecture of some rogue binary?
# Linux
$ file my_bin
my_bin: ELF 64-bit, x86-64, GNU/Linux 3.2.0, stripped
# macOS
$ file my_bin
my_bin: Mach-O 64-bit executable x86_64
# Windows (MSYS2 or WSL2)
> file my_bin.exe
my_bin.exe: PE32+ executable (console) x86-64, for MS Windows
- Linker expects entry point
- C-family standard:
int main()
- OS alternatives e.g.
WinMain
- C-family standard:
- Static dependencies resolved early build
- Dynamic dependencies resolved late run
- Dependency Components
- Headers (
.h
,.hpp
,.hxx
, …) compiler - Libraries (
.a
,.lib
,.so
,.dll
, …) linker os
- Headers (
- Common Dependencies
- System & third-party e.g. libpng (
png.h
+libpng.a
)
- System & third-party e.g. libpng (
OS Family | Extension | Format |
---|---|---|
Unix | none | Executable & Linkable Format (ELF) |
Windows | .exe | Portable Executable (PE/PE32+) |
macOS | none | Mach object (Mach-O) |
+---------------------+----------+ +--------------+ +--------------+ | | | | | | | | | | | | | | | Application 1 | Static | | Application | | Application | | | Lib A | | 3 | | 4 | | | | | | | | +---------------------+----------+ +------\-------+ +------/-------+ \ / \ / +---------------------+----------+ +----\------------/-----+ | | | | | | | | | | | Application 2 | Static | | Shared Library B | | | Lib A | | | | | | | | +---------------------+----------+ +-----------------------+
$ ar -rcs libTrig.a sin.o cos.o tan.o
$ ar -t libTrig.a
sin.o cos.o tan.o
$ nm libTrig.a
0000000000000000 T sin
0000000000001000 T cos
$ ls -l
80K libTrig.a
20K libmath.a
200K tool.o
$ gcc -o tool tool.o libTrig.a
ld: sin.o: undefined reference to 'add(int, int)'
$ gcc -o tool tool.o -ltrig -lmath
$ ls -l tool
300K tool
- An archive of object files linker
- With interface headers e.g.
trig.h
compiler
- With interface headers e.g.
- Code attached to final executable build
- Static/Compile-time linking by linker
- Dependencies aren’t resolved! build
- Final binary to supply dependency
- Toolchain feature; OS uninvolved
- No entry functions
main()
,DllMain()
, …
No “missing dependencies” error for app | No sweeping updates / fixes |
No version mismatches or Dependency Hell | Every app to rebuild on update |
Single executable; simpler package/install | Disk space (fat binaries, multiple copies)¹ |
Apps may ignore breaking lib version | No on-demand loading / plug-ins |
Library needn’t be backward-compatible | Slower build time for app (strip unused) |
1: Doesn’t apply to Windows; each software brings its own (non-system) libraries
Static Linking Considered Harmful - Ulrich Drepper$ g++ -o tool tool.o
$ ls -l
200K tool.o
200K tool
$ g++ -shared -fPIC {sin,cos,tan}.cpp -o trig.dll -lmath
$ nm trig.dll
0000000000000000 T sin
0000000000001000 T cos ...
$ gcc -o tool tool.o trig.dll
$ ls -l
80K trig.dll
200K tool.o
200K tool
- Single library shared across apps run
- Single copy in memory at runtime
- Static dependencies resolved build
- Need dynamic dependencies at launch
a.dll
→b.dll
→ … 😲 dependency chain
- Final executable contains +code+ only jumps
- Dynamic/run-time linking by OS/loader run
- Expects library presence in right path on
- Launch
- Demand:
dlopen
,LoadLibrary
- Expects library presence in right path on
- Entry functions e.g.
DllMain
OS | Name |
---|---|
Windows | Dynamic Link Libraries (.dll ) |
Linux | Shared Objects (.so ) |
macOS | Dynamic Shared Libraries/Bundles (.dylib / .so ) |
Sweeping updates / fixes | Missing dependencies; failure to launch |
Plug-ins / on-demand loads | Versioning / Dependency Hell |
Toolchain independent; cross-toolchain | OS dependent |
No app rebuilding | Many OS-specific binaries; pkg/install hassle |
Lesser disk footprint | Backward-compatible considerations |
Forced updates breaking app |
# GCC/MinGW on Windows
> g++ -std=c++17 -D_DEBUG hello.cpp -g -O0 -flto -o hello.exe
# MSVC on Windows
> cl /EHsc /std:c++17 /D_DEBUG hello.cpp /Zi /Od /LTCG
- Compiler Flags
- Enable debug symbols:
-g
- Disable optimizations:
-O0
- Enable debug symbols:
- Linker Flags
- Link time optimization:
-flto
- Link time optimization:
- Preprocessor Flags
- Define macros, add include dirs, etc.
-D_DEBUG
→#define _DEBUG
-DPI=3.14
→#define PI 3.14
- List of flags can get long, /really long/
- MSVC: 166 (1 platform, arch-neutral)
- GCC: gazillion (multi-arch, multi-platform 🤯)
Conditional compilation of certain pieces of code.
# 2. Conditional Inclusion
# BUILD.gn
if (is_linux || is_chromeos) {
sources += [
"base_paths_posix.cc"
]
}
// 1. Macro
// C++
#if defined(ENABLE_TAB_TOGGLE)
tab_toggler.init();
#endif
- Features are made of code
- Code can guarded by switches
- Macros
- Conditional inclusion of files
- Binary won’t have omitted feature’s bits
- Unlike command-line-flag-enabled features
Configuration: particular combination of all switches¹.
- Theoretically
m × n
switches (toolchain × software)- Strictly speaking
m x n
isn’t possible
- Strictly speaking
- Switches can be inter-dependant
- Example: turning on PDF might need Print support
- Example: turning on logging for Debug builds
- Manual: tedious and error-prone
- Hampers reproducibility, productivity and maintenance
Emojis | Speech | Plugins | Logging | Debug | Optimization | |
---|---|---|---|---|---|---|
Config1 | ✓ | ✓ | ✓ | ✓ | ||
Config2 | ✓ | ✓ | ✓ |
1: Think: args.gn
$ cd ~/edge/src
$ gn args out/release_x64 --list --short | wc -l
887
$ wc -l < out/release_x64/args.gn
11
$ gn args out/release_x64 --list --short --overrides-only | wc -l
20
$ gn args out/release_x64 --list=crashpad_dependencies
crashpad_dependencies
Current value = "chromium"
From //.gn:51
Overridden from the default = "standalone"
From //third_party/crashpad/crashpad/build/crashpad_buildconfig.gni:19
- Debug
- Disable optimizations
- Keep symbols
- Release
- Enable optimizations
- Strip debug symbols
- Debug − logging (DbgNoLog)
- Release + debug (RelDbg)
- Release + size optimization (RelMinSize)
- …
- First step towards build automation
- Minimal enough to learn important build concepts
- Powerful enough; still used in production code
- Good for quick workouts personally
- Cross-platform, cross-toolchain POSIX standard productivity
- GCC/Clang: GNU
make
, BSDmake
; MSVC:nmake
- Most IDEs support Makefile-based projects
- VS 2019+: UNIX makefile project template
- GCC/Clang: GNU
- Rebuild only changed parts speed dry
- Avoids hand-compiling tedium and mistakes
- Enables build reproducibility in a team
# commonly used flags in variable
CXXFLAGS = -std=c++17 -Wall
LDFLAGS = -flto
biryani: rice.o spices.o
g++ $(LDFLAGS) -o biryani rice.o spices.o
cp biryani ./installer/bin
spices.o: spices.cpp spices.h
g++ $(CXXFLAGS) -o spices.o -c spices.cpp
rice.o: rice.cpp rice.h utensils.h spices.h
g++ $(CXXFLAGS) -o rice.o -c rice.cpp
clean:
rm -rf biryani *.o
.PHONY: clean
- Add
Makefile
at project root with rules - Target: final artefact expected
- Considered outdated if older than a dependency
- Dependency: ingredients needed to make target
- Recipe: snippet making target from dependencies
- Target outdated ¹? Re-run recipe!
make
: build first targetmake TARGET
: only buildTARGET
(and its dependencies)
- Golden Rule
- Every target’s recipe should update file naming it.
- Add exceptions to
.PHONY
; always outdated
1: older than a dependency
# commonly used flags in variable
CXXFLAGS = $(USERFLAGS) -std=c++17 -Wall
LDFLAGS = -flto # LTO ON
LDLIBS = -lz -lmath # libMath.a, libZ.a
biryani: rice.o spices.o
$(CC) $(LDFLAGS) -o $@ $^ $(LDLIBS)
cp biryani ./installer/bin
spices.o: spices.cpp spices.h
rice.o: rice.cpp rice.h utensils.h spices.h
doc: ref.html tutorial.html
# Pattern rule
%.html: %.md
pandoc -o $@ $<
# e.g. pandoc -o ref.html ref.md
clean:
$(RM) biryani *.o
commit:
git add $(wildcard *.cpp *.h)
git commit
.PHONY: clean doc commit
- Power to build engineers
- Override settings without editing
Makefile
make CC=clang++
: override toolchain to ClangUSER_FLAGS='-DMY_SHINY_FEATURE=ON -O3' make
- Override settings without editing
- Special variables
- target
$@
, dependencies$^
, first dep:$<
- target
- Pattern rule: map
.X
→.Y
- Make knows how to build
.o
from.cpp
,.c
, etc.- Implicit rule:
$(CC) -c $(CFLAGS) -o $@ $<
- Implicit rule:
- Make isn’t language-specific
make doc
builds documentation using Pandocmake install
: bunch of copies
.POSIX:
COMPILER_FLAGS = -Wall -Werror -pedantic -pedantic-errors
CXXFLAGS = -std=c++17 $(COMPILER_FLAGS)
all: debug release
debug: CXXFLAGS += -g -O0 -D_DEBUG -DDEBUG
debug: hello
release: CXXFLAGS += -O2 -DNDEBUG
release: hello
hello: hello.swift MyCMod/adder.o
swiftc -I . -o $@ $<
MyCMod/adder.o: MyCMod/adder.cpp MyCMod/adder.h
clean:
$(RM) hello MyCMod/adder.o
.PHONY: all clean
- Separate debug and release targets
- Per-target variable values
make debug
andmake release
make
to build both- Convention: Make an
all
target
- Convention: Make an
- Complexity ∝ Configurations × Dependencies
- Natural to any build system
- No on writes
build.ninja
- Ninja’s introduction calls this out!
- A low-level but fast make system
“[…] designed to have its input files generated by a higher-level build system. Ninja build files are human-readable but not especially convenient to write by hand.”
- A generator of build/project files
- Scriptability
- Run code based on environment/parameters and generate
- Copy resources, pre-/post-tasks, make installer package
- Multi-language support
- Ant: Java, rake: Ruby, Cargo: Rust, …
- Cross-platform, multi-IDE support
- Natural evolution of build systems
- Best of both worlds
- CLI: Build automation, speed, correctness
- GUI: Developer-friendly, wider adoption
executable("img_view") { # target
sources = [
"window.cpp",
"filter.cpp",
]
cflags = [ "-Wall" ] # flags
defines = [ "USE_GPU=1" ] # feature macros
include_dirs = [ "./inc" ]
dependencies = [
":libpng", # in-file
# ‘core’ under third_party/animator/BUILD.gn
"//third_party/animator:core" # qualified
# ‘markdown’ under third_party/markdown/BUILD.gn
"//third_party/markdown" # implicit
]
if (is_win) {
sources += [ "d3d11.cpp" ]
sources -= [ "window.cpp" ]
ldflags = [ "/SUBSYSTEM:WINDOWS",
"/DELAYLOAD: d3d11.dll" ]
}
}
static_library("libpng") {
sources = [
"encoder.c",
"decoder.c",
]
public_deps = [
"//third_party/boost:file_io"
]
}
- 5 target types for 5 binaries/artefacts
executable
,static_library
,shared_library
loadable_module
,source_set
rare
- Often used properties of targets
sources
: define (= [ … ]
), add (+=
) or remove (-=
)cflags
/ldflags
: compiler or linker flagsdefines
: (feature) macros
- Labels: name of dependency graph node e.g.
":base"
- Targets, Configurations, Toolchains
- Core ideas from Make
- Targets, Dependencies, Flags, Macros
Executable | Static | Shared | Loadable Module | Source Set | |
---|---|---|---|---|---|
Windows | .exe | .lib | .dll | .dll | .obj |
Linux | none | .a | .so | .so | .o |
macOS | none | .a | .dylib | .so | .o |
# A can use B and C but not super_secret
executable("A") {
deps = [ ":B" ]
}
shared_library("B") {
public_deps = [ ":C" ]
deps = [ ":super_secret" ]
# link no code from evil directory
assert_no_deps = [ "//evil/*" ]
}
- Dependency chain:
A
→B
→C
dependencies
:B
can include/useC
;A
can’tpublic_deps
: A can includeC
too
- This is recursive!
- Public or Private?
B
should publicly depend onC
if it’s part of interfaceB
- Private dependency if it’s just implementation detail
- Shared Libraries
- Final target links to all publicly dependent shared libraries
- Static Libraries don’t resolve dependencies anyway
- Link both
deps
andpublic_deps
to final target - Final target can include from
public_deps
- Link both
assert_no_deps
: disallow targets from linking
//.gn
: defines project root; seegn help dotfile
//build/config/BUILDCONFIG.gn
: global variables and default settings
declare_args() {
enable_command_line = false
use_opengl = true
assert(!(use_opengl && enable_command_line),
"Can’t use OpenGL and terminal together")
}
config("memory_tagging") {
if (current_cpu == "arm64" && is_linux) {
cflags = [ "-march=armv8-a+memtag" ]
}
}
executable("img_view") {
if (use_opengl) {
ldflags += [ "/DELAYLOAD: opengl32.dll" ]
}
configs += [ ":memory_tagging" ]
}
shared_library("cpu_filters") {
sources = [ "shaders.cpp" ]
configs += [ ":memory_tagging" ]
if (use_opengl) {
# using GPU, skip tagging CPU memory
configs -= [ ":memory_tagging" ]
}
}
declare_args
: define arguments for your target- Set in
args.gn
, command-line or toolchain args
- Set in
config
: distil common configuration for reusepublic_config
to propagate up the dependency chainA
inheritspublic_configs
ofC
too
all_dependent_configs
: force configs on dependants rare- Forced on target, its dependents, its dependents …
- Can’t remove (
-=
) these configs
data
: Runtime data dependencies- List files/dirs required to run target
- Paths interpreted relative to current build file
- e.g. strings compiled into binary, XML, …
data_deps
: non-linked runtime dependencies- Built and available for use
- Generally plugins or helper programs
- List a target’s runtime data dependencies
gn desc TARGET
lists inruntime_deps
section- Get for many:
gn --runtime-deps-list-file=INPUT
OUTPUT.runtime_deps
in target’s output directory
action("run_this_guy_once") {
script = "doprocessing.py"
sources = [ "my_configuration.txt" ]
outputs = [ "$target_gen_dir/some_output.txt" ]
# doprocessing.py imports this script; rebuild if it changes
inputs = [ "helper_library.py" ]
# root_build_dir is script’s working dir
args = [ "-i", rebase_path(inputs[0], root_build_dir),
rebase_path(outputs[0], root_build_dir) ]
}
copy("mydll") {
sources = [ "mydll.dll" ]
outputs = [ "$target_out_dir/mydll.dll" ]
}
Useful for pre-/post-build tasks
action
: target to run script onceaction_forach
: run over set of filescopy
: target to copy files- Cross platform abstraction
$ cd out/debug_x64
$ gn ls . '//base/*' # list all base targets
//base:base
//base:base_paths
//base:base_static
//base:build_date
//base:build_utf8_validator_tables
//base:check_example
//base:debugging_flags
//base:i18n
$ gn ls . # list all targets under all paths
# list only static libraries under //base; --type understand all 5 artefacts
$ gn ls . --type=static_library '//base/*'
//base:base
//base:base_static
//base:i18n
# ※ get the actual target to feed ninja ※
$ gn ls . --type=static_library --as=output '//base/*'
obj/base/libbase.a
obj/base/libbase_static.a
obj/base/libbase_i18n.a
$ autoninja obj/base/libbase_static.a # build only libbase_static.a
# ※ what if I want to build just one .cc? drop to ninja level ※
$ ninja -t targets all > all_targets.txt
$ grep 'browser_window_ring' all_targets.txt # Windows: findstr /srip /C:
obj/chrome/browser/ui/ui/browser_window_ring_touch_bar.o: objcxx
obj/chrome/test/unit_tests/browser_window_ring_touch_bar_unittest.o: objcxx
$ autoninja obj/chrome/browser/ui/ui/browser_window_ring_touch_bar.o
# Why can’t I include a header from dependency X? X isn’t a public_dep.
$ gn path . --public //components/history/content/browser //chrome/browser
No public paths found between these two targets.
# find path and depend on a target; include headers
$ gn path . //cc/base //content/browser
//content/browser:browser --[public]-->
//services/viz/public/mojom:mojom --[public]-->
//cc/paint:paint --[public]-->
//cc/base:base
# print dependency tree
$ gn desc . //tools/gn deps --tree
//base:base
//base:base_paths
//base:base_static
//base:build_date
//base:copy_dbghelp.dll
//base:debugging_flags
//base/allocator:allocator
//base/allocator:allocator_shim
//base/allocator:prep_libc
# where did that flag come from?
$ gn desc . //base cflags --blame
From //build/config/compiler:default_optimization
(Added by //build/config/BUILDCONFIG.gn:456)
/Od
/Ob0
/RTC1
From //build/config/compiler:default_symbols
(Added by //build/config/BUILDCONFIG.gn:457)
/Zi
gn check .
ERROR at //base/files/file_path.cc
#include "sql/statement.h"
^--------------
It is not in any dependency of
//base:base
The include file is in the target(s):
//sql:sql
which should somehow be reachable.
gn help
: built-in helpgn help ls
,gn help root_out_dir
gn ls
: list targetsgn desc
: describe targets- Try
--tree
and--blame
- Try
gn path
: dependency path from two targetsgn args
: query current build’s argumentsgn clean
: keep onlyargs.gn
and Ninja files
template("grit") {
...
}
grit("components_strings") {
source = "components.grd"
output = [ ... ]
}
template("component") {
if (is_component_build) {
_component_mode = "shared_library"
} else if (defined(invoker.static_component_type)) {
assert(invoker.static_component_type == "static_library" ||
invoker.static_component_type == "source_set")
_component_mode = invoker.static_component_type
} else if (!defined(invoker.sources) || invoker.sources == []) {
# When there are no sources defined, use a source set to avoid creating
# an empty static library (which generally don't work).
_component_mode = "source_set"
} else {
_component_mode = "static_library"
}
}
component("base") {
# sources, flags, etc.
}
- Create your own target type
- From 5 primitive types class
- Use
.gni
files toimport
- Shared variable and template
- Popular Templates
component
(//build/config/BUILDCONFIG.gn
)- Shared library for component builds like
debug
- Shared library for component builds like
msvc_toolchain
(//build/toolchain/win/BUILD.gn
)clang_toolchain
(//build/toolchain/gcc_toolchain.gni
)apple_toolchain
(//build/toolchain/apple/toolchain.gni
)
- GN usable outside Chromium too
- Work > generic meta build systems (like CMake, premake, etc.)
- Refer GN’s simple_build example
- Define
.gn
at project rootgn help dotfile
- Define configurations:
//build/config/BUILDCONFIG.gn
- Global variables (
is_win
,{target,host}_os
,{target,host}_cpu
, …) - Defaults for targets:
gn help set_defaults
set_defaults(static_library) { configs = [ ":def_flags", ":optimize" ] }
- Global variables (
- Define toolchain(s)
- Supports C, C++, Rust, Objective-C and Swift
# gn.googlesource.com/gn/+/main/examples/simple_build/build/toolchain/BUILD.gn
toolchain("gcc") {
tool("cc") {
command = "gcc {{defines}} {{include_dirs}} {{cflags}} {{cflags_c}} -c {{source}} -o {{output}}"
depsformat = "gcc"
description = "CC {{output}}"
outputs =
[ "{{source_out_dir}}/{{target_output_name}}.{{source_name_part}}.o" ]
}
tool("cxx") {
description = "CXX {{output}}"
# ... snipped ...
}
tool("alink") {
command = "rm -f {{output}} && ar rcs {{output}} {{inputs}}"
description = "AR {{target_output_name}}{{output_extension}}"
outputs =
[ "{{target_out_dir}}/{{target_output_name}}{{output_extension}}" ]
default_output_extension = ".a"
output_prefix = "lib"
}
tool("solink") {
description = "SOLINK $soname"
# ... snipped ...
}
tool("link") {
outfile = "{{target_output_name}}{{output_extension}}"
# reponse file to hold command until successful execution
rspfile = "$outfile.rsp"
rspfile_content = "{{inputs}}"
command = "g++ {{ldflags}} -o $outfile @$rspfile {{solibs}} {{libs}}"
description = "LINK $outfile"
default_output_dir = "{{root_out_dir}}"
outputs = [ outfile ]
}
tool("stamp") {
command = "touch {{output}}"
description = "STAMP {{output}}"
}
tool("copy") {
command = "cp -af {{source}} {{output}}"
description = "COPY {{source}} {{output}}"
}
}
- Identifier Label
- Global variables
OS
,CPU
, etc. - Specify tools of chain
- {C, C++} compiler
- linker
- stamp
- copy
set_default_toolchain
if > 1- Under
//build/toolchain
- GNU Compile Collection Documentation
- Microsoft MSVC compiler Reference
- GNU Makefile Documentation
- MakefileTutorial.com friendly
- GN Documentation
- Quick Start, Reference, Style Guide, Language
- Using GN Build (2015)
- Chromium codebase
- StackOverflow.com
- reveal.js, stunning HTML5 presentations
- Org-mode, your life in plain text
- Emacs, text editor extraordinaire
- org-re-reveal, org → reveal converter