Skip to content

Commit

Permalink
Speed up build by avoiding build config duplication
Browse files Browse the repository at this point in the history
Compile substantially all of workerd in the bazel 'target' configuration
instead of 'exec', which allows us to avoid compiling major components
twice.
This should offer a large performance improvement, especially when
rebuilding after a V8 update or building all targets and drive down the
worst-case CI build time significantly.
Bazel supports an exec configuration for building code that needs to run
on the host machine during the build, e.g. tools used to generate source
files or type definitions. When not cross-compiling exec and target will
be the same, but bazel does not offer a flag to use just one configuration
when the configurations are identical (bazelbuild/bazel#14848),
making manual changes necessary.
Bazel makes it very difficult to set the config for a target at build
time, so default to using cfg "target" globally for now.
  • Loading branch information
fhanau committed Jun 14, 2023
1 parent 6535767 commit feed276
Show file tree
Hide file tree
Showing 4 changed files with 469 additions and 1 deletion.
1 change: 1 addition & 0 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,7 @@ git_repository(
"//:patches/v8/0008-Disable-bazel-whole-archive-build.patch",
"//:patches/v8/0009-Make-v8-Locker-automatically-call-isolate-Enter.patch",
"//:patches/v8/0010-Add-an-API-to-capture-and-restore-the-cage-base-poin.patch",
"//:patches/v8/0011-Use-target-cfg.patch",
],
)

Expand Down
37 changes: 37 additions & 0 deletions build/run_binary_target.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Workaround for bazel not supporting a shared exec and target configuration, even when they are
# identical. https://github.com/bazelbuild/bazel/issues/14848
# Derived from the tensorflow project (https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/api/generator/api_gen.bzl)
# See https://github.com/tensorflow/tensorflow/issues/60167 for discussion

def _run_binary_target_impl(ctx):
tool = ctx.attr.tool[DefaultInfo].files_to_run.executable
flags = [ ctx.expand_location(a) if "$(location" in a else a for a in ctx.attr.args ]

cmd = " ".join([tool.path] + flags)
ctx.actions.run_shell(
inputs = ctx.files.srcs,
outputs = ctx.outputs.outs,
tools = [tool],
use_default_shell_env = True,
command = cmd,
)

run_binary_target = rule(
implementation = _run_binary_target_impl,
attrs = {
"outs": attr.output_list(mandatory = True),
"srcs": attr.label_list(allow_files = True),
"args": attr.string_list(),
# Setting the configuration to "target" to avoid compiling code used in both this
# generator-like target and regular targets twice. For cross-compilation this would need to
# be set to "exec".
# Unfortunately bazel makes it very difficult to set the configuration at build time as
# macros are resolved before select() can be resolved based on the command line. This could
# alternatively be done by defining build targets and the rules used to declare them twice
# (once for exec and for target).
"tool": attr.label(
executable = True,
cfg = "target",
mandatory = True),
},
)
Loading

0 comments on commit feed276

Please sign in to comment.