Skip to content

Add swift.emit_symbol_graph feature #838

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
brentleyjones merged 2 commits into from
Jun 2, 2022
Merged

Add swift.emit_symbol_graph feature #838

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
70a1ff1
Add `swift.emit_symbol_graph` feature
jpsim Jun 1, 2022
c372032
Address feedback from code review
jpsim Jun 1, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions swift/internal/compiling.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ load(
"SWIFT_FEATURE_EMIT_BC",
"SWIFT_FEATURE_EMIT_C_MODULE",
"SWIFT_FEATURE_EMIT_SWIFTINTERFACE",
"SWIFT_FEATURE_EMIT_SYMBOL_GRAPH",
"SWIFT_FEATURE_ENABLE_BATCH_MODE",
"SWIFT_FEATURE_ENABLE_LIBRARY_EVOLUTION",
"SWIFT_FEATURE_ENABLE_SKIP_FUNCTION_BODIES",
Expand Down Expand Up @@ -703,6 +704,15 @@ def compile_action_configs(
SWIFT_FEATURE_USE_PCH_OUTPUT_DIR,
],
),
swift_toolchain_config.action_config(
actions = [
swift_action_names.COMPILE,
],
configurators = [_emit_symbol_graph_configurator],
features = [
SWIFT_FEATURE_EMIT_SYMBOL_GRAPH,
],
),

# When using C modules, disable the implicit search for module map files
# because all of them, including system dependencies, will be provided
Expand Down Expand Up @@ -1608,6 +1618,21 @@ def _pch_output_dir_configurator(prerequisites, args):
paths.join(prerequisites.bin_dir.path, "_pch_output_dir"),
)

def _emit_symbol_graph_configurator(prerequisites, args):
"""Adds flags for `-emit-symbol-graph` configuration to the command line.

This is a directory to persist symbol graph files that can be used by
tools such as DocC or jazzy to generate documentation.
"""
args.add(
"-Xfrontend",
"-emit-symbol-graph",
)
args.add(
"-emit-symbol-graph-dir",
prerequisites.symbol_graph_directory.path,
Copy link
Copy Markdown
Member

@BalestraPatrick BalestraPatrick Jun 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume the build time penalty to generate this is pretty irrelevant, right?

Copy link
Copy Markdown
Collaborator

@brentleyjones brentleyjones Jun 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regardless it's behind a feature flag, similar to indexing while building.

Copy link
Copy Markdown
Member

@BalestraPatrick BalestraPatrick Jun 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah just out of curiosity. If it has a big impact, it might be worth documenting it.

Copy link
Copy Markdown
Contributor Author

@jpsim jpsim Jun 2, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if my methodology is correct, but I see a 2% hit to overall build times with the feature enabled:

$ hyperfine \
  --warmup 5 \
  --prepare "bazel clean && sleep 0.5" \
  "bazel build --features=swift.emit_symbol_graph --output_groups=+swift_symbol_graph //examples/apple/objc_interop:Printer" \
  "bazel build //examples/apple/objc_interop:Printer"
Benchmark 1: bazel build --features=swift.emit_symbol_graph --output_groups=+swift_symbol_graph //examples/apple/objc_interop:Printer
  Time (mean ± σ):     12.694 s ±  0.390 s    [User: 0.063 s, System: 0.055 s]
  Range (min … max):   12.379 s … 13.643 s    10 runs
 
Benchmark 2: bazel build //examples/apple/objc_interop:Printer
  Time (mean ± σ):     12.504 s ±  0.258 s    [User: 0.062 s, System: 0.051 s]
  Range (min … max):   12.105 s … 12.907 s    10 runs
 
Summary
  'bazel build //examples/apple/objc_interop:Printer' ran
    1.02 ± 0.04 times faster than 'bazel build --features=swift.emit_symbol_graph --output_groups=+swift_symbol_graph //examples/apple/objc_interop:Printer'

Since it's opt-in I don't see this needing a perf warning.

Copy link
Copy Markdown
Contributor Author

@jpsim jpsim Jun 2, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mind you, I edited Printer.swift to artificially make its compile time longer by adding a bunch of 2 * 2 * 2 * 2.0 / 2 + 2 statements to make the compiler work harder, taking ~6s in the Swift Compile step.

Without that artificial bloat, I can't measure a difference on a file that small:

Benchmark 1: bazel build --features=swift.emit_symbol_graph --output_groups=+swift_symbol_graph //examples/apple/objc_interop:Printer
  Time (mean ± σ):      6.222 s ±  0.189 s    [User: 0.076 s, System: 0.065 s]
  Range (min … max):    6.013 s …  6.563 s    10 runs
 
Benchmark 2: bazel build //examples/apple/objc_interop:Printer
  Time (mean ± σ):      6.228 s ±  0.217 s    [User: 0.074 s, System: 0.062 s]
  Range (min … max):    5.911 s …  6.587 s    10 runs
 
Summary
  'bazel build --features=swift.emit_symbol_graph --output_groups=+swift_symbol_graph //examples/apple/objc_interop:Printer' ran
    1.00 ± 0.05 times faster than 'bazel build //examples/apple/objc_interop:Printer'

)

def _global_index_store_configurator(prerequisites, args):
"""Adds flags for index-store generation to the command line."""
out_dir = prerequisites.indexstore_directory.dirname.split("/")[0]
Expand Down Expand Up @@ -1867,6 +1892,7 @@ def compile(
all_compile_outputs = compact([
compile_outputs.swiftinterface_file,
compile_outputs.indexstore_directory,
compile_outputs.symbol_graph_directory,
]) + compile_outputs.object_files
all_derived_outputs = compact([
# The `.swiftmodule` file is explicitly listed as the first output
Expand All @@ -1890,6 +1916,7 @@ def compile(
compile_outputs.swiftsourceinfo_file,
compile_outputs.generated_header_file,
compile_outputs.indexstore_directory,
compile_outputs.symbol_graph_directory,
]) + compile_outputs.object_files + other_outputs
all_derived_outputs = []

Expand Down Expand Up @@ -2073,6 +2100,7 @@ def compile(
other_compilation_outputs = struct(
ast_files = compile_outputs.ast_files,
indexstore = compile_outputs.indexstore_directory,
symbol_graph = compile_outputs.symbol_graph_directory,
)

return module_context, cc_compilation_outputs, other_compilation_outputs
Expand Down Expand Up @@ -2511,11 +2539,24 @@ def _declare_compile_outputs(
else:
indexstore_directory = None

emit_symbol_graph = is_feature_enabled(
feature_configuration = feature_configuration,
feature_name = SWIFT_FEATURE_EMIT_SYMBOL_GRAPH,
)
if (emit_symbol_graph):
symbol_graph_directory = derived_files.symbol_graph_directory(
actions = actions,
target_name = target_name,
)
else:
symbol_graph_directory = None

compile_outputs = struct(
ast_files = ast_files,
generated_header_file = generated_header,
generated_module_map_file = generated_module_map,
indexstore_directory = indexstore_directory,
symbol_graph_directory = symbol_graph_directory,
object_files = object_files,
output_file_map = output_file_map,
derived_files_output_file_map = derived_files_output_file_map,
Expand Down Expand Up @@ -2836,6 +2877,11 @@ def output_groups_from_other_compilation_outputs(*, other_compilation_outputs):
other_compilation_outputs.indexstore,
])

if other_compilation_outputs.symbol_graph:
output_groups["swift_symbol_graph"] = depset([
other_compilation_outputs.symbol_graph,
])

return output_groups

def swift_library_output_map(name):
Expand Down
13 changes: 13 additions & 0 deletions swift/internal/derived_files.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,18 @@ def _indexstore_directory(actions, target_name):
"""
return actions.declare_directory("{}.indexstore".format(target_name))

def _symbol_graph_directory(actions, target_name):
"""Declares a directory in which the compiler's symbol graph will be written.

Args:
actions: The context's actions object.
target_name: The name of the target being built.

Returns:
The declared `File`.
"""
return actions.declare_directory("{}.symbolgraph".format(target_name))

def _intermediate_bc_file(actions, target_name, src):
"""Declares a file for an intermediate llvm bc file during compilation.

Expand Down Expand Up @@ -340,6 +352,7 @@ derived_files = struct(
swiftinterface = _swiftinterface,
swiftmodule = _swiftmodule,
swiftsourceinfo = _swiftsourceinfo,
symbol_graph_directory = _symbol_graph_directory,
vfsoverlay = _vfsoverlay,
whole_module_object_file = _whole_module_object_file,
xctest_runner_script = _xctest_runner_script,
Expand Down
3 changes: 3 additions & 0 deletions swift/internal/feature_names.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,9 @@ SWIFT_FEATURE_CODEVIEW_DEBUG_INFO = "swift.codeview_debug_info"
# https://docs.google.com/document/d/1cH2sTpgSnJZCkZtJl1aY-rzy4uGPcrI-6RrUpdATO2Q/
SWIFT_FEATURE_INDEX_WHILE_BUILDING = "swift.index_while_building"

# If enabled, the compilation action for a target will produce a symbol graph.
SWIFT_FEATURE_EMIT_SYMBOL_GRAPH = "swift.emit_symbol_graph"

# If enabled the compilation action will not produce indexes for system modules.
SWIFT_FEATURE_DISABLE_SYSTEM_INDEX = "swift.disable_system_index"

Expand Down