feat: SSA pass impact report

[aakoshh]

Description

Problem*

Part of #8218

Looking at any given SSA pass, they have a few unit tests, but it's not obvious which integration tests (ie. Noir projects under test_programs) are relevant to them.

Summary*

Added a tool to list the integration tests where a certain SSA pass has the most impact on the IR.

It can be executed as follows:

cargo run -q -p nargo_cli --example ssa_pass_impact -- --ssa-pass <ssa-pass-name>

Currently it only looks at execution_success.

Additional Context

Impact

The impact is based on the similarity between the SSA of the pass we're interested in, and the one before it. A pass can appear multiple times in the pipeline, and these are all considered separately.

The similarity is calculated based on the number of lines where the string representation of the two SSA equal each other, which is computed using the similar crate. Before comparing them, the numeric values from the variable, block and function IDs are stripped away, so a re-numbering resulting from removing or adding instructions is ignored, and only the general structure and instruction names count.

The formulas are:

similarity(a, b) = 2 * lines_equal(a, b) / (lines(a) + lines(b))
impact(a, b) = 1 - similarity(a, b)

Build pipeline

I ended up duplicating a bunch of boilerplate from the compile_cli and nargo crates, and bypassing the SsaBuilder to run passes one by one. This has some pros and cons:

Pros:

• I can stop the compilation as soon as we have passed the last occurrence of the pass we're interested in
• There is no need to compile into Brillig and ACIR

Cons:

• The code does not do everything the main compiler does, and can get out of sync
• I don't execute the secondary pipeline with the Brillig VM passes
• Every execution of the tool compiles every program

These could be alleviated if instead of compiling in the tool we relied on what Tom started in #7853 and looked for the printed SSA saved into files, just doing the string comparisons.

Audited passes

The following is the impact on tests of the passes selected for audit:

Remove unreachable functions

❯ cargo run -q -p nargo_cli --example ssa_pass_impact -- --ssa-pass "Removing Unreachable Functions"
Packages: 267
Passes total: 1068
Passes with no impact: 1063
Passes most impacted by 'Removing Unreachable Functions' (top 50):
	0.156 impact: step 35 following 'Brillig Entry Point Analysis' in regression_fuzz_testing
	0.100 impact: step 35 following 'Brillig Entry Point Analysis' in global_var_multiple_entry_points_nested
	0.088 impact: step 35 following 'Brillig Entry Point Analysis' in brillig_calls_array
	0.080 impact: step 35 following 'Brillig Entry Point Analysis' in global_var_func_with_multiple_entry_points
	0.006 impact: step 10 following '`as_slice` optimization' in hashmap

Defunctionalize

❯ cargo run -q -p nargo_cli --example ssa_pass_impact -- --ssa-pass "Defunctionalization"
Packages: 267
Passes total: 267
Passes with no impact: 256
Passes most impacted by 'Defunctionalization' (top 50):
	0.302 impact: step 2 following 'Removing Unreachable Functions' in brillig_fns_as_values
	0.275 impact: step 2 following 'Removing Unreachable Functions' in 1327_concrete_in_generic
	0.121 impact: step 2 following 'Removing Unreachable Functions' in higher_order_functions
	0.050 impact: step 2 following 'Removing Unreachable Functions' in reference_only_used_as_alias
	0.026 impact: step 2 following 'Removing Unreachable Functions' in hashmap
	0.026 impact: step 2 following 'Removing Unreachable Functions' in slices
	0.024 impact: step 2 following 'Removing Unreachable Functions' in uhashmap
	0.024 impact: step 2 following 'Removing Unreachable Functions' in trait_impl_base_type
	0.024 impact: step 2 following 'Removing Unreachable Functions' in debug_logs
	0.019 impact: step 2 following 'Removing Unreachable Functions' in array_sort
	0.014 impact: step 2 following 'Removing Unreachable Functions' in regression_11294

Inline simple functions

❯ cargo run -q -p nargo_cli --example ssa_pass_impact -- --ssa-pass "Inlining functions with at most one instruction"
Packages: 267
Passes total: 267
Passes with no impact: 32
Passes most impacted by 'Inlining functions with at most one instruction' (top 50):
	0.421 impact: step 3 following 'Defunctionalization' in overlapping_dep_and_mod
	0.253 impact: step 3 following 'Defunctionalization' in trait_as_return_type
	0.247 impact: step 3 following 'Defunctionalization' in struct
	0.182 impact: step 3 following 'Defunctionalization' in double_verify_proof
	0.182 impact: step 3 following 'Defunctionalization' in double_verify_nested_proof
	0.179 impact: step 3 following 'Defunctionalization' in brillig_loop_size_regression
	0.177 impact: step 3 following 'Defunctionalization' in conditional_1
	0.150 impact: step 3 following 'Defunctionalization' in diamond_deps_0
	0.148 impact: step 3 following 'Defunctionalization' in regression_4709
	0.143 impact: step 3 following 'Defunctionalization' in 1327_concrete_in_generic
	0.130 impact: step 3 following 'Defunctionalization' in regression_7323
	0.119 impact: step 3 following 'Defunctionalization' in brillig_acir_as_brillig
	0.118 impact: step 3 following 'Defunctionalization' in loop_keyword
	0.118 impact: step 3 following 'Defunctionalization' in comptime_variable_at_runtime
	0.115 impact: step 3 following 'Defunctionalization' in global_array_rc_regression_8259
	0.112 impact: step 3 following 'Defunctionalization' in derive
	0.111 impact: step 3 following 'Defunctionalization' in field_attribute
	0.102 impact: step 3 following 'Defunctionalization' in array_oob_regression_7965
	0.100 impact: step 3 following 'Defunctionalization' in single_verify_proof
	0.098 impact: step 3 following 'Defunctionalization' in regression_method_cannot_be_found
	0.098 impact: step 3 following 'Defunctionalization' in global_slice_rc_regression_8259
	0.096 impact: step 3 following 'Defunctionalization' in brillig_fns_as_values
	0.095 impact: step 3 following 'Defunctionalization' in nested_if_then_block_same_cond
	0.093 impact: step 3 following 'Defunctionalization' in binary_operator_overloading
	0.092 impact: step 3 following 'Defunctionalization' in wrapping_operations
	0.091 impact: step 3 following 'Defunctionalization' in references
	0.090 impact: step 3 following 'Defunctionalization' in regression
	0.090 impact: step 3 following 'Defunctionalization' in poseidon_bn254_hash_width_3
	0.086 impact: step 3 following 'Defunctionalization' in prelude
	0.082 impact: step 3 following 'Defunctionalization' in brillig_calls
	0.082 impact: step 3 following 'Defunctionalization' in regression_4124
	0.078 impact: step 3 following 'Defunctionalization' in slice_regex
	0.077 impact: step 3 following 'Defunctionalization' in regression_3051
	0.077 impact: step 3 following 'Defunctionalization' in bool_not
	0.077 impact: step 3 following 'Defunctionalization' in poseidonsponge_x5_254
	0.074 impact: step 3 following 'Defunctionalization' in debug_logs
	0.073 impact: step 3 following 'Defunctionalization' in array_len
	0.073 impact: step 3 following 'Defunctionalization' in brillig_calls_conditionals
	0.069 impact: step 3 following 'Defunctionalization' in regression_fuzz_testing
	0.069 impact: step 3 following 'Defunctionalization' in simple_print
	0.065 impact: step 3 following 'Defunctionalization' in strings
	0.063 impact: step 3 following 'Defunctionalization' in trait_impl_base_type
	0.063 impact: step 3 following 'Defunctionalization' in conditional_2
	0.062 impact: step 3 following 'Defunctionalization' in regression_3394
	0.062 impact: step 3 following 'Defunctionalization' in 4_sub
	0.062 impact: step 3 following 'Defunctionalization' in comptime_println
	0.059 impact: step 3 following 'Defunctionalization' in slice_loop
	0.054 impact: step 3 following 'Defunctionalization' in unary_operator_overloading
	0.054 impact: step 3 following 'Defunctionalization' in hashmap
	0.054 impact: step 3 following 'Defunctionalization' in regression_5252

Documentation*

Check one:

• No documentation needed.
• Documentation included in this PR.
• [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

• I have tested the changes locally.
• I have formatted the changes with Prettier and/or cargo fmt on default settings.

[github-actions]

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Execution Time'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20.

Benchmark suite	Current: 6d95024	Previous: a686856	Ratio
rollup-merge	0.004 s	0.003 s	1.33

This comment was automatically generated by workflow using github-action-benchmark.

CC: @TomAFrench