chore: generate compilation tests from frontend tests

[michaeljklein]

Description

Problem*

Frontend tests are run in an environment that could become out of sync with nargo compile.

Summary*

During get_program_with_options and similar, we can:

• Generate a crate name by hashing the source from the test name
• Use the source as the contents of main.nr
• Generate a simple Nargo.toml
• Only regenerate when the source test changes

Note: when the cached version is out of date in CI, there's an error like the following:

--- STDERR:              noirc_frontend tests::traits::trait_alias_with_where_clause_has_equivalent_errors ---

thread 'tests::traits::trait_alias_with_where_clause_has_equivalent_errors' panicked at compiler/noirc_frontend/src/tests.rs:254:17:
test generated from frontend unit test noirc_frontend::tests::traits::trait_alias_with_where_clause_has_equivalent_errors is out of date: run `cargo test` to update

Additional Context

Questions:

1. My current approach is to generate the tests during get_program, but is there a better way to get this path than using the CARGO_MANIFEST_DIR? I considered trying to generate using build.rs, but I'm not sure how to do that without a two-step process of a) generating ephemeral output files when running the noirc_frontend tests, b) converting the ephemeral files to test_programs crates with build.rs.

    // "compiler/noirc_frontend"
    let noirc_frontend_path = Path::new(std::env!("CARGO_MANIFEST_DIR"));

2. The following snippet is from nargo_cli/src/cli/init_cmd: is there a sane way to import this without cyclic dependencies or a bunch of refactoring?

        let package_type = "bin"; // nargo::package::PackageType::Binary;
        let toml_contents = format!(
            r#"
            [package]
            name = "{package_name}"
            type = "{package_type}"
            authors = [""]
            
            [dependencies]"#
        );

Documentation*

Check one:

• No documentation needed.
• Documentation included in this PR.
• [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

• I have tested the changes locally.
• I have formatted the changes with Prettier and/or cargo fmt on default settings.

[github-actions]

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Test Suite Duration'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20.

Benchmark suite	Current: 918dd60	Previous: 25fd5c2	Ratio
olehmisar_noir_webauthn_	2 s	1 s	2
zkemail_zkemail.nr_lib	2 s	1 s	2

This comment was automatically generated by workflow using github-action-benchmark.

CC: @TomAFrench

[jfecher]

(waiting for tests to pass until I review)

[jfecher]

👍

[TomAFrench]

Looping back to this. @michaeljklein Do you have examples of where you were getting different behaviour between the two testing methods?

[michaeljklein]

The main examples are the skipped tests here:

• Recursive call that does not terminate panics when compiled #7763
• Array type in frontend trait test fails to resolve when compiled #7783
• Trait generic that passes frontend test fails to resolve with nargo #7766

[TomAFrench]

#7763

This strikes me as expected (minus the panic that we should turn into an error). Here we're testing whether the frontend will catch this test or not. The fact that cases which we call out that we don't catch early fail elsewhere in the compiler doesn't invalidate the test in my mind.

[michaeljklein]

@TomAFrench

The fact that cases which we call out that we don't catch early fail elsewhere in the compiler doesn't invalidate the test in my mind.

I'm not sure that it would "invalidate" the test, but I think that it might be good to at least add a comment r.e. the expected behavior of the test, if that makes sense?