Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#[test] attribute and related changes #218

Merged
merged 5 commits into from
Dec 13, 2020
Merged

Conversation

tgolsson
Copy link
Contributor

This does the following:

  • Adds #[test] as an function attribute
  • Adds a --test flag to rune-cli, allowing tests to be executed
  • Adds support for callstacks in Sync code
  • Changes how the error messages are printed to split cause, location, and detail into different parts instead of always using "internal vm error" as the highlighted error.

I don't think there are any major issues with the current impl so I'd like to merge this (after some cleanup) and open a bunch of issues for follow-up work. If you think any of these are blockers for merging (or something else) I'm open to discussion! I'm starting to shoehorn more and more things into this PR because the main feature is done, which makes it harder to work with and review.

These are good topics for follow-up work:

  • Add support for cfg-flags for conditional compilation
  • Rework rune-cli to have test as a subcommand instead of a flag (and add rune run as the default command)
    • Add a --short mode to tests (...F..N.E...F style)
    • Add stdout capture by default for tests and add --no-capture
  • Add more detail to DebugInst with instruction-specific info, e.g., lhs/rhs spans for ops, and other things.
  • Expose test output/running to rune-lsp (maybe dependent/complicates/prevents feature: a format for external definitions #73)
  • Fix tree-shaking to make test-discovery easier (tracked in Unreferenced public imports are stripped #207)
  • Convert some tests to Rune level tests instead of testing with Rust

@@ -174,6 +178,10 @@ async fn try_main() -> Result<ExitCode> {
options.parse_option(opt)?;
}

if args.test {
options.test(true);
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

options.test(args.test)

Err(e) => {
// TODO: store output here
failures.insert(test.1.item.item.clone(), FailureReason::Crash(e));
writeln!(out, "crashed")?;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"failed"

println!("Return value: {:?}\n", e);
}
}
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think moving all of this information into a test-case struct would make this clearer.

tests.len(),
failure_count,
elapsed.as_secs_f64()
);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this go above or below the detailed output?

format!("type mismatch for operation `{}`", op),
vec![
format!("left hand side has type `{}`", lhs),
format!("rhs hand side has type `{}`", rhs),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, not rhs

let line_range = files.line_range(frame.source_id, line).expect("a range");
let source = files.get(frame.source_id)?;
let name = source.name();
let slice = &source.source()[line_range];
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cleanup this with a helper function

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should take the span from the inst instead... This is a bit odd right now with closures, you can end up repeating the same multiple times.

// span,
// "#[test] is not supported on nested",
// ));
// }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this commented code

@@ -712,7 +740,7 @@ impl Index for ast::ItemFn {
};

idx.query.insert_meta(span, meta)?;
} else if is_toplevel && item.visibility.is_public() {
} else if is_toplevel && item.visibility.is_public() || is_test {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this isn't enough to make #[test] in mods auto-discovered, we need to be even more eager about them.

@@ -194,6 +197,9 @@ pub enum CompileMetaKind {
ConstFn {
/// Opaque identifier for the constant function.
id: Id,

/// Whether this function has a test annotation
is_test: bool,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@udoprog I think you had opinions on this. See rust-lang/rust#51999 and related threads for rust background on static contexts and asserts. Seems like a reduced feature set is supported, which allows implementing tests as const fn. Not sure whether it is required to interact with the test system - I guess by definition any assert in const code blocks the build, which precludes testing it. It'd just become weird if you want to test the code and you end failing tests during compilation...

} else {
write!(f, "{{root}}")
formatter.pad("{root}")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is ugly, but I found no better way - this makes sure padding specifiers in format strings work as intended.

@udoprog
Copy link
Collaborator

udoprog commented Dec 13, 2020

Ok, merging it in since it's fairly isolated and patching it up in follow ups. Thanks!

@udoprog udoprog merged commit 4b4683a into rune-rs:main Dec 13, 2020
udoprog added a commit that referenced this pull request Dec 13, 2020
@udoprog udoprog added changelog Issue has been added to the changelog enhancement New feature or request labels Jan 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog Issue has been added to the changelog enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants