-
Notifications
You must be signed in to change notification settings - Fork 12.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make tests capture the error printed by a Result return #102794
Conversation
r? @thomcc (rust-highfive has picked a reviewer for you, use r? to override) |
Hey! It looks like you've submitted a new PR for the library teams! If this PR contains changes to any Examples of
|
This comment has been minimized.
This comment has been minimized.
/checkout/src/test/rustdoc-gui/basic-code.goml basic-code... FAILED
[ERROR] (line 3) expected 1 elements, found 0: for command `assert-count: (".src-line-numbers", 1)` Same failure as #102744 (comment) which succeeded after retry, so very likely unrelated to this PR. |
Huh, yeah, the termination impl directly had Anyway, this is a really nice improvement even outside of an external test runner. The current output comes out pretty weirdly in this situation too; it's not where you'd expect, and often is intermixed with the list of running tests. This can be pretty confusing, since it's not somewhere that is supposed to have information, and it's quite bad that the thing that caused the test to fail is not in the area where we display the information about the test failures. For example
for #[test]
fn test1() -> Result<(), &'static str> {
println!("test1 stdout");
eprintln!("test1 stderr");
Err("test1 return")
}
#[test]
fn test2() -> Result<(), &'static str> {
println!("test2 stdout");
eprintln!("test2 stderr");
Err("test2 return")
}
#[test]
fn test3() -> Result<(), &'static str> {
println!("test3 stdout");
eprintln!("test3 stderr");
Err("test3 return")
} I'm pretty sure I've seen this before too, and I'm sure many others have as well -- but it only happens when you also have a failing test to deal with, so probably generally a distracted state. Thanks. @bors r+ |
Make tests capture the error printed by a Result return An error returned by tests previously would get written directly to stderr, instead of to the capture buffer set up by the test harness. This PR makes it write to the capture buffer so that it can be integrated as part of the test output by build tools such as `buck test`, since being able to read the error message returned by a test is pretty critical to debugging why the test failed. <br> **Before:** ```rust // tests/test.rs #[test] fn test() -> Result<(), &'static str> { println!("STDOUT"); eprintln!("STDERR"); Err("RESULT") } ``` ```console $ cargo build --test test $ target/debug/deps/test-???????????????? -Z unstable-options --format=json { "type": "suite", "event": "started", "test_count": 1 } { "type": "test", "event": "started", "name": "test" } Error: "RESULT" { "type": "test", "name": "test", "event": "failed", "stdout": "STDOUT\nSTDERR\n" } { "type": "suite", "event": "failed", "passed": 0, "failed": 1, "ignored": 0, "measured": 0, "filtered_out": 0, "exec_time": 0.00040313 } ``` **After:** ```console $ target/debug/deps/test-???????????????? -Z unstable-options --format=json { "type": "suite", "event": "started", "test_count": 1 } { "type": "test", "event": "started", "name": "test" } { "type": "test", "name": "test", "event": "failed", "stdout": "STDOUT\nSTDERR\nError: \"RESULT\"" } { "type": "suite", "event": "failed", "passed": 0, "failed": 1, "ignored": 0, "measured": 0, "filtered_out": 0, "exec_time": 0.000261894 } ```
Rollup of 6 pull requests Successful merges: - rust-lang#99696 (Uplift `clippy::for_loops_over_fallibles` lint into rustc) - rust-lang#102055 (Move some tests to more reasonable directories) - rust-lang#102786 (Remove tuple candidate, nothing special about it) - rust-lang#102794 (Make tests capture the error printed by a Result return) - rust-lang#102853 (Skip chained OpaqueCast when building captures.) - rust-lang#102868 (Rename `AssocItemKind::TyAlias` to `AssocItemKind::Type`) Failed merges: r? `@ghost` `@rustbot` modify labels: rollup
An error returned by tests previously would get written directly to stderr, instead of to the capture buffer set up by the test harness. This PR makes it write to the capture buffer so that it can be integrated as part of the test output by build tools such as
buck test
, since being able to read the error message returned by a test is pretty critical to debugging why the test failed.Before:
After: