This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
Comments for "https://os.phil-opp.com/testing/" #591
Comments
|
I'm not sure if I did something wrong, but by following along with this guide exactly I was receiving an error about fmt being undeclared after setting up the CompareMessage implementation. I fixed it by adding |
|
@leggettc18 You did nothing wrong, I forgot to mention these imports. Fixed in 1f97103. Thanks for reporting! |
phil-opp
changed the title
|
I'm unable to run
|
|
Try I have the same problem but couldn't figure out why the file isn't automatically found. |
|
@AntoineSebert Do you have a @Beidah Sounds like forgot a |
|
@phil-opp Yes it's there. The problem seems to have disappread today, and anyway it's not a big deal. |
|
The recent improvements in the test framework now produce text outputs that are not colored in green, thus making the test result less visible within the terminal. |
|
Yes, there is! You can use ANSI escape codes for coloring. For example, you can create a type to print something in green: use core::fmt;
struct Green(&'static str);
impl fmt::Display for Green {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
write!(f, "\x1B[32m")?; // prefix code
write!(f, "{}", self.0)?;
write!(f, "\x1B[0m")?; // postfix code
Ok(())
}
}Now you can do I opened #603 for adding this to the post. |
|
In the process of adding tests, I encountered some strange errors. In this test, I simply created a writer, but failed, and judging from the error message, it seems that it did not execute any tests at all, because I did not see the information printed by test_runner or panic_handler on the console.
lws@lsw:/document/project/flandre-os/FlandreOS$ cargo xtest test_default_writer
Compiling flandre_os v0.0.1 (/document/project/flandre-os/FlandreOS)
Finished dev [unoptimized + debuginfo] target(s) in 0.33s
Running target/x86_64-flandre_os/debug/deps/flandre_os-d120248f92b70111
Building bootloader
Compiling bootloader v0.6.0 (/work/tool/rust/cargo/registry/src/mirrors.ustc.edu.cn-61ef6e0cd06fb9b8/bootloader-0.6.0)
Finished release [optimized + debuginfo] target(s) in 0.72s
Running: `qemu-system-x86_64 -drive format=raw,file=/document/project/flandre-os/FlandreOS/target/x86_64-flandre_os/debug/deps/bootimage-flandre_os-d120248f92b70111.bin -device isa-debug-exit,iobase=0xf4,iosize=0x04 -serial stdio -display none test_default_writer`
qemu-system-x86_64: -display none: drive with bus=0, unit=0 (index=0) exists
error: test failed, to rerun pass '--lib'
|
|
I did all kinds of printing tests and the printing function seemed to work normally, but when I tried to add some tests to ensure that these functions would not be damaged in future modifications, I got the above errors. |
|
I think passing a test name to |
Yes, as you see from the "Running: What you can do is to run the tests of the library component through |
I tried adding the "x" as you indicated in the above paragraph. The test did not fail. WIth some more fooling about i see that the I have submitted a pull request #610 |
|
Hi, thank you for taking your time and effort into making this brilliant blog! I've learnt so much about operating systems and the Rust language in general! Can't wait to see how this blog unfolds and what new things you'll bring. I just wanted to ask if it is the expected behaviour for It seems to me that perhaps this was the desired behaviour? If so, is there a way for me to send data to the serial port while the QEMU instance is running as normal? |
|
@tahscenery Thanks so much for the nice words! Yes, this is the expected behavior. The reason is that the To also print the serial output for |
|
@phil-opp Of course, I knew it was that simple 🤦♂️! Thank you so much! |
|
Hi, thank you so much for writing this awesome blog. I'm a very fun of it!
I'm not sure why we need to remove |
Yes, exactly. Only the leaf crate is compiled in test mode, dependencies are not. And the |
|
I'm having trouble I did exactly as said in the tutirial |
|
@GrayJack Did you add the |
|
@GrayJack Thanks! You already have a |
|
Thanks, now it's complaining that that there is no #[panic_handler] function on the lib part, do I have to create a panic handler inside lib.rs as well? |
|
Yes, see the "Create a Library" section of the post:
|
|
I keep getting this supremely annoying error when trying to write the panic handler test: It seems like the panic handler defined in Any ideas @phil-opp |
|
Great to hear that! |
|
Hi, thanks for the wonderful tutorials! I have a problem. When I run |
|
@tomadimitrie The reason for that is that the library, binary, and integration tests are tested separately. So there are indeed multiple compilations. If your binary, for example, contains no tests, you see the "running 0 tests" output. This is expected and also the way that the Rust test framework works for normal projects. If you want to run tests only for one component, you can use |
Oh, I understand now. Thank you so much for your help and for your tutorials! |
|
After making the required changes to the
I am running it on ubuntu 18.04 64bit. here are the other environment details The weird thing is that the |
|
Did you add [build]
target = "x86_64-blog_os.json"and [target.'cfg(target_os = "none")']
runner = "bootimage runner"to |
|
Hi! Thanks for the great tutorials! I have encountered one problem with My [target.'cfg(target_os = "none")']
runner = "bootimage runner"
[build]
target = "../x86_64-catalyst.json"
[unstable]
build-std = ["core", "compiler_builtins"]
build-std-features = ["compiler-builtins-mem"]
[package.metadata.bootimage]
test-args = [
"-serial", "stdio",
]
|
|
@ShardulNalegave See https://github.com/rust-osdev/bootimage#configuration for all configuration options of |
|
@phil-opp That is the problem! |
|
Do you have your code online somewhere so that I can take a look? |
|
@phil-opp Yes, the code is at https://github.com/ShardulNalegave/catalyst-os |
|
The |
|
@phil-opp Oh! I don't know why I put it there. Anyways, thanks a lot! 😅 |
|
Thanks for the blog, its great. I'm following along on my computer, and when I run integration tests, I get this:
❯ RUST_BACKTRACE=full cargo test
Updating crates.io index
warning: Patch `adler v0.2.3 (/Users/ianwahbe/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/vendor/adler)` was not used in the crate graph.
Patch `gimli v0.22.0 (/Users/ianwahbe/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/vendor/gimli)` was not used in the crate graph.
Patch `addr2line v0.13.0 (/Users/ianwahbe/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/vendor/addr2line)` was not used in the crate graph.
Patch `object v0.20.0 (/Users/ianwahbe/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/vendor/object)` was not used in the crate graph.
Patch `miniz_oxide v0.4.0 (/Users/ianwahbe/.rustup/toolchains/nightly-x86_64-apple-darwin/lib/rustlib/src/rust/vendor/miniz_oxide)` was not used in the crate graph.
Check that the patched package version and available features are compatible
with the dependency requirements. If the patch has a different version from
what is locked in the Cargo.lock file, run `cargo update` to use the new
version. This may also occur with an optional dependency that is not enabled.
Compiling blog_os v0.1.0 (/Users/ianwahbe/Projects/BlogOS)
warning: unused import: `blog_os::println`
--> src/main.rs:7:5
|
7 | use blog_os::println;
| ^^^^^^^^^^^^^^^^
|
= note: `#[warn(unused_imports)]` on by default
warning: 1 warning emitted Building bootloader Caused by: Kernel executable at Stack backtrace: It looks like my tests work, then cargo rebuilds the boot loader, which crashes. Any ideas? |
|
@iwahbe Cargo runs the tests for each compilation unit of your crate separately. This means that it first runs the tests for your library and then for your binary. The test output you're seeing is from the library tests, which seem to work. Testing the binary, however, throws the error. The error message indicates the problem:
So the test executable for your If that doesn't fix it, I can take a look at your code if you upload it somewhere. |
|
@phil-opp your right, that fixed it. I thought that running |
|
Hey, Great blog, learning so much from this, and really like the writing style and lots of external reference links ! After implementing the Note, that this probably goes against all Rust conventions, and uses asm!, which, I read somewhere, is never going to be stabilized, but I wanted to try something, and someone might find a better approach from this 😅 This is the code modified from
static mut REMAINING_TESTS:Option<&'static [&'static dyn Fn()]> = None;
...
pub fn test_runner(tests: &'static [&'static dyn Fn()]) {
...
}
static mut SP:u64 = 0;
#![feature(asm)]
...
pub fn test_runner(tests: &'static [&'static dyn Fn()]) {
serial_println!("Running {} tests", tests.len());
unsafe{
REMAINING_TESTS = Some(tests); // store tests in static, will always be valid reference
// as lifetime is static, and running tests is the only thing we do
asm!("mov {}, rsp",sym SP); // save stack pointer in global variable
test_runner_helper(); // call the function which will actually run the code
}
}
pub unsafe fn test_runner_helper()->! {
match REMAINING_TESTS{
Some(t) =>{
if t.len() == 1{ // last test
REMAINING_TESTS = None;
}else{ // more tests remaining
REMAINING_TESTS = Some(&t[1..]); // shift starting of REMAINING_TESTS to next element
}
t[0](); // call the test
exit_qemu(QemuExitCode::Failed); // as test should panic, reaching here is failure
loop{}
},
None => { // no tests remaining
exit_qemu(QemuExitCode::Success);
loop {} // just to satisfy diverging function
},
}
}
#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
serial_println!("[ok]");
// changes
unsafe{
// First move the Stack Pointer to the value before we started tests.
// As test functions are meant to panic at some point, only which, will lead here,
// we can reset stack pointer, as the panicking test would not need the stack anymore
// Then we call the test_runner_helper again
asm!(
"mov rsp, {}","call {}",
sym SP,sym test_runner_helper,
)
}
exit_qemu(QemuExitCode::Failed); // This is theoretically unreachable, I think...
loop {} // for divergent function
}I checked this using five test functions, all five ran; and SP as well as BP was same for each function testing (tested by printing the value before invoking test function in test_runner_helper). |
|
@YJDoc2 Thanks for your comment! I'm glad that you like the blog :). Nice solution, even if it's probably a bit too hacky to be included in the post. Some small suggestions to make this a bit safer:
Using inline assembly in some cases is fine, you just have to be careful. What you read was probably about the old LLVM-style inline assembly macro, which is now named |
|
Hey, Thanks for the quick reply :) |
|
i got multiple should_panic tests working without using the SP register hack. but my rust-fu isn't great enough to get rid of the "static mut" bit. please have a look: static mut TESTS: &'static [&'static dyn Testable] = &[];
pub fn test_runner(tests: &'static [&'static dyn Testable]) {
serial_println!("Running {} tests", tests.len());
unsafe {
TESTS = tests;
test_runner_helper();
}
}
unsafe fn test_runner_helper() -> ! {
match TESTS.split_first() {
None => exit_qemu(QemuExitCode::Success),
Some((test, tests)) => {
TESTS = tests;
test.run();
serial_println!("[test did not panic]");
exit_qemu(QemuExitCode::Failed);
}
}
loop {}
}
#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
serial_println!("[ok]");
unsafe {
test_runner_helper();
}
}
#[test_case]
fn should_fail1() {
assert_eq!(0, 1);
}
#[test_case]
fn should_fail2() {
assert_eq!(0, 1);
} |
|
Building off the solutions of @CAD97, @YJDoc2, and @weihsiu I was able to get multiple should panic tests to technically work. However, the stack does not seem to be properly aligned afterwards. The stack pointer is saved in the test runner function then reset before each test invocation in the panic handler. However, when emulating the stack pointer reset functionality on Linux a segmentation fault is thrown after completing the test run. All the tests are run successfully, without a stack overflow, but the segmentation fault occurs when attempting to return from the test run. When run on the custom os target execution never continues beyond the test run so a misaligned stack is never discovered.
#![no_std]
#![no_main]
#![feature(custom_test_frameworks)]
#![test_runner(os::test::should_panic::runner)]
#![reexport_test_harness_main = "test_main"]
use core::panic::PanicInfo;
// Entry point.
#[no_mangle]
pub extern "C" fn _start() -> ! {
test_main();
loop {}
}
// Panic handler.
#[panic_handler]
fn panic(info: &PanicInfo) -> ! { os::test::should_panic::panic(info); }
// Tests.
#[cfg(test)]
mod tests {
#[test_case]
pub fn one() {
let actual = 1;
assert_eq!(0, actual);
}
...
}
use super::qemu;
use crate::serial_print;
use crate::serial_println;
use core::panic::PanicInfo;
use core::slice::Iter;
use spin::Mutex;
use spin::Once;
pub trait Testable: Sync {
fn run(&self);
}
impl<T: Fn() + Sync> Testable for T {
fn run(&self) {
serial_print!("{} ... ", core::any::type_name::<T>());
self();
serial_println!("failed");
qemu::exit(qemu::ExitCode::Failure);
}
}
// Test runner.
pub fn runner(tests: &'static [&dyn Testable]) {
serial_println!("Running {} test(s).", tests.len());
// Save tests and stack pointer.
let rsp: u64;
unsafe {
asm!("mov {}, rsp", out(reg) rsp);
}
Tests::init(tests, rsp);
run_next();
}
// Test mode panic handler.
pub fn panic(_: &PanicInfo) -> ! {
serial_println!("ok");
// Reset stack pointer.
if let Some(tests) = Tests::get() {
let rsp = tests.stack_pointer();
unsafe { asm!("mov rsp, {}", in(reg) rsp) }
}
run_next();
loop {}
}
fn run_next() {
let next = Tests::get().and_then(|mut i| i.next());
match next {
// All test ran successfully.
None => qemu::exit(qemu::ExitCode::Success),
// Run next test.
Some(&test) => test.run(),
}
}
struct TestsIter {
iter: Iter<'static, &'static dyn Testable>,
stack_pointer: u64,
}
impl TestsIter {
fn new(items: &'static [&dyn Testable], stack_pointer: u64) -> TestsIter {
let iter = items.iter();
TestsIter { iter, stack_pointer }
}
}
struct Tests {
inner: &'static Mutex<TestsIter>,
}
impl Iterator for Tests {
type Item = &'static &'static dyn Testable;
fn next(&mut self) -> Option<Self::Item> { self.inner.lock().iter.next() }
}
impl Tests {
fn instance() -> &'static Once<Mutex<TestsIter>> {
static INSTANCE: Once<Mutex<TestsIter>> = Once::new();
&INSTANCE
}
pub fn init(items: &'static [&dyn Testable], stack_pointer: u64) -> Tests {
let inner = Tests::instance().call_once(|| Mutex::new(TestsIter::new(items, stack_pointer)));
Tests { inner }
}
pub fn get() -> Option<Tests> { Tests::instance().get().map(|inner| Tests { inner }) }
pub fn stack_pointer(&self) -> u64 { self.inner.lock().stack_pointer }
}Full code on GitHub. |
|
Changing the
#[panic_handler]
fn panic(info: &PanicInfo) -> ! {
os::test::should_panic::panic(info); // Causes segmentation fault on return.
loop {}
}
pub fn panic(_: &PanicInfo) {
serial_println!("ok");
// Reset stack pointer.
if let Some(tests) = Tests::get() {
let rsp = tests.stack_pointer();
unsafe { asm!("mov rsp, {}", in(reg) rsp) }
}
run_next();
}
fn run_next() {
let next = Tests::get().and_then(|mut i| i.next());
match next {
// All test ran successfully. Return to demonstrate segmentation fault.
None => (),
// Run next test.
Some(&test) => test.run(),
}
} |
|
It is not allowed to change the stack pointer inside |
|
Restoring the stack pointer at the end of the |
|
Thanks for the great blog! I ran into a strange issue where it doesn't appear my tests are running once, I complete this part of the blog: https://os.phil-opp.com/testing/#custom-test-frameworks The Hello line prints but it doesn't look like the tests run when I run "cargo test" Any thoughts? My code is posted here: https://github.com/byoboo/tiny_os |
I figured out my issue above, in /cargo.toml I had the following below [package.metadata.bootimage] I commented out the "run-command" line and everything worked like a charm. |
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
and others
This is a general purpose comment thread for the Testing post.
The text was updated successfully, but these errors were encountered: