Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for compiling examples with --test to rustdoc & update the testing guide #15039

Closed
wants to merge 3 commits into from
Closed

Add support for compiling examples with --test to rustdoc & update the testing guide #15039

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a17b042
rustdoc: refactor code block language info into a struct.
huonw Jun 19, 2014
11bdeea
rustdoc: add the ability to run tests with --test.
huonw Jun 19, 2014
cb6219f
testing guide: update to use `test_harness` & fix problems.
huonw Jun 19, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 21 additions & 25 deletions src/doc/guide-testing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

To create test functions, add a `#[test]` attribute like this:

~~~
~~~test_harness
fn return_two() -> int {
2
}
Expand Down Expand Up @@ -37,7 +37,7 @@ test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured
Rust has built in support for simple unit testing. Functions can be
marked as unit tests using the `test` attribute.

~~~
~~~test_harness
#[test]
fn return_none_if_empty() {
// ... test code ...
Expand All @@ -55,7 +55,7 @@ other (`assert_eq`, ...) means, then the test fails.
When compiling a crate with the `--test` flag `--cfg test` is also
implied, so that tests can be conditionally compiled.

~~~
~~~test_harness
#[cfg(test)]
mod tests {
#[test]
Expand All @@ -80,11 +80,11 @@ Tests that are intended to fail can be annotated with the
task to fail then the test will be counted as successful; otherwise it
will be counted as a failure. For example:

~~~
~~~test_harness
#[test]
#[should_fail]
fn test_out_of_bounds_failure() {
let v: [int] = [];
let v: &[int] = [];
v[0];
}
~~~
Expand Down Expand Up @@ -204,26 +204,22 @@ amount.

For example:

~~~
# #![allow(unused_imports)]
~~~test_harness
extern crate test;

use std::slice;
use test::Bencher;

#[bench]
fn bench_sum_1024_ints(b: &mut Bencher) {
let v = slice::from_fn(1024, |n| n);
b.iter(|| {v.iter().fold(0, |old, new| old + *new);} );
let v = Vec::from_fn(1024, |n| n);
b.iter(|| v.iter().fold(0, |old, new| old + *new));
}

#[bench]
fn initialise_a_vector(b: &mut Bencher) {
b.iter(|| {slice::from_elem(1024, 0u64);} );
b.iter(|| Vec::from_elem(1024, 0u64));
b.bytes = 1024 * 8;
}

# fn main() {}
~~~

The benchmark runner will calibrate measurement of the benchmark
Expand Down Expand Up @@ -266,19 +262,16 @@ benchmarking what one expects. For example, the compiler might
recognize that some calculation has no external effects and remove
it entirely.

~~~
# #![allow(unused_imports)]
~~~test_harness
extern crate test;
use test::Bencher;

#[bench]
fn bench_xor_1000_ints(b: &mut Bencher) {
b.iter(|| {
range(0, 1000).fold(0, |old, new| old ^ new);
});
range(0, 1000).fold(0, |old, new| old ^ new);
});
}

# fn main() {}
~~~

gives the following results
Expand All @@ -297,8 +290,11 @@ cannot remove the computation entirely. This could be done for the
example above by adjusting the `bh.iter` call to

~~~
# struct X; impl X { fn iter<T>(&self, _: || -> T) {} } let bh = X;
bh.iter(|| range(0, 1000).fold(0, |old, new| old ^ new))
# struct X; impl X { fn iter<T>(&self, _: || -> T) {} } let b = X;
b.iter(|| {
// note lack of `;` (could also use an explicit `return`).
range(0, 1000).fold(0, |old, new| old ^ new)
});
~~~

Or, the other option is to call the generic `test::black_box`
Expand All @@ -309,10 +305,10 @@ forces it to consider any argument as used.
extern crate test;

# fn main() {
# struct X; impl X { fn iter<T>(&self, _: || -> T) {} } let bh = X;
bh.iter(|| {
test::black_box(range(0, 1000).fold(0, |old, new| old ^ new));
});
# struct X; impl X { fn iter<T>(&self, _: || -> T) {} } let b = X;
b.iter(|| {
test::black_box(range(0, 1000).fold(0, |old, new| old ^ new));
});
# }
~~~

Expand Down
12 changes: 12 additions & 0 deletions src/doc/rustdoc.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,18 @@ You can specify that the code block should be compiled but not run with the
```
~~~

Lastly, you can specify that a code block be compiled as if `--test`
were passed to the compiler using the `test_harness` directive.

~~~md
```test_harness
#[test]
fn foo() {
fail!("oops! (will run & register as failure)")
}
```
~~~

Rustdoc also supplies some extra sugar for helping with some tedious
documentation examples. If a line is prefixed with `# `, then the line
will not show up in the HTML documentation, but it will be used when
Expand Down
119 changes: 76 additions & 43 deletions src/librustdoc/html/markdown.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,8 +174,7 @@ pub fn render(w: &mut fmt::Formatter, s: &str, print_toc: bool) -> fmt::Result {
slice::raw::buf_as_slice((*lang).data,
(*lang).size as uint, |rlang| {
let rlang = str::from_utf8(rlang).unwrap();
let (_,_,_,notrust) = parse_lang_string(rlang);
if notrust {
if LangString::parse(rlang).notrust {
(my_opaque.dfltblk)(ob, &buf, lang,
opaque as *mut libc::c_void);
true
Expand All @@ -196,7 +195,7 @@ pub fn render(w: &mut fmt::Formatter, s: &str, print_toc: bool) -> fmt::Result {
stripped_filtered_line(l).unwrap_or(l)
}).collect::<Vec<&str>>().connect("\n");
let krate = krate.as_ref().map(|s| s.as_slice());
let test = test::maketest(test.as_slice(), krate, false);
let test = test::maketest(test.as_slice(), krate, false, false);
s.push_str(format!("<span id='rust-example-raw-{}' \
class='rusttest'>{}</span>",
i, Escape(test.as_slice())).as_slice());
Expand Down Expand Up @@ -309,16 +308,16 @@ pub fn find_testable_code(doc: &str, tests: &mut ::test::Collector) {
lang: *hoedown_buffer, opaque: *mut libc::c_void) {
unsafe {
if text.is_null() { return }
let (should_fail, no_run, ignore, notrust) = if lang.is_null() {
(false, false, false, false)
let block_info = if lang.is_null() {
LangString::all_false()
} else {
slice::raw::buf_as_slice((*lang).data,
(*lang).size as uint, |lang| {
let s = str::from_utf8(lang).unwrap();
parse_lang_string(s)
LangString::parse(s)
})
};
if notrust { return }
if block_info.notrust { return }
slice::raw::buf_as_slice((*text).data, (*text).size as uint, |text| {
let opaque = opaque as *mut hoedown_html_renderer_state;
let tests = &mut *((*opaque).opaque as *mut ::test::Collector);
Expand All @@ -327,7 +326,9 @@ pub fn find_testable_code(doc: &str, tests: &mut ::test::Collector) {
stripped_filtered_line(l).unwrap_or(l)
});
let text = lines.collect::<Vec<&str>>().connect("\n");
tests.add_test(text.to_string(), should_fail, no_run, ignore);
tests.add_test(text.to_string(),
block_info.should_fail, block_info.no_run,
block_info.ignore, block_info.test_harness);
})
}
}
Expand Down Expand Up @@ -365,33 +366,52 @@ pub fn find_testable_code(doc: &str, tests: &mut ::test::Collector) {
}
}

fn parse_lang_string(string: &str) -> (bool,bool,bool,bool) {
let mut seen_rust_tags = false;
let mut seen_other_tags = false;
let mut should_fail = false;
let mut no_run = false;
let mut ignore = false;
let mut notrust = false;

let mut tokens = string.as_slice().split(|c: char|
!(c == '_' || c == '-' || c.is_alphanumeric())
);

for token in tokens {
match token {
"" => {},
"should_fail" => { should_fail = true; seen_rust_tags = true; },
"no_run" => { no_run = true; seen_rust_tags = true; },
"ignore" => { ignore = true; seen_rust_tags = true; },
"notrust" => { notrust = true; seen_rust_tags = true; },
"rust" => { notrust = false; seen_rust_tags = true; },
_ => { seen_other_tags = true }
#[deriving(Eq, PartialEq, Clone, Show)]
struct LangString {
should_fail: bool,
no_run: bool,
ignore: bool,
notrust: bool,
test_harness: bool,
}

impl LangString {
fn all_false() -> LangString {
LangString {
should_fail: false,
no_run: false,
ignore: false,
notrust: false,
test_harness: false,
}
}

let notrust = notrust || (seen_other_tags && !seen_rust_tags);
fn parse(string: &str) -> LangString {
let mut seen_rust_tags = false;
let mut seen_other_tags = false;
let mut data = LangString::all_false();

let mut tokens = string.as_slice().split(|c: char|
!(c == '_' || c == '-' || c.is_alphanumeric())
);

for token in tokens {
match token {
"" => {},
"should_fail" => { data.should_fail = true; seen_rust_tags = true; },
"no_run" => { data.no_run = true; seen_rust_tags = true; },
"ignore" => { data.ignore = true; seen_rust_tags = true; },
"notrust" => { data.notrust = true; seen_rust_tags = true; },
"rust" => { data.notrust = false; seen_rust_tags = true; },
"test_harness" => { data.test_harness = true; seen_rust_tags = true; }
_ => { seen_other_tags = true }
}
}

data.notrust |= seen_other_tags && !seen_rust_tags;

(should_fail, no_run, ignore, notrust)
data
}
}

/// By default this markdown renderer generates anchors for each header in the
Expand Down Expand Up @@ -425,19 +445,32 @@ impl<'a> fmt::Show for MarkdownWithToc<'a> {

#[cfg(test)]
mod tests {
use super::parse_lang_string;
use super::LangString;

#[test]
fn test_parse_lang_string() {
assert_eq!(parse_lang_string(""), (false,false,false,false))
assert_eq!(parse_lang_string("rust"), (false,false,false,false))
assert_eq!(parse_lang_string("sh"), (false,false,false,true))
assert_eq!(parse_lang_string("notrust"), (false,false,false,true))
assert_eq!(parse_lang_string("ignore"), (false,false,true,false))
assert_eq!(parse_lang_string("should_fail"), (true,false,false,false))
assert_eq!(parse_lang_string("no_run"), (false,true,false,false))
assert_eq!(parse_lang_string("{.no_run .example}"), (false,true,false,false))
assert_eq!(parse_lang_string("{.sh .should_fail}"), (true,false,false,false))
assert_eq!(parse_lang_string("{.example .rust}"), (false,false,false,false))
fn test_lang_string_parse() {
fn t(s: &str,
should_fail: bool, no_run: bool, ignore: bool, notrust: bool, test_harness: bool) {
assert_eq!(LangString::parse(s), LangString {
should_fail: should_fail,
no_run: no_run,
ignore: ignore,
notrust: notrust,
test_harness: test_harness,
})
}

t("", false,false,false,false,false);
t("rust", false,false,false,false,false);
t("sh", false,false,false,true,false);
t("notrust", false,false,false,true,false);
t("ignore", false,false,true,false,false);
t("should_fail", true,false,false,false,false);
t("no_run", false,true,false,false,false);
t("test_harness", false,false,false,false,true);
t("{.no_run .example}", false,true,false,false,false);
t("{.sh .should_fail}", true,false,false,false,false);
t("{.example .rust}", false,false,false,false,false);
t("{.test_harness .rust}", false,false,false,false,true);
}
}
17 changes: 11 additions & 6 deletions src/librustdoc/test.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,8 +102,10 @@ pub fn run(input: &str,
}

fn runtest(test: &str, cratename: &str, libs: HashSet<Path>, should_fail: bool,
no_run: bool) {
let test = maketest(test, Some(cratename), true);
no_run: bool, as_test_harness: bool) {
// the test harness wants its own `main` & top level functions, so
// never wrap the test in `fn main() { ... }`
let test = maketest(test, Some(cratename), true, as_test_harness);
let input = driver::StrInput(test.to_string());

let sessopts = config::Options {
Expand All @@ -116,6 +118,7 @@ fn runtest(test: &str, cratename: &str, libs: HashSet<Path>, should_fail: bool,
prefer_dynamic: true,
.. config::basic_codegen_options()
},
test: as_test_harness,
..config::basic_options().clone()
};

Expand Down Expand Up @@ -200,7 +203,7 @@ fn runtest(test: &str, cratename: &str, libs: HashSet<Path>, should_fail: bool,
}
}

pub fn maketest(s: &str, cratename: Option<&str>, lints: bool) -> String {
pub fn maketest(s: &str, cratename: Option<&str>, lints: bool, dont_insert_main: bool) -> String {
let mut prog = String::new();
if lints {
prog.push_str(r"
Expand All @@ -220,7 +223,7 @@ pub fn maketest(s: &str, cratename: Option<&str>, lints: bool) -> String {
None => {}
}
}
if s.contains("fn main") {
if dont_insert_main || s.contains("fn main") {
prog.push_str(s);
} else {
prog.push_str("fn main() {\n ");
Expand Down Expand Up @@ -255,7 +258,8 @@ impl Collector {
}
}

pub fn add_test(&mut self, test: String, should_fail: bool, no_run: bool, should_ignore: bool) {
pub fn add_test(&mut self, test: String,
should_fail: bool, no_run: bool, should_ignore: bool, as_test_harness: bool) {
let name = if self.use_headers {
let s = self.current_header.as_ref().map(|s| s.as_slice()).unwrap_or("");
format!("{}_{}", s, self.cnt)
Expand All @@ -277,7 +281,8 @@ impl Collector {
cratename.as_slice(),
libs,
should_fail,
no_run);
no_run,
as_test_harness);
}),
});
}
Expand Down