Add support for --example

[nixpulvis]

I was looking for a solution to running an example from a README when I came across this project. In an ideal world, the README's rust code blocks with main functions would be exposed as --examples and also included in tests. I'm actually not sure the best way to be consistent with the rest of the way Cargo tests examples, but an example failing to compile should be caught by cargo test IMO.

Anyway, since this crate clearly does more than enough to run an example, I figured it would be a good place to open the issue.

[djc]

Hi, thanks for the feedback. Just as somewhat of a warning, this crate is only lightly being maintained. If you need/want extra features in the reasonably short term, you'll probably need to submit some code implementing them (although ideally you'd submit some kind of design ideas on how to implement them first so we can calibrate before you spend too much time going off in a direction which I disagree with).

(Alternatively, you can pay for my time to work on it.)

As for this issue specifically, I'm a little unsure what you are asking. This crate can already make sure the code from your README is run as a test, so I'm not sure what examples have to do with that.

[nixpulvis]

As for this issue specifically, I'm a little unsure what you are asking.

In short;
Given a README.md with the following contents:

```rust
fn main() { println!("hello world"); }
```

I am asking for a way to make cargo run --example README produce the following output:

hello world

There are some open questions in my mind about how to best implement this, but I'll wait to get too much further into this since it's unclear if you are interested.

[djc]

After thinking about it some more, I guess the scope of this project that makes sense is to "gather Rust code from Markdown files and format it into something usable as code", so I think generating an example is probably within scope.

Next, would you like to write a few sentences about how you'd change the public and internal APIs to accomplish this?

[nixpulvis]

OK, I'm still reading into the source code so I'll have to leave another comment (or draft PR) with more details later, but as I can tell so far:

Public Interface

• Test section headers and line numbers map to test names, which should be similar, if not the same as how example names are generated
• Discoverability of example names matters much more than test names, since people will generally run all tests, whereas examples must be run individually by name
  • There should be a default test named by the file it's contained within, as illustrated by my previous comment, in addition to individual example names
  • The default example should probably just be the first valid fenced rust block
• I'd be tempted to say we should avoid allowing examples generated by templates, but I also like the idea that tests and examples work mostly the same way

Internal Details

Most (if not all) of the code refers to everything as a "test", e.g. struct Test and fn generate_doc_tests, so it's probably worth doing a bit of renaming where appropriate. Otherwise, I expect to need an addition of fn generate_examples, but I'm not quite sure how this will place the needed binaries in the correct place yet. A quick attempt to copy an additional binary into target/debug/examples/ didn't work, so cargo itself may need to be consulted.

On that subject, I was looking into testing for this and came up with the following code, though I'm thinking using std::process might be vastly better. The biggest issue this currently has is that it needs to recompile everything separately. I'm assuming I could figure out how to avoid that by ensuring the config, ws and args mirror what's set by shell invocations of cargo carefully, but cargo's API really isn't meant to be used this way it seems.

extern crate cargo;
use std::path::Path;
use cargo::core::{compiler::CompileMode, Workspace};
use cargo::ops::{run, CompileOptions};
use cargo::util::command_prelude::{App, Arg, ArgMatchesExt, ProfileChecking};
use cargo::util::config::Config;


#[test]
fn readme_example() {
    let config = Config::default().unwrap();
    let cli = ["cargo".into(), "--example".into(), "README".into()];
    let args = App::new("cargo")
        .arg(Arg::with_name("example")
            .long("example")
            .required(true)
            .takes_value(true))
        .get_matches_from(&cli);
    let ws = args.workspace(&config).unwrap();
    let compile_opts = args.compile_options(
        &config,
        CompileMode::Build,
        Some(&ws),
        ProfileChecking::Custom,
    ).unwrap();
    run(&ws, &compile_opts, &cli).unwrap();
    assert!(false);
}

Thoughts?

[djc]

Sorry for the slow response, it's been busy and this project is not a priority for me...

How about using the cargo-metadata crate? I've been using it in other projects and I think it will probably solve the problems here with much fewer dependencies/build time required.