Initial Book loader and summary parsing

[Michael-F-Bryan]

This is the first step towards #359, creating a Loader type which is in charge of reading the source directory, parsing the SUMMARY.md, and then constructing an internal representation of the book.

This PR will refactor the existing summary parsing to return some Summary struct which indicates how the book is structured so that later steps can locate files and populate the book from disk.

[azerupi]

Is the loader's only purpose to look for and parse the summary? In that case I would make the name more like: SummaryLoader or put it in the summary module.

Also, I feel like we might want the parsing method to not return immediately on the first syntax error. It might be more interesting to continue parsing to report multiple errors. I'm not sure how that is best handled, but since this exploratory work we can think a little more about this :)

[Michael-F-Bryan]

I was thinking the Loader can parse the summary and then using that parsing result it'll load the book from disk.

So in very rough pseudo-code:

impl Loader {
  ...

  pub fn load(&self) -> Result<Book> {
    let summary = self.parse_summary().chain_err(|| "Invalid SUMMARY.md")?;
    
    let mut book = Book::new();

    for item in summary {
      match item {
        SummaryItem::Chapter(ch) => {
          let chapter = self.load_chapter()?;
          book.add_section(chapter);
        }
        SummaryItem::Separator => book.add_section(BookItem::Separator),
     }
    }
  }
  
  Ok(book)
}

Another idea would be to accumulate errors in some Vec while loading, then add a variant to the error-chain called "LoadError" which contains a Vec<Error> to indicate all the errors encountered while parsing.

[Michael-F-Bryan]

Oops... I think I accidentally did git merge instead of git rebase to pull in the latest changes from master. 346ddef upped the changes from ~300 lines and 2 files touched to ~30 files. I'll have to do a little cherry picking to fix up the history.

Anyways, I think the easiest thing for me to do to parse the summary is take advantage of pulldown_cmark and create a simple state machine which will continually munch new events from pulldown until there are none left, changing behaviour and incrementally building the Summary depending on what state you are in. After the Summary is created (it's just a tree representing of SUMMARY.md), you can walk it recursively and generate the corresponding BookItem node using the visitor pattern(e.g. a method may be fn visit_link(&self, node: Link) -> BookItem).

@budziq, @azerupi, and @steveklabnik, what are your thoughts on taking that approach compared to the current implementation which recursively walks the SUMMARY.md file, building up the MdBook all in one hit?

[Michael-F-Bryan]

@azerupi, I've finally gotten things to the point where you can correctly parse a SUMMARY.md file and the full integration test passes! 😄

This PR is actually a lot smaller than it looks. There are only 362 executable lines of code in the loader module and something like 300-400 lines of tests. cargo kcov says the new summary parser has the largest number of lines covered by tests.

Now I need to clean things up, deal with as many TODOs and FIXMEs as possible, and get other people to review so I know what bits can be improved.

[azerupi]

Nice!
I will try to take a look at this later today :)

[Michael-F-Bryan]

I'm not sure if its worth the effort to keep two separate PR's Why not incorporate all of the changes to #360 change it's title and close this one?

After @budziq's comment I think I'll closing this PR and will put merge #360 and #371 into a single pull request.