Tracking issue (Rustdoc): hoedown -> pulldown migration #44229
Comments
nrc
added
T-rustdoc
|
cc @rust-lang/dev-tools @rust-lang/docs |
QuietMisdreavus
added
the
C-tracking-issue
What is happening?We're changing the way that Rustdoc renders markdown in your docs. We're moving to use a new parser and renderer (Pulldown) which is standards compliant (CommonMark), more robust, less buggy, and written in Rust. You can use Pulldown on nightly rustdoc by using the However, there are some changes to how your docs might be rendered. The known issues are:
We are currently providing warnings (when running Rustdoc) if the documentation for your crate will change with the new renderer. You should check these warnings to ensure that your documentation still renders correctly. |
Improve the Pulldown/hoedown warnings cc #44229 r? @QuietMisdreavus
|
Update: #44238 and #44329 addressed some issues with the warnings and it is now off by default (it was on by default for one nightly release). The next step is to assess the false positive situation. @QuietMisdreavus is going to list some good test crates and obtain the warning output for them. @GuillaumeGomez is going to try some experiments to reduce the false positive rate. |
|
ping @QuietMisdreavus did you get a chance to look at good test crates? |
|
After getting the first false positive i didn't dig much farther. However, a quick check of a handful of popular crates shows some more leads:
Something similar happens in I'll add more samples over time. |
|
The results for |
|
On a farther read, the EDIT: gonna add farther notes to this comment rather than spamming the thread.
EDIT 2: EDIT 3: EDIT 4: Here, the entire docstring is a single asterisk. Hoedown leaves it alone, puts it into the output as-is. Pulldown creates a new unordered list with an single, empty item. |
|
In sum, rocket continues to deliver as the quintessential rustdoc stress test. There are a few new rendering differences in here, a couple of which are technically bugs in pulldown.
|
|
I did all of this testing using the approved-but-not-yet-merged #44368. Judging from how all of these turned out, i'm willing to say that as soon as that lands, all the rendering warnings left will be true rendering differences. It doesn't mean that Pulldown is perfect, just that we're accurately talking about what it's doing. |
|
Next step here is that @GuillaumeGomez will draft a blog post about how to address the changes required for crate authors. We'll advertise that as widely as possible and then (depending on fallout) change the warnings to on by default a few weeks later. |
This fixes formatting problems when the Markdown processor is switched to pulldown-cmark (rust-lang/rust#44229).
This fixes formatting problems when the Markdown processor is switched to pulldown-cmark (rust-lang/rust#44229).
This makes documentation work correctly with the new pulldown-cmark Markdown parser (rust-lang/rust#44229).
This makes formatting correct with the new pulldown-cmark Markdown parser (rust-lang/rust#44229).
Lists now require an empty line before them, and a link can no longer be split on multiple lines. See rust-lang/rust#44229.
|
With the release of 1.23.0 last week, the warnings-by-default have hit stable! Since we haven't heard much by way of people reacting to it, we're moving on with the next step: rendering with Pulldown by default, with an option to go back to Hoedown. This is being done in #47398. |
|
I checked the checkbox (to avoid someone doing a duplicate). |
|
Sorry if this isn't the right place to ask. I'm trying to figure out how to migrate turtle's documentation to the new renderer. There are lots of warnings but I'm not really clear on how to fix anything. Is there a way for me to see the actual output differences? There are some weird things going on where rustdoc seems to be rendering Also, smaller issue, the warnings for certain dependencies still seem to be outputted even with |
|
There's no interactive difference view, but you can render the docs with the different renderers to see the differences. With a nightly, run once with |
|
How massive a lift would it be to add a difference view? (Trying to get a feel for effort level vs. value.)
|
That is exactly what I was looking for. Thanks! Though an integrated diff view would certainly be nice too, being able to manually do it myself is good enough for me. It turns out that the In case it helps anyone else, here's what I did to find the differences between the renderers: $ # Start clean by removing any previously generated docs
$ rm -rf target/doc*
$ # Generate docs with the old renderer
$ cargo +nightly doc --no-deps
$ mv target/doc target/doc-hoedown
$ # Generate docs with the new renderer
$ cargo +nightly rustdoc -- -Z unstable-options --enable-commonmark
$ # There are now two folders: `target/doc-hoedown` (old renderer) and `target/doc` (new renderer)
$ diff -r target/doc-hoedown/turtle target/doc/turtleNote that turtle is the name of my crate, so anyone else using these instructions should replace |
|
@chriskrycho too much, as we're planning on phasing the whole thing out very soon. Very few people at this point have commented so far, so it seems that the majority of issues have already been taken care of. Lots of work for little reward. |
|
@steveklabnik 👍 I did not want to go after it if that was the case! |
This fixes warnings like the following when doing "cargo doc":
WARNING: documentation for this crate may be rendered differently using the new Pulldown renderer.
See rust-lang/rust#44229 for details.
WARNING: rendering difference in `External types as defined at https://webassembly.g ... .html#external-types`
--> src/parser.rs:104:0
/html[0]/body[1]/p[0] Text differs:
expected: `... https://webassembly.github.io/spec/syntax/types.html#external-types`
found: `...`
|
I just noticed that tables aren't being rendered with the new pulldown renderer. I believe it's a trivial fix though (see pulldown-cmark/pulldown-cmark#104). For Example: /// Deserialize a raw CAN packet, validating its fields.
///
/// # Format
///
/// When deserializing the raw packet's payload, we expect it to be laid out
/// using the following
///
/// | Offset | Name | Description |
/// |----------+--------+----------------------------------|
/// | 0 | Bus ID | Which bus this packet came in on |
/// | 1 | CAN ID | The associated CAN ID |
/// | 3 | Length | The length of the data section |
/// | 4.. | Data | The packet's payload |
pub fn parse(data: &[u8]) -> Result<(CanPacket, usize), Error> {
...
}When you viewed using
And on nightly:
|
|
We do enable tables in the Pulldown parser. Your row separating the header from the data isn't what it's expecting, though. See https://github.com/webuni/commonmark-table-extension for the syntax - it wants you to use |
|
It seems like comments on trait methods get unescaped when they are shown as implemented traits. Something like |
|
@Ogeon: Please open a new issue for this. |
|
This #48274 got merged, the C cursed has now definitely been removed. Let's all celebrate by closing this issue! \o/ |
|
@GuillaumeGomez Alright, submitted #48332. |
- Fix mismatched code blocks - Remove newline from link (revealed by rust-lang/rust#44229)
…1 (from mbrubeck:doc); r=Manishearth Fixes warnings from rust-lang/rust#44229 when `--enable-commonmark` is passed to rustdoc. This is mostly a global find-and-replace for bare URIs on lines by themselves in doc comments. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes do not require tests because they are doc formatting changes only Source-Repo: https://github.com/servo/servo Source-Revision: 0e62a5829b7c29ae2667a21a439aff1e89201bf3 UltraBlame original commit: 27bc43d5d11ced40c6014d9055825993935d185e
…1 (from mbrubeck:doc); r=Manishearth Fixes warnings from rust-lang/rust#44229 when `--enable-commonmark` is passed to rustdoc. This is mostly a global find-and-replace for bare URIs on lines by themselves in doc comments. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes do not require tests because they are doc formatting changes only Source-Repo: https://github.com/servo/servo Source-Revision: 0e62a5829b7c29ae2667a21a439aff1e89201bf3 UltraBlame original commit: 27bc43d5d11ced40c6014d9055825993935d185e
…1 (from mbrubeck:doc); r=Manishearth Fixes warnings from rust-lang/rust#44229 when `--enable-commonmark` is passed to rustdoc. This is mostly a global find-and-replace for bare URIs on lines by themselves in doc comments. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes do not require tests because they are doc formatting changes only Source-Repo: https://github.com/servo/servo Source-Revision: 0e62a5829b7c29ae2667a21a439aff1e89201bf3 UltraBlame original commit: 27bc43d5d11ced40c6014d9055825993935d185e
If you've arrived via a warning and what to know what is going on, see this comment.
html-diffdependency, etc (Remove hoedown from rustdoc #48274)Blocking warning cycle
The text was updated successfully, but these errors were encountered: