format_code_in_doc_comments makes multiline comments invalid

[vmx]

Describe the bug

All this happens only with format_code_in_doc_comments=true.

This is a follow up issue of #4420. When reformatting the output of that issue, another issue occurs. Two subsequent multiline comments (/* */), though in one line each, are transformed to invalid syntax as the opening indicator /* of the second line gets transformed to an asterisk * only.

To Reproduce

I use the (wrong) output from #4420 as basis and make it even more minimal (it can also be reproduced with the full output of that issue). When running the code blow with rustfmt --config format_code_in_doc_comments=true

enum Minimal {
    /* thatsleft */
    /* canbeanything */
}

The output is:

enum Minimal {
    /* thatsleft */
 * canbeanything */}

That is even invalid syntax.

Expected behavior

I would expect that the code is untouched by rustfmt.

Meta

• rustfmt version: rustfmt 1.4.18-nightly (c1e9b7b 2020-06-13)
• From where did you install rustfmt?: rustup
• How do you run rustfmt: rustfmt, but it also happens with cargo fmt

[ytmimi]

confirming this is reproducible with rustfmt 1.5.1-nightly (9c8ccc32 2022-07-26)

Linking tracking issue for format_code_in_doc_comments #3348

[CosmicHorrorDev]

@rustbot claim

[CosmicHorrorDev]

Had some time to look into this today. It seems like the root cause is that the rewriting is pretty messy in terms of determining separation of comments when block comment are involved. This also includes issues with the normalize_comments config since it shares the same rewriting logic. Here's a more elaborate example

enum Original {
    /* foo */
    /* bar */
}

enum DocComments {
    /** foo */
    /** bar */
    Variant,
}

enum Mix {
    // inline /* still inline */
    /* block1 */ /* block2 */
    Foo,
    /* block */
    // inline
    /* block */
    Bar,
}

With format_code_in_doc_comments=true

enum Original {
    /* foo */
     * bar */
}

enum DocComments {
    /** foo */
     * bar */
    Variant,
}

enum Mix {
    // inline /* still inline */
    /* block1 */ /* block2 */
    Foo,
    /* block */
     * inline
     * block */
    Bar,
}

With normalize_comments=true

enum Original {
    // foo */
    // bar
}

enum DocComments {
    /// foo */
    /// bar
    Variant,
}

enum Mix {
    // inline /* still inline */
    // block1 */ /* block2
    Foo,
    // block */
    // inline
    // block
    Bar,
}

It seems like the solution is properly handling parsing where comments start and end. That's all I have time for today though

[CosmicHorrorDev]

From poking around it looks like #5306 fixes all of the issues when using normalize_comments=true except for leaving */ /* when using multiple block comments in the same line, but doesn't fix any of the issues for format_code_in_doc_comments=true. It's possible that some small tweaks on top of that PR could also resolve this issue