Rustdoc JSON: Invalid output for core

[aDotInTheVoid]

Running

./x doc library/core/ --json
./x test ./tests/rustdoc-json/generic_impl.rs # to force a jsondoclint build.
./build/host/stage0-tools-bin/jsondoclint ./build/host/json-doc/core.json

Outputs a large number of errors

• Identify the core issue, find MCVE's, file issues
  • Rustdoc JSON Produces dangling ID's when pub useing #[doc(hidden)] items. #106379
  • rustdoc-json: Dangling ID for trait impl with pub struct in #[doc(hidden)] mod #112852
  • rustdoc-json: Dangling ID when private struct used in bound for pub-on-pub impl #113674
  • more
• Fix issues
  • rustdoc: Strip imports of items which are #[doc(hidden)] #106538
  • Strip impl if not re-exported and is doc(hidden) #113574
  • more
• Run jsondoclint on core.json in CI, so this doesn't regress again.

Originally posted by @aDotInTheVoid in #104064 (comment)

[barbogast]

Hi all,

is this issue beginner-friendly? I'd like to tackle this, but I'm new-ish to Rust (the language) and completely new to the Rust project, so I'm not sure whether it's efficient for you to guide me where necessary.

Anyway, I managed to follow the instructions and reproduce the issue. The first culprit I found is the variant type. Here the content of core.json deviates from the types in src/rustdoc-json-types/lib.rs in several ways.

What's the right way to fix this: Adjusting the types, or the generation of core.json? Adjusting core.json would also involve changing the consumers of this file, I guess.

If the types need to be adjusted I could provide a PR. If core.json should be adjusted I'd have to figure out how that works and then could work on a fix.

[barbogast]

Here some details for the variant issue. I'm not exactly sure how to best describe it in words, so maybe a diff with a potential fix helps:

src/rustdoc-json-types/lib.rs:

@@ -336,13 +336,14 @@ pub struct Enum {
 #[derive(Clone, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
 pub struct Variant {
     /// Whether the variant is plain, a tuple-like, or struct-like. Contains the fields.
-    pub kind: VariantKind,
+    #[serde(flatten)]
+    pub variant_kind: VariantKind,
     /// The discriminant, if explicitly specified.
     pub discriminant: Option<Discriminant>,
 }

 #[derive(Clone, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
-#[serde(rename_all = "snake_case")]
+#[serde(tag = "variant_kind", content = "variant_inner", rename_all = "snake_case")]
 pub enum VariantKind {
     /// A variant with no parentheses
     ///

---

With these fixes in place another issue pops up: a variant of type struct differs between types and json:

The type specifies the contents in variant_inner as a struct with the fields fields and fields_stripped, whereas the json contains the fields expr and value.

• Types: https://github.com/rust-lang/rust/blob/master/src/rustdoc-json-types/lib.rs#L376
• JSON:

           "inner": {
                "variant_kind": "struct",
                "variant_inner": {
                    "expr": "1",
                    "value": "1"
                }
            }

[aDotInTheVoid]

is this issue beginner-friendly? I'd like to tackle this, but I'm new-ish to Rust (the language) and completely new to the Rust project, so I'm not sure whether it's efficient for you to guide me where necessary.

I'm not sure, but I'd be happy to answer questions you have, either here or (preferably) on rust zulip (in the rustdoc channel)

[aDotInTheVoid]

How are you generating core.json? What is the reported format_version? It looks like your version of core.json is before #106354, which changed this area.

[barbogast]

Hi @aDotInTheVoid, thanks for your replies. Happy to continue the conv on Zulip once I know how ;-)

I'm on ddad1e1, so #106354 is included, and format_version is 23.

[aDotInTheVoid]

Hmmm, #106354 bumped FORMAT_VERSION (the const which generates the field) to 24. Where are you getting core.json from?

If you want to use zulip, https://forge.rust-lang.org/platforms/zulip.html describes how to get on it.

[makspll]

Could somebody point me to how you get a json rustdoc for the core crates?

[aDotInTheVoid]

@makspll See the comment at the top of this issue. ./x doc library/core/ --json will create ./build/host/json-doc/core.json

[makspll]

Ah, guessing there is no way to use rustdoc directly to get these, for anybody looking:

rustup component add --toolchain  <toolchain> rust-docs-json
ls ~/.rustup/toolchains/<toolchain>-x86_64-unknown-linux-gnu/share/doc/rust/json/

does the job