Switch JSON serialization from rustc_serialize to serde

[jyn514]

Proposal

Right now, rustc_serialize is rather big:

> tokei compiler/rustc_serialize/ -f
====================================================================================================================
 Language                                                 Files        Lines         Code     Comments       Blanks
====================================================================================================================
 TOML                                                         1           12           10            0            2
--------------------------------------------------------------------------------------------------------------------
 compiler/rustc_serialize/Cargo.toml                                      12           10            0            2
--------------------------------------------------------------------------------------------------------------------
 Rust                                                        10         6263         5449          126          688
 |- Markdown                                                  5          290           82          168           40
 (Total)                                                                6553         5531          294          728
--------------------------------------------------------------------------------------------------------------------
 compiler/rustc_serialize/src/leb128.rs                                  156          129            3           24
 compiler/rustc_serialize/tests/leb128.rs                                 94           71            5           18
 compiler/rustc_serialize/tests/opaque.rs                                276          232            0           44
 compiler/rustc_serialize/src/json/tests.rs                              147          132            1           14
 compiler/rustc_serialize/tests/json.rs                                 1269         1107           10          152
-- compiler/rustc_serialize/src/lib.rs -----------------------------------------------------------------------------
 |- Rust                                                                  32           23            3            6
 |- Markdown                                                               1            0            1            0
 compiler/rustc_serialize/src/lib.rs                                      32           23            3            6
-- compiler/rustc_serialize/src/collection_impls.rs ----------------------------------------------------------------
 |- Rust                                                                 314          287            1           26
 |- Markdown                                                               1            0            1            0
 compiler/rustc_serialize/src/collection_impls.rs                        314          287            1           26
-- compiler/rustc_serialize/src/opaque.rs --------------------------------------------------------------------------
 |- Rust                                                                 714          531           53          130
 |- Markdown                                                               6            0            6            0
 compiler/rustc_serialize/src/opaque.rs                                  714          531           53          130
-- compiler/rustc_serialize/src/serialize.rs -----------------------------------------------------------------------
 |- Rust                                                                 736          635           14           87
 |- Markdown                                                              29            0           24            5
 compiler/rustc_serialize/src/serialize.rs                               736          635           14           87
-- compiler/rustc_serialize/src/json.rs ----------------------------------------------------------------------------
 |- Rust                                                                2525         2302           36          187
 |- Markdown                                                             147            0          124           23
 compiler/rustc_serialize/src/json.rs                                   2525         2302           36          187
====================================================================================================================
 Total                                                       11         6275         5459          126          690
====================================================================================================================

The JSON serialization/deserialization especially is almost exactly the same as the serde API, and there's no need to rewrite it. I propose removing all JSON handling in rustc_serialize.

Why only JSON?

This doesn't propose replacing rustc_serialize anywhere else, because the serde API does have one significant difference: it chooses the (de)serializer per function, not per type:

current rustc_serialize:

trait Decode<D: Decoder> { fn decode ... }

old rustc_serialize (including the deprecated crates.io version):

trait Decode { fn decode<D: Decoder> ... }

serde:

trait Deserialize { fn deserialize<D: Deserializer> ... }

The last two can't have per-decoder behavior, without specialization, which is unsound for e.g. the lifetime parameter of an arena allocator (which the compiler uses extensively). See #329 for more details.

Prior art

• replace serialize with serde in rustdoc rust#62359 successfully switched JSON serialization to serde in rustdoc.
• Start using serde_derive in a couple places in the compiler rust#56795 was closed due to build issues - I think x.py may not have support proc-macros at the time? I would be surprised if the same issue happened again.
• Rework rustc_serialize #329 is what changed the rustc_serialize API to avoid unsound specialization. @eddyb tells me the motivation was to allow arena allocating within the deserializer itself.

Mentors or Reviewers

@eddyb

Process

The main points of the Major Change Process are as follows:

• File an issue describing the proposal.
• A compiler team member or contributor who is knowledgeable in the area can second by writing @rustbot second.
  • Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a -C flag, then full team check-off is required.
  • Compiler team members can initiate a check-off via @rfcbot fcp merge on either the MCP or the PR.
• Once an MCP is seconded, the Final Comment Period begins. If no objections are raised after 10 days, the MCP is considered approved.

You can read more about Major Change Proposals on forge.

Comments

Everywhere I say "rustc_serialize" I mean the in-tree version; I don't know if the crates.io version is maintained or used.

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

[rustbot]

This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.

cc @rust-lang/compiler @rust-lang/compiler-contributors

[eddyb]

#329 is what changed the rustc_serialize API to be sound for specialization. @eddyb tells me the motivation was to allow arena allocating within the deserializer itself.

To be clear: we've always done this for decoding e.g. Ty<'tcx>, there's no way around it (since it's interned in TyCtxt<'tcx>).
What changed is that we were (ab)using unsound specialization before that MCP.

EDIT: I tweaked some of the wording in the original post to make it a bit more clear as to the overall situation.

[eddyb]

@rustbot second