Skip to content

Commit

Permalink
Unrolled build for rust-lang#122245
Browse files Browse the repository at this point in the history
Rollup merge of rust-lang#122245 - saethlin:check-dep-graph-size, r=petrochenkov

Detect truncated DepGraph files

I suspect that the following issues are caused by truncated incr comp files:

* rust-lang#120582
* rust-lang#121499
* rust-lang#122210

We fail with an allocation failure or capacity overflow in this case because we assume that the ending bytes of an DepGraph file are the lengths of arrays. If the file has somehow been truncated then the ending bytes are probably some of our varint encoding, which tries to eliminate zero bytes, so interpreting a random 8 bytes as an array length has a very high chance of producing a byte capacity over `isize::MAX`.

Now theoretically since rust-lang#119510 merged I have fixed the out-of-disk issues and yet in rust-lang#120894 (comment) I still see some decoding failures that look like out-of-disk ICEs, for example https://crater-reports.s3.amazonaws.com/beta-1.77-1/beta-2024-02-10/gh/scottfones.aoc_2022/log.txt

So this PR should ensure that we get an ICE that clearly identifies if the file in question is truncated.
  • Loading branch information
rust-timer authored Mar 12, 2024
2 parents 7de1a1f + 11f8866 commit 15aa49b
Showing 1 changed file with 7 additions and 3 deletions.
10 changes: 7 additions & 3 deletions compiler/rustc_query_system/src/dep_graph/serialized.rs
Original file line number Diff line number Diff line change
Expand Up @@ -179,13 +179,15 @@ impl SerializedDepGraph {
pub fn decode<D: Deps>(d: &mut MemDecoder<'_>) -> SerializedDepGraph {
// The last 16 bytes are the node count and edge count.
debug!("position: {:?}", d.position());
let (node_count, edge_count) =
d.with_position(d.len() - 2 * IntEncodedWithFixedSize::ENCODED_SIZE, |d| {
let (node_count, edge_count, graph_size) =
d.with_position(d.len() - 3 * IntEncodedWithFixedSize::ENCODED_SIZE, |d| {
debug!("position: {:?}", d.position());
let node_count = IntEncodedWithFixedSize::decode(d).0 as usize;
let edge_count = IntEncodedWithFixedSize::decode(d).0 as usize;
(node_count, edge_count)
let graph_size = IntEncodedWithFixedSize::decode(d).0 as usize;
(node_count, edge_count, graph_size)
});
assert_eq!(d.len(), graph_size);
debug!("position: {:?}", d.position());

debug!(?node_count, ?edge_count);
Expand Down Expand Up @@ -491,6 +493,8 @@ impl<D: Deps> EncoderState<D> {
debug!("position: {:?}", encoder.position());
IntEncodedWithFixedSize(node_count).encode(&mut encoder);
IntEncodedWithFixedSize(edge_count).encode(&mut encoder);
let graph_size = encoder.position() + IntEncodedWithFixedSize::ENCODED_SIZE;
IntEncodedWithFixedSize(graph_size as u64).encode(&mut encoder);
debug!("position: {:?}", encoder.position());
// Drop the encoder so that nothing is written after the counts.
let result = encoder.finish();
Expand Down

0 comments on commit 15aa49b

Please sign in to comment.