Check for overflow in DroplessArena and align returned memory #73237

tmiasko · 2020-06-11T13:07:35Z

Check for overflow when calculating the slice start & end position.
Align the pointer obtained from the allocator, ensuring that it
satisfies user requested alignment (the allocator is only asked for
layout compatible with u8 slice).
Remove an incorrect assertion from DroplessArena::align.
Avoid forming references to an uninitialized memory in DroplessArena.

rust-highfive · 2020-06-11T13:07:38Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @petrochenkov (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

petrochenkov · 2020-06-11T17:34:19Z

r? @nnethercote

src/librustc_arena/lib.rs

nnethercote · 2020-06-12T05:32:08Z

src/librustc_arena/lib.rs

+        assert!(bytes != 0);
+        loop {
+            if let Some(a) = self.alloc_raw_fast(bytes, align) {
+                break a;
            }


What happens here on overflow? It'll call self.grow(bytes). Then what will happen -- panic, success? I think the possibilities need to be clearer here. In particular, alloc_raw_fast() returns None in two cases: "overflow", and "need to grow". I feel like those two cases need to be distinguished.

From perspective of code that tried to use the current chunk the situation is the same: there is no capacity left and arena needs to grow. Overflow does not necessarily preclude later success if say the chunk had merely a high address, requested capacity was large, and the calculation overflowed.

The code then delegates the responsibility to handle the situation to grow and subsequently to RawVec. Under assumption that grow doesn't hit checked_mul(2).unwrap() (this could be saturated to isize::MAX to be a little bit friendlier to 32-bit systems), the allocation is performed by RawVec which panics when requested capacity exceeds isize::MAX (on 32-bit systems) and aborts on OOM.

Thanks for the explanation. Could you put that information into a comment? :)

Now with additional comments.

Thanks for that. Sorry to nitpick this further, but you've added a comment explaining the high-level behaviour, which is fairly obvious, while not adding a comment distinguishing the overflow vs. need-to-grow cases, which is less obvious.

Can you add a comment inside the function explaining those two cases, much like your comment above? I specifically would like to see explanation of the "high address + large capacity" case, in particular. Thanks.

I added a different comment explaining the abstract check we want to perform.
In my view the overflow is the same as the need-to-grow case, which is why I
didn't want to comment about the distinction. I just don't see any.

src/librustc_arena/lib.rs

nnethercote · 2020-06-15T06:21:44Z

src/librustc_arena/lib.rs

+        assert!(bytes != 0);
+        loop {
+            if let Some(a) = self.alloc_raw_fast(bytes, align) {
+                break a;
            }


Thanks for that. Sorry to nitpick this further, but you've added a comment explaining the high-level behaviour, which is fairly obvious, while not adding a comment distinguishing the overflow vs. need-to-grow cases, which is less obvious.

Can you add a comment inside the function explaining those two cases, much like your comment above? I specifically would like to see explanation of the "high address + large capacity" case, in particular. Thanks.

* Check for overflow when calculating the slice start & end position. * Align the pointer obtained from the allocator, ensuring that it satisfies user requested alignment (the allocator is only asked for layout compatible with u8 slice). * Remove an incorrect assertion from DroplessArena::align.

Return a pointer from `alloc_raw` instead of a slice. There is no practical use for slice as a return type and changing it to a pointer avoids forming references to an uninitialized memory.

tmiasko · 2020-06-15T08:25:08Z

I also changed the return type of alloc_raw to a pointer instead of a slice.
Not only it is already used like that, but also avoids forming references to
uninitialized memory as described in #72624.

nnethercote · 2020-06-15T09:14:24Z

@bors r+

bors · 2020-06-15T09:14:26Z

📌 Commit 1f08951 has been approved by nnethercote

Check for overflow in DroplessArena and align returned memory * Check for overflow when calculating the slice start & end position. * Align the pointer obtained from the allocator, ensuring that it satisfies user requested alignment (the allocator is only asked for layout compatible with u8 slice). * Remove an incorrect assertion from DroplessArena::align. * Avoid forming references to an uninitialized memory in DroplessArena. Helps with rust-lang#73007, rust-lang#72624.

@ghost

Rollup of 8 pull requests Successful merges: - rust-lang#73237 (Check for overflow in DroplessArena and align returned memory) - rust-lang#73339 (Don't run generator transform when there's a TyErr) - rust-lang#73372 (Re-order correctly the sections in the sidebar) - rust-lang#73373 (Use track caller for bug! macro) - rust-lang#73380 (Add more info to `x.py build --help` on default value for `-j JOBS`.) - rust-lang#73381 (Fix typo in docs of std::mem) - rust-lang#73389 (Use `Ipv4Addr::from<[u8; 4]>` when possible) - rust-lang#73400 (Fix forge-platform-support URL) Failed merges: r? @ghost

yoshuawuyts · 2020-07-07T16:13:03Z

@tmiasko I didn't realize #73007 was merged; thanks so much for enabling that with this PR! 🙏

rust-highfive assigned petrochenkov Jun 11, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 11, 2020

rust-highfive assigned nnethercote and unassigned petrochenkov Jun 11, 2020

nnethercote requested changes Jun 12, 2020

View reviewed changes

tmiasko force-pushed the arena branch 2 times, most recently from 194318a to 03360b4 Compare June 12, 2020 13:13

nnethercote requested changes Jun 15, 2020

View reviewed changes

tmiasko added 2 commits June 15, 2020 09:56

Avoid forming references to an uninitialized memory in DroplessArena

1f08951

Return a pointer from `alloc_raw` instead of a slice. There is no practical use for slice as a return type and changing it to a pointer avoids forming references to an uninitialized memory.

tmiasko force-pushed the arena branch from 03360b4 to 1f08951 Compare June 15, 2020 08:20

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 15, 2020

Manishearth mentioned this pull request Jun 16, 2020

Rollup of 14 pull requests #73392

Closed

Dylan-DPC-zz mentioned this pull request Jun 16, 2020

Rollup of 8 pull requests #73402

Merged

bors merged commit c65f39d into rust-lang:master Jun 16, 2020

tmiasko deleted the arena branch June 16, 2020 18:27

mati865 mentioned this pull request Jun 22, 2020

impl ToSocketAddrs for (String, u16) #73007

Merged

tmiasko mentioned this pull request Aug 6, 2020

1.45.0 regression macro-related ICE on 32-bit arm/ppc in tests #75219

Closed

cuviper added this to the 1.46 milestone May 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check for overflow in DroplessArena and align returned memory #73237

Check for overflow in DroplessArena and align returned memory #73237

tmiasko commented Jun 11, 2020 •

edited

Loading

rust-highfive commented Jun 11, 2020

petrochenkov commented Jun 11, 2020

nnethercote Jun 12, 2020

tmiasko Jun 12, 2020

nnethercote Jun 12, 2020

tmiasko Jun 12, 2020

nnethercote Jun 15, 2020

tmiasko Jun 15, 2020

nnethercote Jun 15, 2020

tmiasko commented Jun 15, 2020

nnethercote commented Jun 15, 2020

bors commented Jun 15, 2020

yoshuawuyts commented Jul 7, 2020

Check for overflow in DroplessArena and align returned memory #73237

Check for overflow in DroplessArena and align returned memory #73237

Conversation

tmiasko commented Jun 11, 2020 • edited Loading

rust-highfive commented Jun 11, 2020

petrochenkov commented Jun 11, 2020

nnethercote Jun 12, 2020

Choose a reason for hiding this comment

tmiasko Jun 12, 2020

Choose a reason for hiding this comment

nnethercote Jun 12, 2020

Choose a reason for hiding this comment

tmiasko Jun 12, 2020

Choose a reason for hiding this comment

nnethercote Jun 15, 2020

Choose a reason for hiding this comment

tmiasko Jun 15, 2020

Choose a reason for hiding this comment

nnethercote Jun 15, 2020

Choose a reason for hiding this comment

tmiasko commented Jun 15, 2020

nnethercote commented Jun 15, 2020

bors commented Jun 15, 2020

yoshuawuyts commented Jul 7, 2020

tmiasko commented Jun 11, 2020 •

edited

Loading