Add small clarification around using pointers derived from references #103996

SUPERCILEX · 2022-11-05T03:34:33Z

One question about your example from rust-lang/libs-team#122: at what point does UB arise? If writing 0 does not cause UB and the reference x is never read or written to (explicitly or implicitly by being wrapped in another data structure) after the call to foo, does UB only arise when dropping the value? I don't really get that since I thought references were always supposed to point to valid data?

fn foo(x: &mut NonZeroI32)  {
  let ptr = x as *mut NonZeroI32;
  unsafe { ptr.cast::<i32>().write(0); } // no UB here
  // What now? x is considered garbage when?
}

rustbot · 2022-11-05T03:34:42Z

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

Stabilizing library features
Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
Changing public documentation in ways that create new stability guarantees
Changing observable runtime behavior of library APIs

Noratrieb · 2022-11-05T13:00:31Z

library/core/src/ptr/mod.rs

+//!   access the same memory. That is, reference and pointer accesses cannot be
+//!   interleaved—they must follow stacked borrows.


Stacked borrows is an experimental unofficial memory model, so the official docs cannot say that people "must follow stacked borrows". Maybe saying something along the lines of "The restrictions of using raw pointers in combination with references have not been decided yet, so it is advised to not interleave accesses using raw pointers and references.

Yeah, agreed. What question is this even answering? It seems unrelated to the other change.

Oh yeah, sorry this should probably be in another PR. The current wording makes it sound like you cannot have a reference and pointer around at the same time. I'm trying to clarify that... maybe I can just remove "—they must follow stacked borrows"?

It already says that no reference can be "used to access" the same memory while the pointer is used. Not sure how "accesses cannot be interleaved" adds any new information here?

I guess I am fine with clarifying, but I really don't see how the current wording "makes it sound like you cannot have a reference and pointer around at the same time" in any way that the new wording would fix.

Hmmm, yeah this may not be the best clarification, though sometimes saying the same thing in different ways is still helpful. 🤷‍♂️ If I'm remembering correctly, I kind of got the feeling that the reference was "consumed" by casting it to a pointer when first reading this. So maybe saying this would be better? "That is, the reference may only be used when there are no more accesses through the pointer." Though I feel like interleaving expresses that idea more clearly?

I guess the "that is" already states that this sentence is a clarification, not a new rule.

So if you remove the mention of stacked borrows this is probably fine to land.

RalfJung · 2022-11-05T16:09:32Z

One question about your example from rust-lang/libs-team#122: at what point does UB arise? If writing 0 does not cause UB and the reference x is never read or written to (explicitly or implicitly by being wrapped in another data structure) after the call to foo, does UB only arise when dropping the value? I don't really get that since I thought references were always supposed to point to valid data?

That's an open question: rust-lang/unsafe-code-guidelines#84.

References must point to valid data whenever they are used as a value (passed to a function, assigned, anything like that). But when they are just sitting there, the situation is much less clear.

RalfJung · 2022-11-05T16:12:35Z

library/core/src/ptr/mod.rs

+//! it is read, either through the raw pointer or the reference it originated from. This means a type
+//! may freely transition between valid and invalid states when being written to by raw pointers.
+//! Thus, when discussing safety, it may be useful to separately assert the validity of the pointer vs.
+//! the validity of the data it points to.


This means a type may freely transition between valid and invalid states when being written to by raw pointers.

This has not really been decided yet either -- see rust-lang/unsafe-code-guidelines#84. I possibly used too definite language in rust-lang/libs-team#122.

I am also not sure what you mean by pointer and pointee invariant here. It is already the case that https://doc.rust-lang.org/nightly/std/ptr/fn.read.html says "src must point to a properly initialized value of type T". https://doc.rust-lang.org/nightly/std/ptr/fn.write.html does not say anything like that because it takes a T as argument, which already implicitly says that it must be a valid T.

Sorry, but large parts of unsafe Rust are simply still uncharted territory. :(

Haha yep, the deeper I dig the more that seems to be true. :)

pointee invariant here.

If we decide this should go through I'll clarify that.

which already implicitly says that it must be a valid T.

This is specifically about the as_bytes case, so maybe I should make the idea of breaking down a larger type into smaller pieces clearer. I want to be explicit that it is ok to write valid data that causes a type to become invalid so long as the type becomes valid again by the time it's read.

I want to be explicit that it is ok to write valid data that causes a type to become invalid so long as the type becomes valid again by the time it's read.

And I'm saying I think we can't way this in the official docs yet because rust-lang/unsafe-code-guidelines#84 has not been officially answered yet.

Gotya, so unfortunately this whole paragraph is dead? I'll just keep the small clarification if so.

Yeah I am afraid so. :(

RalfJung · 2022-11-05T16:15:54Z

I think there is general consensus that this is sound

fn foo(x: &mut NonZeroI32)  {
  let ptr = x as *mut NonZeroI32;
  unsafe { ptr.cast::<i32>().write(0); }
  unsafe { ptr.cast::<i32>().write(1); }
}

but even that would need lang-team FCP.

These are probably questions that we simply cannot answer for now and need to defer until the to-be-created operational semantics team gets around to them, and an RFC or FCP is passed on the relevant aspect of the semantics. Sorry, but large parts of unsafe Rust are simply still uncharted territory. :(

Signed-off-by: Alex Saveau <[email protected]>

RalfJung · 2022-11-13T11:59:56Z

@bors r+ rollup

bors · 2022-11-13T11:59:57Z

📌 Commit 28ea002 has been approved by RalfJung

It is now in the queue for this repository.

…iaskrgr Rollup of 5 pull requests Successful merges: - rust-lang#103996 (Add small clarification around using pointers derived from references) - rust-lang#104315 (Improve spans with `use crate::{self}`) - rust-lang#104320 (Use `derive_const` and rm manual StructuralEq impl) - rust-lang#104357 (add is_sized method on Abi and Layout, and use it) - rust-lang#104365 (Add x tool to triagebot) Failed merges: r? `@ghost` `@rustbot` modify labels: rollup

Add small clarification around using pointers derived from references r? `@RalfJung` One question about your example from rust-lang/libs-team#122: at what point does UB arise? If writing 0 does not cause UB and the reference `x` is never read or written to (explicitly or implicitly by being wrapped in another data structure) after the call to `foo`, does UB only arise when dropping the value? I don't really get that since I thought references were always supposed to point to valid data? ```rust fn foo(x: &mut NonZeroI32) { let ptr = x as *mut NonZeroI32; unsafe { ptr.cast::<i32>().write(0); } // no UB here // What now? x is considered garbage when? } ```

…iaskrgr Rollup of 5 pull requests Successful merges: - rust-lang#103996 (Add small clarification around using pointers derived from references) - rust-lang#104315 (Improve spans with `use crate::{self}`) - rust-lang#104320 (Use `derive_const` and rm manual StructuralEq impl) - rust-lang#104357 (add is_sized method on Abi and Layout, and use it) - rust-lang#104365 (Add x tool to triagebot) Failed merges: r? `@ghost` `@rustbot` modify labels: rollup

rustbot assigned RalfJung Nov 5, 2022

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Nov 5, 2022

inquisitivecrystal added the A-docs Area: documentation for any part of the project, including the compiler, standard library, and tools label Nov 5, 2022

Noratrieb reviewed Nov 5, 2022

View reviewed changes

RalfJung reviewed Nov 5, 2022

View reviewed changes

Add small clarification around using pointers derived from references

28ea002

Signed-off-by: Alex Saveau <[email protected]>

SUPERCILEX force-pushed the docs branch from dfb910a to 28ea002 Compare November 6, 2022 20:10

SUPERCILEX changed the title ~~Add clarifying docs on pointer read write invariants~~ @SUPERCILEX Add small clarification around using pointers derived from references Nov 6, 2022

SUPERCILEX changed the title ~~@SUPERCILEX Add small clarification around using pointers derived from references~~ Add small clarification around using pointers derived from references Nov 6, 2022

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 13, 2022

matthiaskrgr mentioned this pull request Nov 13, 2022

Rollup of 5 pull requests #104370

Merged

bors merged commit a1b0702 into rust-lang:master Nov 13, 2022

rustbot added this to the 1.67.0 milestone Nov 13, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add small clarification around using pointers derived from references #103996

Add small clarification around using pointers derived from references #103996

SUPERCILEX commented Nov 5, 2022

rustbot commented Nov 5, 2022

Noratrieb Nov 5, 2022

RalfJung Nov 5, 2022

SUPERCILEX Nov 5, 2022

RalfJung Nov 5, 2022 •

edited

Loading

SUPERCILEX Nov 5, 2022

RalfJung Nov 6, 2022

RalfJung commented Nov 5, 2022

RalfJung Nov 5, 2022 •

edited

Loading

SUPERCILEX Nov 5, 2022

RalfJung Nov 6, 2022

SUPERCILEX Nov 6, 2022

RalfJung Nov 6, 2022

SUPERCILEX Nov 6, 2022

RalfJung commented Nov 5, 2022 •

edited

Loading

RalfJung commented Nov 13, 2022

bors commented Nov 13, 2022

		//! access the same memory. That is, reference and pointer accesses cannot be
		//! interleaved—they must follow stacked borrows.

Add small clarification around using pointers derived from references #103996

Add small clarification around using pointers derived from references #103996

Conversation

SUPERCILEX commented Nov 5, 2022

rustbot commented Nov 5, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RalfJung Nov 5, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RalfJung commented Nov 5, 2022

RalfJung Nov 5, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RalfJung commented Nov 5, 2022 • edited Loading

RalfJung commented Nov 13, 2022

bors commented Nov 13, 2022

RalfJung Nov 5, 2022 •

edited

Loading

RalfJung Nov 5, 2022 •

edited

Loading

RalfJung commented Nov 5, 2022 •

edited

Loading