Tracking issue for const <*const T>::is_null

[oli-obk]

This is tracking #[feature(const_ptr_is_null)].

impl<T: ?Sized> *const T {
    pub const unsafe fn as_ref<'a>(self) -> Option<&'a T>;
    pub const fn is_null(&self) -> bool;
}

impl<T: ?Sized> *mut T {
    pub const unsafe fn as_mut<'a>(self) -> Option<&'a mut T>;
    pub const unsafe fn as_ref<'a>(self) -> Option<&'a T>;
    pub const fn is_null(&self) -> bool;
}

Comparing pointers in const eval is dangerous business.

But checking whether a pointer is the null pointer is actually completely fine, as Rust does not support items being placed at the null address. Any otherwise created null pointers are supposed to return true for is_null anyway, so that's ok. Thus, we implement is_null as ptr.guaranteed_eq(ptr::null()), which returns true if it's guaranteed that ptr is null, and there are no cases where it will return false where it may be null, but we don't know.

[RalfJung]

and there are no cases where it will return false where it may be null, but we don't know.

That's not entirely correct. It will at some point be possible to create out-of-bounds raw pointers in const-eval (it might be possible already with enough hacks, I am not sure). Once that is possible, a pointer could be offset enough such that it is NULL at run-time but still non-NULL when checked during CTFE -- we cannot know if out-of-bounds pointers are NULL as that depends on their concrete base address.

[oli-obk]

So, some examples that we ran reliably check:

• std::ptr::null::<T>().is_null() will always be true
• NonZeroUsize::new(x).unwrap().is_null() will always be false
• `(&x as *const T).is_null() will always be false
• ((&x as *const T as usize - &x as *const T as usize) as *const T).is_null() will always be true (if you get it or an equivalent to compile)

The problematic case (&x as *const T).wrapping_offset(496).is_null() will be false, but at runtime could be true. We could set it up so it would be true instead, but then is_null() can return true when at runtime it would return false.

Another alternative is to panic if no exact solution is known (this can be done in a way that it compiles away to nothing at runtime)

Basically the options are:

• use ptr.guaranteed_eq(std::ptr::null()), but then you get false for out of bounds pointers that at runtime are null
• use !ptr.guaranteed_ne(std::ptr::null()), but then you get true for all out of bounds pointers, even the ones that are not null at runtime.
• use both variants and assert that they return the same thing.

[josephlr]

• use both variants and assert that they return the same thing.

As a user, this seems the most natural to me.

• Integer pointer that's zero => return true
• Integer pointer that's non-zero => return false
• Real pointer that's in-bounds => return false
• Real pointer that's out-of-bounds => panic, could be either true/false at runtime.

In general, panicking when doing something fundamentally non-const a cosnt fn seems fine.

[kupiakos]

Is there anything blocking stabilization of this? This also blocks NonNull::new from stabilization.

[RalfJung]

We still need to decide how "maybe null" pointers should be handled, see the discussion above.

[RalfJung]

Given in particular the use of is_null in NonNull::new/ptr::as_ref, I think the only possible interpretation it can have is "might be null": if we allow NonNull::new(ptr.wrapping_sub(0x120000)).unwrap() and later it turns out that pointer is at address 0x120000, we have a problem. Therefore, ptr.wrapping_sub(0x120000).is_null must be true.

Or of course we could panic and thus abort const-eval. We'll never panic at runtime so this may be acceptable.

@rust-lang/wg-const-eval any opinions on this?

[D0liphin]

Stabilization Report

Summary

Current tracking issue

Pointers can be definitely null:

std::ptr::null().is_null() // always `true`

Or definitely not null:

(&x as *const T).is_null() // always `false` 

Sometimes though, pointers are "maybe null", which is what makes stabilizing
this tricky.

(&x as *const T).wrapping_offset(-0x12345).is_null() // who knows?

Given that pointer comparison is a deterministic operation,
the "end-to-end behavior" must be the same for both the const fn and
fn. This rules out selecting either true or false consistently for the
maybe case; it could be different at runtime.

What we're left with then is no option but to abort const-evaluation if
the pointer may or may not be null. Note that this does not introduce
"observable behavior differences" between compile-time and run-time evaluation;
since compilation stops, there is "no compile-time behavior" so to speak.
Basically, this declares is_null to be supported in const contexts for some inputs
but not others, with a runtime check ensuring it only gets used on supported inputs.

Concretely, the supported inputs are:

• all "integer" pointers (created via ptr::without_provenance or int-to-ptr cast)
• all pointers that are inbounds or "at the end" of a current or former allocated object (i.e., the allocated object does not have to be live any more)

(also) Stabilize <*const T>.as_ref() as a const fn

This is currently blocked on const_ptr_is_null, so follows immediately
from stabilizing it. Both this and the mut variant can work now,
since const_mut_refs is stable.

(also) Stabilize NonNull::new() as a const fn

Current tracking issue

This follows immediately from stabilizing the above, but would need a separate stabilization process, I suppose?

Tests

• const_nonnull_new is tested in a const context here.
• const_ptr_is_null is tested in a const context hereven that

[workingjubilee]

I have updated the tracking issue with all const fn listed under this feature flag.

[RalfJung]

This message has 👍 from all of wg-const eval, so what remains is the lang team. @rust-lang/lang we'd like to make is_null const-callable, so here for the detailed report about the tricky part of this question. :)

[nikomatsakis]

My mental model for const eval is that it can sometimes fail to evaluate (panic) in value-dependent ways that are not tracked by the type system. For example, integers created by (exposed) pointers or other things may support a limited set of operations, etc, or (under nightly extensions to const generics) we may wish to distinguish &-references that refer to a static from other &-references.

This seems to fit that model fairly well-- do something complex enough in a const fn, and the value you produce may not be usable in all downstream operations, even if the type system would otherwise allow it.

Is that a reasonable mental model or am I missing something?

If this roughly tracks, I am in favor of exposing is_null this way.

Another question: if we do expose is_null with panic behavior now, could we plausibly allow some other behavior (e.g., conservatively return false) in the future? I don't know that we would ever want to, I'm inclined to say not, but I'm curious.

[RalfJung]

That sounds right. Note however that there is no UB-free way to turn a pointer into an integer in const eval (unless it was created by casting an integer to a pointer).

[scottmcm]

We discussed this in the lang triage meeting today, and are happy with the "compiler error in odd cases" as proposed by the two bullets in #74939 (comment)

So let's lang+libs-api stabilize this!

@rfcbot fcp merge

---

As I understood the discussion, we liked this for a couple reasons:

• A compiler error gives us flexibility to tweak those in future if we really did need to, since making those compilation failures start "working" later is allowed
• The cases of checking is_null for outside-of-symbolic-allocation pointers seem likely to be rare anyway, as that's typically runtime non-deterministic anyway. The cases that jump to mind for extensive wrapping_offset outside those bounds are things like the old slice iterator implementation -- which never needed to check is_null and was typically a without-provenance pointer anyway -- and doing pointer math like the internals of with_addr -- but that already doesn't work in const eval because it needs the pointer's address as an integer.
• Any difference in const that's not a compiler error felt particularly difficult to track down here
• We could always add additional methods that don't fail in const for these cases if needed (albeit that might be awkward in that it could imply additional variants of NonNull::new and such as well)
• Defaulting either to true or to false from this felt sketchy, since getting it wrong could plausibly immediately lead to validity invariant problems anyway.

[rfcbot]

Team member @scottmcm has proposed to merge this. The next step is review by the rest of the tagged team members:

• @Amanieu
• @BurntSushi
• @dtolnay
• @joshtriplett
• @m-ou-se
• @nikomatsakis
• @pnkfelix
• @scottmcm
• @tmandry
• @traviscross

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

[traviscross]

@rfcbot reviewed

@rustbot labels -I-lang-nominated

[RalfJung]

@Amanieu @BurntSushi @m-ou-se @nikomatsakis @pnkfelix @tmandry FCP checkbox reminder ping :)

[nikomatsakis]

@rfcbot reviewed

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

[RalfJung]

Stabilization PR is up. :)
#133116