Stabilize Rng and SystemRng

[joshtriplett]

View all comments

Stabilization report

This partial stabilization provides enough of an interface for people to obtain random bytes, which is a common need in the ecosystem, currently fulfilled via the getrandom crate.

There have been many requests for a fill_bytes interface in the standard library. Per previous libs-api discussions, SystemRng.fill_bytes can serve that function, rather than adding a separate free function.

Alternatives and Future Work

Uninitialized buffers

We're likely to add a fill_buf function to fill a BorrowedCursor<'_, u8>. We can do so once BorrowedBuf/BorrowedCursor is stable. Deferring this means we will need to support trait impls that provide fill_bytes but not fill_buf, which we might not need to if we waited until after BorrowedBuf/BorrowedCursor is stable. However, that isn't any worse of a problem than we already have with io::Read, and we don't necessarily want to couple the stabilization of BorrowedBuf/BorrowedCursor with

Distributions

The Distribution trait and the random function remain unstable; those don't need to block stabilization of Rng and SystemRng.

Optimized paths for u32/u64

Some RNGs can provide faster results for generating a whole u32/u64 rather than individual bytes.

The definition and documentation of fill_bytes says:

Note that calling fill_bytes multiple times is not equivalent to calling fill_bytes once
with a larger buffer. A RandomSource is allowed to return different bytes for those two
For instance, this allows a RandomSource to generate a word at a time and throw
of it away if not needed.

We hope that this will allow RNGs that can generate whole words to do so efficiently as a fast path in fill_bytes/fill_buf. If dedicated next_u32/next_u64 functions still end up being substantially faster, we can always add them as optional trait methods in the future.

Some experimentation suggests that it's possible to match the performance.

Result versus panicking

There's been extensive discussion about whether the function should return a Result rather than panicking, or providing an additional such function. The previous conclusion from libs-api was that while it's possible for the first such call to fail (e.g. because the OS or sandbox provides no access to randomness at all), subsequent calls should never fail, and user code will not be prepared to deal with such failure.

Furthermore, an API returning Result would propagate throughout higher-level calls, forcing operations as simple as "roll a d20" to either return Result or call expect/unwrap. And even providing a try variant will lead to higher-level APIs having to consider which variant to call. We should, instead, make the guarantee that a well-behaved underlying OS won't panic after the first call.

Note, in particular, that HashMap already fails via panic if it can't get data from its RandomState.

If there's a need to allow error recovery for the "no OS/sandbox support" case, we could provide a one-time call to check for an error. Or, such users could continue using getrandom or the underlying OS APIs.

If we did want to make every call fallible, we have the capability, using upcoming language features ("supertrait auto impl"), to add a TryRng supertrait without breaking backwards compatibility.

[jdahlstrom]

If it’s only the first call that can fail, could we put DefaultRandomSource behind a fallible constructor and guarantee that fill_bytes won’t panic?

[joshtriplett]

@jdahlstrom That would force every caller to deal with it, albeit only once. If we (in the future) provide a fallible have_random function or similar, then people who want to rule out failure can call that (and the standard library can make sure it gets evaluated only once), but most users won't have to care about it.

[joshtriplett]

I'm un-marking this as a draft.

Based on experiments with next_u32/next_u64, it's not clear we need them for performance. (Thanks to @hanna-kruppe for providing crates and benchmarking to help explore this! I got nerdsniped into doing some optimization on chacha8rand as a result, to verify this.)

As for fill_buf, it definitely has some value (based on benchmarks), but that doesn't mean we should block waiting on it.

[rustbot]

r? @Mark-Simulacrum

rustbot has assigned @Mark-Simulacrum.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: @ChrisDenton, libs
• @ChrisDenton, libs expanded to 8 candidates
• Random selection from Mark-Simulacrum, jhpratt

[hanna-kruppe]

cc @dhardy @newpavlov

[newpavlov]

Personally, I do not support this stabilization.

The most pressing needs can be alleviated by stabilizing a free-standing (potentially panicking) fill_bytes function. The API ignores rand_core experience and misses the SeedableRng/CryptoRng traits important in practice. I also believe there should be a clear future path for overriding the "default" RNG and using it on no_std targets.

We could call this Rng/DefaultRng rather than RandomSource/DefaultRandomSource.

IMO they should be named Rng/SysRng. For my taste, RandomSource/DefaultRandomSource are simply abhorrent.

Optimized paths for u32/u64

I don't think that added next_u32/u64 methods should have blanket impls, see here.

The previous conclusion from libs-api was that while it's possible for the first such call to fail (e.g. because the OS or sandbox provides no access to randomness at all), subsequent calls should never fail, and user code will not be prepared to deal with such failure.

This does not apply to HW-based RNGs used in cryptography. Not only they are IO-based, but also commonly use internal security checks. The same somewhat applies to RNGs built-in into CPUs. For example, RDRAND may in theory fail at any moment and some buggy AMD CPUs are known to produce bad values (e.g. after hybernation) which are guarded against with runtime checks.

In some niche cases it's also important to prove absence of panics and the suggested potentially panicking behavior will be an annoying hindrance.

Checking for errors could also be useful in scenarios where we mix entropy from different sources where failure of one source does not stop the system.

[hanna-kruppe]

Regarding next_u32/next_u64, while I really want DefaultRandomSource.fill_bytes (in some form) stabilized ASAP, I have reservations about leaving it at "it's not clear we need them for performance and we can add them later". Personally I'd rather err on the side of adding these methods, unless we're quite sure we will never need them, or it's clear that we can't resolve the question in a reasonable time frame.

Adding the methods after stabilization has a cost (even besides opportunity cost). As @dhardy pointed out in the past, adding provided method later means existing implementers that want to offer reproducibility (as in stability of produced values) can't override the provided methods without breaking reproducibility for users who started using those methods. And for libraries that use RNGs to sample some distribution and want to promise reproducibility of that sampling, the same problem applies if they're first written against fill_bytes and later want to use next_uN.

Another (smaller) reason to err on the side of including these is to ease the ecosystem's transition from rand traits (which have always had next_u32/u64) to the std trait. If std doesn't have the methods at first and adds them later, that's two unnecessarily transitions (rand::Rng::next_uN -> fill_bytes + uN::from_*e_bytes -> RandomSource::next_uN). Stabilizing some subset of distributions would avoid this, but the distributions are far from ready for stabilization.

Finally, while the benchmarks in #157193 and on Zulip don't have a smoking gun that the methods are necessary for performance, it's also not clear that we won't want them. Even those benchmarks show a benefit for dyn RandomSource (the only argument is whether you consider that compatible with "cares about performance"), and @dhardy previously mentioned that rand has benchmarks justifying the methods in rand's context. At minimum we should look at those benchmarks as well and see if the fill_bytes semantics (which I think matches rand's) actually works for those benchmarks as well.

View changes since the review

[conradludgate]

IIRC, it's possible to use inlining and https://doc.rust-lang.org/std/intrinsics/fn.is_val_statically_known.html to perform these optimisations without needing the API surface.

[hanna-kruppe]

Inlining doesn't work for dyn RandomSource. And is_val_statically_known only helps when the two implementations that have the same behavior, but in this case, some potentially desirable optimizations change behavior. (Also, the intrinsic doesn't seem to have a clear path to being exposed on stable.)

[orlp]

I oppose this stabilization, as I've mentioned before I don't think we are at a point where we want to stabilize traits or anything that represents or implies a canonical "way to do random number generation". The current proposal with RandomSource being a trait and potentially having multiple sources with the provided one only "being a default" is way too close to that.

There is one real need from the standard library: a (no_std overridable) source of random bytes. This should simply be a function without further baggage or API precedent.

Only once we have a clear view of what an opinionated std random API should look like should we stabilize any generic traits, distributions or generators. Not a piece-wise stabilization that will only end up shooting us in the foot later.

[dhardy]

Based on experiments with next_u32/next_u64, it's not clear we need them for performance. (Thanks to @hanna-kruppe for providing crates and benchmarking to help explore this! I got nerdsniped into doing some optimization on chacha8rand as a result, to verify this.)

next_u32/next_u64 aren't important for block-PRNGs like ChaCha. They're for word-PRNGs like Xoshiro, PCG, SFC. (It's still unclear to me whether the RandomSource trait is supposed to be a complete replacement for rand_core::Rng or only a trait over entropy sources, though in the latter case I don't see much justification in using a trait over a free function.)

To quote the DefaultRandomSource docs:

If security is a concern, consult the platform documentation below for the specific guarantees your target provides.

This is rather vague. Would a report of a defective implementation be considered a security issue? E.g. esp_fill_random advertises "true random values" but only given some additional criteria which might enable some form of side-channel attack (or worse, given that the documentation doesn't specify that the underlying PRNG is a CSPRNG).

But my biggest concern is what happens on unsupported platforms, e.g. wasm32-unknown-unknown? I think if the rand crate were to switch to DefaultRandomSource over the getrandom crate we'd have a minor rebellion (fork) due to the lost support for wasm32-unknown-unknown (this isn't the only target of concern, but by far the most widely used from the feedback we've had).

[hanna-kruppe]

next_u32/next_u64 aren't important for block-PRNGs like ChaCha. They're for word-PRNGs like Xoshiro, PCG, SFC.

This was my understanding as well, but when I sat down and worked through it, I couldn’t come up with a benchmark that shows a difference (between next_uN vs fill_bytes+uN::from_le_bytes). When the fill_bytes loop generates words and writes them to the buffer, I’d expect LLVM to simplify all of that away when fill_bytes can be inlined. There is a difference for the dyn Rng case since that prevents inlining, but that also hurts block based RNGs similarly. If rand has benchmarks that show something different, it would be great to know.

Maybe this has changed over time as LLVM has improved? The way rand derives fill_bytes from next_uN generically involves a lot of small fixed sized memcpys and LLVM has historically been been pretty bad at optimizing those (it’s still not great but much better now).

[ChrisDenton]

But my biggest concern is what happens on unsupported platforms, e.g. wasm32-unknown-unknown? I think if the rand crate were to switch to DefaultRandomSource over the getrandom crate we'd have a minor rebellion (fork) due to the lost support for wasm32-unknown-unknown (this isn't the only target of concern, but by far the most widely used from the feedback we've had).

The specific problem with wasm32-unknown-unknown is its dual nature as both a -none target and a -web target. Which one it is depends on the user of the target. I don't think this is solvable by std without separating out those use cases. In the meantime, it seems unfortunate to block improvements to std on that.

[dhardy]

@hanna-kruppe I tried benchmarking Xoshiro256++, Sfc32 and Sfc64 using rand_core::utils::next_word_via_fill to implement next_uN, and only Sfc64 was significantly slower (~10%). I then tried running the rand_distr benches with a modified Pcg64Mcg without significant affect on performance (thirty benches with <1% deltas, four 2-3% faster, nine 3-6% slower, one 15% slower). So, yes, it looks like LLVM can optimise over most performance issues of next_uN -> fill_bytes -> next_uN conversions; at least, on my (Zen 3) machine using static-dispatch with lots of function inlining.

If there's desire to use only a single method, I would consider using fill_words(&mut [u32; _]) instead. This guarantees alignment >= 4, and I can't recall ever seeing a use-case for less than one u32 word of output. The caveat is worse compatibility with every other RNG interface, but ultimately does that matter? Only a few trait impls (most in std and RNG libraries) are required and most users will want a higher-level interface like RngExt or choose anyway.

[ChrisDenton]

Are there any code examples in the docs? If not it'd be great to add them before stabilisation.

[the8472]

From zulip discussion, there seems to be some tension between goals

• we want to promise cryptographic quality
• people argue for the infallible API, which requires panicking when entropy can't be supplied
• ESP-IDF can't unconditionally provide crypto-quality (thus has to panic)
• libs-api has some precedent that stubbing APIs via panics can be something that disqualifies a target from advancing beyond tier 3 (but this isn't hard policy)
• Promote tier 3 riscv32 ESP-IDF targets to tier 2 compiler-team#864

[tarcieri]

people argue for the infallible API, which requires panicking when entropy can't be supplied

An alternative to panicking is to seed an infallible RNG from a fallible RNG. This at least defers the error condition to something that happens once up-front, and is avoidable thereafter.

I'd probably only recommend that for bare metal embedded use cases though. Anywhere you have a proper kernel entropy pool (and potentially have to worry about forking) you're better off using that.

[joshtriplett]

I tried benchmarking Xoshiro256++, Sfc32 and Sfc64 using rand_core::utils::next_word_via_fill to implement next_uN, and only Sfc64 was significantly slower (~10%).

Can you provide the benchmarking code? I'd love to see if we can optimize that in the style of hanna-kruppe/chacha8rand#1 .

[dhardy]

@joshtriplett here's a diff against rand_pcg code.
pcg128.diff.txt

[tarcieri]

on embedded targets hardware sources may fail at any moment and it's not always desirable to pull a whole CSPRNG.

I would generally caution against directly using the output of embedded hardware (T)RNGs. Many of them generate biased outputs even during normal operation, and that can be further exploited by things like physical attacks. "Whitening" the output of such RNGs by seeding a CSPRNG is standard practice.

[orlp]

What about a compromise? We can add both a fallible overridable cryptographically secure entropy device, and the convenient infallible Rng trait, with a blanket impl of the latter for the former.

That is is, we add the following to std::random:

/// When read from, returns uniformly random bytes.
///
/// # Safety
/// The random bytes read must be of cryptographic strength.
unsafe trait EntropySource : std::io::Read {}

/// Implements EntropySource, always the implementation from stdlib,
/// may unconditionally return ErrorKind::Unsupported.
pub struct SystemEntropy;

/// Implements EntropySource, defaults to SystemEntropy,
/// overridable with `#[global_entropy_source]`.
pub struct GlobalEntropy;

// Unchanged.
pub trait Rng {
    fn fill_bytes(&mut self, bytes: &mut [u8]);
}

// Can pass entropy source where Rng is expected.
impl<T: EntropySource> Rng for T {
    fn fill_bytes(&mut self, bytes: &mut [u8]) {
        self.read_exact(bytes).unwrap();
    }
}

[Kobzol]

Is convenience (i.e. not returning Result) an important factor for this specific API? I don't expect that most users would be using this low-level interface manually in their programs, I think that people usually just use a higher level interface to generate randomness. rand has 10x more dependant crates than getrandom.

[dhardy]

@orlp that's somewhat similar to the rand_core traits in design, though with a fixed error type (std::io::Error; #154046 tracks moving this type to core) and using unsafe (inappropriate and not particularly useful IMO).
Also, this is supposedly a stabilisation PR, not a development one.

std could easily provide a ThreadRng-like source, with potentially fallible initialization, but with infallible implementation of the RNG trait, but IMO it should be separate from SystemRng.

Agreed (though I'm unsure thread-locality is a sufficiently useful property in this case).

As a maintainer of rand_core and getrandom I hope that std/core eventually will supplant them completely for all use-cases. Until then, I believe they could play role of a good testing ground for developing RNG interfaces instead of rushing their stabilization in std. It also would make migration of the ecosystem from rand_core/getrandom to std/core easier.

In the near future I'd like to see std provide a random source, leaving getrandom to fill in where std is insufficient (no-std, maybe some use-cases wanting to fill an un-init buffer or use getrandom::u64).

In the longer term I'd be happy to see rand_core traits move into std (or core), but as mentioned there are some changes we might want to make to these traits depending on unfinished Rust features.

An important question here is whether we need rand_core::TryRng. It is used pretty-much exclusively used by getrandom::SysRng (ignoring impls with Error = Infallible, i.e. Rng). This isn't to say we don't need error handling, but that uses which do need error handling may not need to go through this trait.

[From @BurntSushi] Fair enough here. I re-read through the comments using this lens and I agree this seems like a not-great outcome. Particularly around not being able to rely on fill_bytes being a secure source of bytes. I think that's worse than the situation that @hanna-kruppe brought up regarding dependency injection. Namely, if we start with a fill_bytes free function, then our future progression seems complicated. If we reject EII, then we're back to the trait design. And if we do that, then we have a somewhat confusing story around what folks should be using. If we accept EII, then we run the risk of folks not being able to rely on the global resource not always providing the right guarantees.

Is it possible to require a cfg be set when using EII to provide an entropy source, as used by getrandom for custom backends? This makes it hard to overlook usage of a custom entropy source when buliding an application (some wasm-unknown users have complained about this being too tedious, but otherwise it appears to work for getrandom and rand).

As @tarcieri pointed out, some non-OS entropy sources are biased and should only be used to seed a user-space CSPRNG; I believe this should be the responsibility of the external EII impl since in other cases a user-space CSPRNG is not wanted.

At this point the design in my mind looks fairly similar to what is proposed for stabilisation here, with an additional method:

// Free function, supporting EII
// Must be unbiased on success.
// Uses core::io::Error type?
pub fn try_fill_bytes(&mut bytes) -> Result<(), Error>;

// Trait primarily for PRNGs
pub trait Rng {
    // Required method
    fn fill_bytes(&mut self, bytes: &mut [u8]);

    // Also next_u32(), next_u64() ?
}

// Infallible system RNG; try_fill_bytes may be used instead for a fallible API
pub struct SystemRng;
impl Rng for SystemRng { /* .. */ }

// Other PRNGs implementing Rng may be added in the future

There are compromises here (e.g. no getrandom::u64, no support for fill-uninit-buffer).

[bstrie]

I would like to suggest that notions of cryptographic suitability should be a separate discussion for a separate API. IMO what we're asking for here is I/O access to the system's entropy source. This is why I've been suggesting an API namespaced under std::io rather than std::random; for an MVP I'd even be content with a collection of platform-specific APIs under std::os, even if that would be step backward in usability from what the getrandom crate provides. The reason why I classify this omission as egregious is that the stdlib already includes all the code necessary to make use of these APIs (for HashMap), and yet it doesn't expose this functionality to user code.

However, if there are concerns that making a general-purpose high-level API is blocked on the question of cryptographic use cases (e.g. the Rng trait), then I'd be happy to completely punt on that high-level API for now. I agree with Kobzol that we should expect this to be a low-level I/O API that most users will not interact with directly, and the users that do interact with it will probably use it only once to seed a userspace PRNG (be it a CSPRNG or otherwise).

[joshtriplett]

Firstly, the default source may be overwritten with a custom potentially fallible source

That depends on whether we allow custom sources to override the default.

We need a solution for no_std, and a solution for wasm-unknown-unknown. But that doesn't necessarily mean that we should have a fully general override mechanism, rather than a mechanism for saying "you can provide an implementation if we don't have one".

For no_std, an override mechanism alone wouldn't actually help, because SystemRng is in std, not core. For wasm-unknown-unknown, I honestly think the right long-term solution might be a new target for "wasm with std and I'll supply external implementations of everything". But that doesn't mean that we should support external implementations for everything on every target just to support that case.

[joshtriplett]

@dhardy wrote:

In the near future I'd like to see std provide a random source, leaving getrandom to fill in where std is insufficient (no-std, maybe some use-cases wanting to fill an un-init buffer or use getrandom::u64).

👍. I'm hoping that we ship "fill uninit buffer" the moment we have stable BorrowedBuf/BorrowedCursor, and I'm hoping we have the equivalent of getrandom::u64 as soon as we ship Distribution. I think no-std use cases should keep using getrandom, at least until we have a general solution for "supply a std for a no-std build".

[orlp]

For no_std, an override mechanism alone wouldn't actually help, because SystemRng is in std, not core.

This is putting the cart before the horse. Putting SystemRng in std is a choice. It could also be in a no_std-available extern crate random; like alloc is.

[joshtriplett]

For no_std, an override mechanism alone wouldn't actually help, because SystemRng is in std, not core.

This is putting the cart before the horse. Putting SystemRng in std is a choice. It could also be in a no_std-available extern crate random; like alloc is.

The split between std/alloc/core is on balance a thing we'd love to move away from, not further towards. Any proposal to introduce an entirely new separate crate would need to be a separate proposal, and not one we should block on or anticipate.

Also, we don't want to proliferate different axes to filter on having (e.g. "OS" vs "random" vs "allocator" vs "filesystem" vs "threads") in a fashion that adds a thread for each.

[tbu-]

For wasm-unknown-unknown, I honestly think the right long-term solution might be a new target for "wasm with std and I'll supply external implementations of everything".

For wasm-unknown-unknown, the proper solution is that we should stop abusing that target for use cases that actually want to plug in an implementation of std. These use cases should ideally provide their own target, implementing SystemTime/fill_random in their std lib using #[wasm_bindgen] or whatever they use.

[dhardy]

I'm hoping we have the equivalent of getrandom::u64 as soon as we ship Distribution

What do they have to do with each other? The first is a special API to get a random u64 from the OS (mostly unnecessary IMO, though it may have uses beyond my experience) and the latter is a tool to generate variates.

[joboet]

I wonder: Should Rng be implemented generically for &mut R where R: Rng, just like Iterator is?

Edit: Filed as #159435

[joshtriplett]

I'm hoping we have the equivalent of getrandom::u64 as soon as we ship Distribution

What do they have to do with each other? The first is a special API to get a random u64 from the OS (mostly unnecessary IMO, though it may have uses beyond my experience) and the latter is a tool to generate variates.

Purely that random::<u64>(..) would be the equivalent of getrandom::u64().

[joboet]

As I read it, much of the opposition to this feature results from the significant API difference between getrandom::getrandom() and SystemRng.fill_bytes(), which make SystemRng not quite suitable as a drop-in replacement – even though it has been advertised as such.

To resolve some of these concerns I want to make the point that SystemRng shouldn't be viewed as a replacement of getrandom, but rather of rand's ThreadRng – and for that I think it is a good design. The main difference between SystemRng and ThreadRng is that std does not maintain its own per-thread RNG, but rather relies on platform APIs like arc4random_buf (BSDs), the vDSO getrandom (Linux), CCRandomGenerateBytes (Apple) and ProcessPrng (Windows) to do so. While this looses a bit of performance due to the added indirection, these functions have the inimitable benefit of tight platform integration, which means they are able to address all of the weaknesses of ThreadRng with (VM-)fork-detection and better in-memory state protection. As an added benefit std doesn't have to include its own CSPRNG implementation, at least on the major platforms.

But apart from the internal details, SystemRng works just like ThreadRng – namely, it will panic (or abort, for arc4random_buf) upon failure to seed the internal CSPRNG (though that is exceedingly unlikely) and thus implements Rng and not some TryRng-shaped trait, and it is suitable for generating large amounts of data suitable for cryptographic purposes.

This framing opens up the question of whether std should separately provide the getrandom functionality, allowing direct access to sources like getentropy. But that should be a separated effort, SystemRng should be stabilised now and independently.

[orlp]

@joboet I must ask then why the opening sentence of the stabilization report is:

This partial stabilization provides enough of an interface for people to obtain random bytes, which is a common need in the ecosystem, currently fulfilled via the getrandom crate.

[newpavlov]

@joboet

much of the opposition to this feature results from the significant API difference between getrandom::getrandom() and SystemRng.fill_bytes()

I wouldn't say so. Note that in future we are likely to move getrandom from free-standing functions to SysRng. The main problem which I tried to raise repeatedly is design of the RNG traits. Error handling is one of the primary issues, but not the only one. In my opinion, there is absolutely no reason to rush stabilization of the Rng trait. Instead a simple free-standing non-fallible fill_bytes function should be stabilized first.

SystemRng works just like ThreadRng

Rust targets a wide range of platforms. It's up to the system to decide how exactly it's RNG operates and I really hope that in future we will get a way to override the default source (UPD: I am talking about a #[global_allocator]-like override, not being generic over Rng). Note that I speak as someone who professionally developed certified cryptographic software in Rust.

If you don't want to deal with error-handling headache you always could introduce a proper ThreadRng-like source or a separate panicking version of SystemRng.

[joboet]

@joboet I must ask then why the opening sentence of the stabilization report is:

This partial stabilization provides enough of an interface for people to obtain random bytes, which is a common need in the ecosystem, currently fulfilled via the getrandom crate.

Yes, that's how SystemRng is currently framed. But I don't think that's the best way to think about it!

[joboet]

Rust targets a wide range of platforms. It's up to the system to decide how exactly it's RNG operates [...]

But I think it's very much std's responsibility is to make the choice between the different sources where multiple exist.

If you don't want to deal with error-handling headache you always could introduce a proper ThreadRng-like source or a separate panicking version of SystemRng.

My argument is that an RNG based on arc4random_buf-shaped APIs is very useful separately from getrandom::SysRng, and because of its better security attributes is the better choice for std over including ThreadRng – though it should fulfil the same role.

The main problem which I tried to raise repeatedly is design of the RNG traits. Error handling is one of the primary issues, but not the only one.

Under the SystemRng-is-ThreadRng framing it makes sense for SystemRng to be infallible and implement Rng. That means error handling is unnecessary for this minimum viable product as it is irrelevant for the only RNG the standard library provides under the MVP. I'd think it safe to add TryRng later with a blanket implementation for all Rngs, should there be demand for it?

The only other issue I could find is your concern about next_u32/next_64. Is that still active? I'd file an FCP-concern on your behalf but I don't think I have the permissions to do so.

[newpavlov]

My argument is that an RNG based on arc4random_buf-shaped APIs is very useful separately from getrandom::SysRng

I agree. But it can be handled using a separate type from SystemRng. I considered doing something similar in rand for ThreadRng, i.e. to use system-provided infallible RNG instead of the ChaCha-based reseeded RNG on platforms where it's available (e.g. modern Windows and BSD).

That means error handling is unnecessary for this minimum viable product as it is irrelevant for the only RNG the standard library provides under the MVP.

The Rng trait is also "unnecessary" for MVP, i.e. why not begin with a free-standing non-fallible function as I suggested?

I'd think it safe to add TryRng later with a blanket implementation for all Rngs, should there be demand for it?

Stabilizing Rng trait in its current form could be restrictive. For example, it will not be compatible with the approach currently used in rand_core. The RNG traits in std/core should aim to completely replace rand_core for all use cases. The current sole Rng trait is frankly half-baked in my opinion.

P.S.: I re-iterated the same points many times already with seemingly no success, so I guess I will stop posting here.

[ChrisDenton]

Rust targets a wide range of platforms. It's up to the system to decide how exactly it's RNG operates and I really hope that in future we will get a way to override the default source. Note that I speak as someone who professionally developed certified cryptographic software in Rust.

The argument is that a trait naturally allows overriding (i.e. you can implement Rng for custom types). Therefore there's not a need for another mechanism if you have a trait.

[dhardy]

Under the SystemRng-is-ThreadRng framing it makes sense for SystemRng to be infallible and implement Rng.

But it can be handled using a separate type from SystemRng. I considered doing something similar in rand for ThreadRng, i.e. to use system-provided infallible RNG instead of the ChaCha-based reseeded RNG on platforms where it's available (e.g. modern Windows and BSD).

Even on Linux, even using only a single thread, there's a pretty substantial difference in performance between rand's ThreadRng and SysRng. I don't advocate copying this design; having a (lock- or system-based) DefaultRng plus an InsecureThreadRng would be a better one in my opinion (something like Xoshiro or SFC requires a lot less thread-local state and doesn't rely on amortisation for performance).

SystemRng as-it-is is a low-level building block for DefaultRng etc. Should it be part of the public API? Perhaps, but it would make more sense to do so in a way that allows error handling. (Or don't; expose a higher-level DefaultRng instead, including better guarantees of security and performance.)

P.S.: I re-iterated the same points many times already with seemingly no success, so I guess I will stop posting here.

I feel the same way.

[tgross35]

It would be very helpful to see an RFC on this API before stabilization. There is an unbelievable amount of information scattered across hundreds of comments in the ACP, the tracking issue, and here. It's pretty much impossible for anyone from the community to get up to speed unless they've been following along the whole time.

A handful of possible questions to answer there that (as I understand it) aren't documented outside of a comment or aren't completely answered:

• Is there a plan to allow overriding? Stabilize Rng and SystemRng #157168 (comment), Stabilize Rng and SystemRng #157168 (comment)
• How should users think about this API? More like getrandom, rand::ThreadRng, or something else? Stabilize Rng and SystemRng #157168 (comment)
• Can/should this replace the above? Or generally, what is the interaction with ecosystem crates?
• What are no_std users expected to do?
• What is expected of users that care about handling errors? Or targets that can't always provide randomness and have to panic? Stabilize Rng and SystemRng #157168 (comment)
• Is a defective implementation a security issue? Stabilize Rng and SystemRng #157168 (comment)
• Examples for all of the above

Having the threaded format would also make everything easier to discuss.

[ChrisDenton]

Rust targets a wide range of platforms. It's up to the system to decide how exactly it's RNG operates and I really hope that in future we will get a way to override the default source. Note that I speak as someone who professionally developed certified cryptographic software in Rust.

The argument is that a trait naturally allows overriding (i.e. you can implement Rng for custom types). Therefore there's not a need for another mechanism if you have a trait.

Since this has been meet with a thumb down and confusion I'll expand on this, especially as people seem to be talking past each other. In past libs-api meeting, it has been the consistent position of libs-api that a trait allows for something like:

struct MyCryptography<R: Rng = SystemRng> { ... };

So long as libraries make the rng generic on the Rng trait then users can override it with whatever they want. Thus there's no need for any other override mechanism. If this thinking is flawed it'd be good to convince libs-api so that we don't end up on a wrong path.

[ChrisDenton]

It would be very helpful to see an RFC on this API before stabilization.

It was agreed in a previous meeting that at the very least there should be a design document the goes over all the issues and makes sure everyone's perspective is documented.

[joshtriplett]

having a (lock- or system-based) DefaultRng plus an InsecureThreadRng would be a better one in my opinion (something like Xoshiro or SFC requires a lot less thread-local state and doesn't rely on amortisation for performance).

We've had at this point an excessive number of arguments about the name of SystemRng, and trying to call something DefaultRng would reopen those.

But big 👍 for having an InsecureRng that makes zero promises about security, supports seeding, and can optimize for performance. (It's unclear at this point whether we'd end up offering one such Rng, or explicitly naming the algorithm. But it seems pretty clear at this point that we're likely to defer that to not block the initial round of API.)

[dhardy]

Rust targets a wide range of platforms. It's up to the system to decide how exactly it's RNG operates and I really hope that in future we will get a way to override the default source. Note that I speak as someone who professionally developed certified cryptographic software in Rust.

The argument is that a trait naturally allows overriding (i.e. you can implement Rng for custom types). Therefore there's not a need for another mechanism if you have a trait.

Since this has been meet with a thumb down and confusion I'll expand on this, especially as people seem to be talking past each other. In past libs-api meeting, it has been the consistent position of libs-api that a trait allows for something like:

Objections to this have been raised many times already. Dependency-injection via traits is not suitable for use-cases requiring a simple API (e.g. rand::rng()) or deep inside a function call tree.

---

having a (lock- or system-based) DefaultRng plus an InsecureThreadRng would be a better one in my opinion (something like Xoshiro or SFC requires a lot less thread-local state and doesn't rely on amortisation for performance).

We've had at this point an excessive number of arguments about the name of SystemRng, and trying to call something DefaultRng would reopen those.

I am not discussing the name, but rather two different things: a simple interface over the system random source and a derived interface (possibly using a local CSPRNG) which can provide additional guarantees (performance, definitely not failing after successful seeding).