Skip to content

Commit 468abc7

Browse files
boazsegevboazsegev
committed
finish testing randomness for up to to 2TB of randomness and doc tests.
1 parent 3b8f98a commit 468abc7

8 files changed

+1951
-52
lines changed

‎fio-stl.md

+330
Original file line numberDiff line numberDiff line change
@@ -1845,6 +1845,53 @@ The functions support up to 2048 bits of multi-precision. The 4096 bit type is m
18451845
- `void fio_u512_mul(fio_u512 *t, fio_u256 *a, fio_u256 *b)` - performs MUL (`t=a*b`) operation.
18461846

18471847

1848+
-------------------------------------------------------------------------------
1849+
1850+
## Core Randomness
1851+
1852+
The core module provides macros for generating semi-deterministic Pseudo-Random Number Generator functions.
1853+
1854+
#### `FIO_DEFINE_RANDOM128_FN`
1855+
1856+
```c
1857+
#define FIO_DEFINE_RANDOM128_FN(extern, name, reseed_log, seed_offset)
1858+
```
1859+
1860+
Defines a semi-deterministic Pseudo-Random 128 bit Number Generator function.
1861+
1862+
The following functions will be defined:
1863+
1864+
```c
1865+
extern fio_u128 name##128(void); // returns 128 bits
1866+
extern uint64_t name##64(void); // returns 64 bits (simply half of the 128 bit result)
1867+
extern void name##_bytes(void *buffer, size_t len); // fills a buffer
1868+
extern void name##_reset(void); // resets the state of the PRNG
1869+
```
1870+
1871+
If `reseed_log` is non-zero and less than 32, the PNGR is no longer deterministic, as it will automatically re-seeds itself every `1 << reseed_log` iterations using a loop measuring both time and CPU 'jitter'.
1872+
1873+
If `extern` is `static` or `FIO_SFUNC`, a `static` function will be defined.
1874+
1875+
#### `FIO_DEFINE_RANDOM64_FN`
1876+
1877+
```c
1878+
#define FIO_DEFINE_RANDOM64_FN(extern, name, reseed_log, seed_offset)
1879+
```
1880+
1881+
Defines a semi-deterministic Pseudo-Random 64 bit Number Generator function.
1882+
1883+
The following functions will be defined:
1884+
1885+
```c
1886+
extern uint64_t name##64(void); // returns 64 bits
1887+
extern void name##_bytes(void *buffer, size_t len); // fills buffer
1888+
extern void name##_reset(void); // resets the state of the PRNG
1889+
```
1890+
1891+
If `reseed_log` is non-zero and less than 32, the PNGR is no longer deterministic, as it will automatically re-seeds itself every `1 << reseed_log` iterations using a loop measuring both time and CPU 'jitter'.
1892+
1893+
If `extern` is `static` or `FIO_SFUNC`, static function will be defined.
1894+
18481895
-------------------------------------------------------------------------------
18491896

18501897
## Core Binary Strings and Buffer Helpers
@@ -16342,6 +16389,289 @@ Verification value is 0x00000001 - Testing took 1063.513065 seconds
1634216389
-------------------------------------------------------------------------------
1634316390
```
1634416391

16392+
-------------------------------------------------------------------------------
16393+
## Pseudo-Random Function Testing
16394+
16395+
Testing the Pseudo-Random Number Generator (PRNG) Functions isn't somewhat of a chore, as a complete test could take a week or so to complete and my laptop wasn't designed for such long running intensive tasks.
16396+
16397+
Tests were conducted by utilizing [PractRand](https://pracrand.sourceforge.net) as well as the [xoshiro](http://xoshiro.di.unimi.it/hwd.php) test code, running both up to 2TB of random data (which took about 1 night).
16398+
16399+
Results are as follows:
16400+
16401+
-------------------------------------------------------------------------------
16402+
### `fio_rand`
16403+
16404+
The following are the tests for the core `fio_rand64` and `fio_rand_bytes` functions provided when using `FIO_RAND`.
16405+
16406+
```txt
16407+
# ./tmp/rnd -p fio | RNG_test stdin
16408+
RNG_test using PractRand version 0.95
16409+
RNG = RNG_stdin, seed = unknown
16410+
test set = core, folding = standard(unknown format)
16411+
16412+
rng=RNG_stdin, seed=unknown
16413+
length= 256 megabytes (2^28 bytes), time= 3.0 seconds
16414+
no anomalies in 217 test result(s)
16415+
16416+
rng=RNG_stdin, seed=unknown
16417+
length= 512 megabytes (2^29 bytes), time= 6.5 seconds
16418+
no anomalies in 232 test result(s)
16419+
16420+
rng=RNG_stdin, seed=unknown
16421+
length= 1 gigabyte (2^30 bytes), time= 13.0 seconds
16422+
no anomalies in 251 test result(s)
16423+
16424+
rng=RNG_stdin, seed=unknown
16425+
length= 2 gigabytes (2^31 bytes), time= 25.2 seconds
16426+
no anomalies in 269 test result(s)
16427+
16428+
rng=RNG_stdin, seed=unknown
16429+
length= 4 gigabytes (2^32 bytes), time= 48.2 seconds
16430+
no anomalies in 283 test result(s)
16431+
16432+
rng=RNG_stdin, seed=unknown
16433+
length= 8 gigabytes (2^33 bytes), time= 96.5 seconds
16434+
Test Name Raw Processed Evaluation
16435+
BCFN(2+7,13-2,T) R= -9.1 p =1-3.9e-5 unusual
16436+
...and 299 test result(s) without anomalies
16437+
16438+
rng=RNG_stdin, seed=unknown
16439+
length= 16 gigabytes (2^34 bytes), time= 190 seconds
16440+
no anomalies in 315 test result(s)
16441+
16442+
rng=RNG_stdin, seed=unknown
16443+
length= 32 gigabytes (2^35 bytes), time= 376 seconds
16444+
no anomalies in 328 test result(s)
16445+
16446+
rng=RNG_stdin, seed=unknown
16447+
length= 64 gigabytes (2^36 bytes), time= 752 seconds
16448+
no anomalies in 344 test result(s)
16449+
16450+
rng=RNG_stdin, seed=unknown
16451+
length= 128 gigabytes (2^37 bytes), time= 1498 seconds
16452+
no anomalies in 359 test result(s)
16453+
16454+
rng=RNG_stdin, seed=unknown
16455+
length= 256 gigabytes (2^38 bytes), time= 2978 seconds
16456+
no anomalies in 372 test result(s)
16457+
16458+
rng=RNG_stdin, seed=unknown
16459+
length= 512 gigabytes (2^39 bytes), time= 9897 seconds
16460+
no anomalies in 387 test result(s)
16461+
16462+
rng=RNG_stdin, seed=unknown
16463+
length= 1 terabyte (2^40 bytes), time= 21004 seconds
16464+
no anomalies in 401 test result(s)
16465+
16466+
rng=RNG_stdin, seed=unknown
16467+
length= 2 terabytes (2^41 bytes), time= 43108 seconds
16468+
no anomalies in 413 test result(s)
16469+
```
16470+
16471+
16472+
-------------------------------------------------------------------------------
16473+
### `FIO_DEFINE_RANDOM128_FN`
16474+
16475+
The following are the tests for the built-in `FIO_DEFINE_RANDOM128_FN` macro using the deterministic PRNG (where the auto-reseeding `reseed_log` is set to `0`).
16476+
16477+
```txt
16478+
# ./tmp/rnd -p M128 | RNG_test stdin
16479+
RNG_test using PractRand version 0.95
16480+
RNG = RNG_stdin, seed = unknown
16481+
test set = core, folding = standard(unknown format)
16482+
16483+
rng=RNG_stdin, seed=unknown
16484+
length= 256 megabytes (2^28 bytes), time= 2.4 seconds
16485+
no anomalies in 217 test result(s)
16486+
16487+
rng=RNG_stdin, seed=unknown
16488+
length= 512 megabytes (2^29 bytes), time= 5.0 seconds
16489+
no anomalies in 232 test result(s)
16490+
16491+
rng=RNG_stdin, seed=unknown
16492+
length= 1 gigabyte (2^30 bytes), time= 10.1 seconds
16493+
no anomalies in 251 test result(s)
16494+
16495+
rng=RNG_stdin, seed=unknown
16496+
length= 2 gigabytes (2^31 bytes), time= 20.6 seconds
16497+
no anomalies in 269 test result(s)
16498+
16499+
rng=RNG_stdin, seed=unknown
16500+
length= 4 gigabytes (2^32 bytes), time= 41.2 seconds
16501+
no anomalies in 283 test result(s)
16502+
16503+
rng=RNG_stdin, seed=unknown
16504+
length= 8 gigabytes (2^33 bytes), time= 83.9 seconds
16505+
no anomalies in 300 test result(s)
16506+
16507+
rng=RNG_stdin, seed=unknown
16508+
length= 16 gigabytes (2^34 bytes), time= 167 seconds
16509+
no anomalies in 315 test result(s)
16510+
16511+
rng=RNG_stdin, seed=unknown
16512+
length= 32 gigabytes (2^35 bytes), time= 330 seconds
16513+
no anomalies in 328 test result(s)
16514+
16515+
rng=RNG_stdin, seed=unknown
16516+
length= 64 gigabytes (2^36 bytes), time= 665 seconds
16517+
no anomalies in 344 test result(s)
16518+
16519+
rng=RNG_stdin, seed=unknown
16520+
length= 128 gigabytes (2^37 bytes), time= 1329 seconds
16521+
no anomalies in 359 test result(s)
16522+
16523+
rng=RNG_stdin, seed=unknown
16524+
length= 256 gigabytes (2^38 bytes), time= 2630 seconds
16525+
no anomalies in 372 test result(s)
16526+
16527+
rng=RNG_stdin, seed=unknown
16528+
length= 512 gigabytes (2^39 bytes), time= 6846 seconds
16529+
no anomalies in 387 test result(s)
16530+
16531+
rng=RNG_stdin, seed=unknown
16532+
length= 1 terabyte (2^40 bytes), time= 18633 seconds
16533+
no anomalies in 401 test result(s)
16534+
16535+
rng=RNG_stdin, seed=unknown
16536+
length= 2 terabytes (2^41 bytes), time= 39061 seconds
16537+
no anomalies in 413 test result(s)
16538+
```
16539+
16540+
-------------------------------------------------------------------------------
16541+
### `FIO_DEFINE_RANDOM64_FN`
16542+
16543+
The following are the tests for the built-in `FIO_DEFINE_RANDOM64_FN` macro using the deterministic PRNG (where the auto-reseeding `reseed_log` is set to `0`).
16544+
16545+
16546+
```txt
16547+
# ./tmp/rnd -p M64 | RNG_test stdin
16548+
RNG_test using PractRand version 0.95
16549+
RNG = RNG_stdin, seed = unknown
16550+
test set = core, folding = standard(unknown format)
16551+
16552+
rng=RNG_stdin, seed=unknown
16553+
length= 256 megabytes (2^28 bytes), time= 2.6 seconds
16554+
no anomalies in 217 test result(s)
16555+
16556+
rng=RNG_stdin, seed=unknown
16557+
length= 512 megabytes (2^29 bytes), time= 5.6 seconds
16558+
no anomalies in 232 test result(s)
16559+
16560+
rng=RNG_stdin, seed=unknown
16561+
length= 1 gigabyte (2^30 bytes), time= 11.0 seconds
16562+
no anomalies in 251 test result(s)
16563+
16564+
rng=RNG_stdin, seed=unknown
16565+
length= 2 gigabytes (2^31 bytes), time= 21.9 seconds
16566+
no anomalies in 269 test result(s)
16567+
16568+
rng=RNG_stdin, seed=unknown
16569+
length= 4 gigabytes (2^32 bytes), time= 42.9 seconds
16570+
no anomalies in 283 test result(s)
16571+
16572+
rng=RNG_stdin, seed=unknown
16573+
length= 8 gigabytes (2^33 bytes), time= 85.3 seconds
16574+
no anomalies in 300 test result(s)
16575+
16576+
rng=RNG_stdin, seed=unknown
16577+
length= 16 gigabytes (2^34 bytes), time= 168 seconds
16578+
no anomalies in 315 test result(s)
16579+
16580+
rng=RNG_stdin, seed=unknown
16581+
length= 32 gigabytes (2^35 bytes), time= 331 seconds
16582+
no anomalies in 328 test result(s)
16583+
16584+
rng=RNG_stdin, seed=unknown
16585+
length= 64 gigabytes (2^36 bytes), time= 666 seconds
16586+
no anomalies in 344 test result(s)
16587+
16588+
rng=RNG_stdin, seed=unknown
16589+
length= 128 gigabytes (2^37 bytes), time= 1328 seconds
16590+
no anomalies in 359 test result(s)
16591+
16592+
rng=RNG_stdin, seed=unknown
16593+
length= 256 gigabytes (2^38 bytes), time= 2626 seconds
16594+
no anomalies in 372 test result(s)
16595+
16596+
rng=RNG_stdin, seed=unknown
16597+
length= 512 gigabytes (2^39 bytes), time= 6834 seconds
16598+
no anomalies in 387 test result(s)
16599+
16600+
rng=RNG_stdin, seed=unknown
16601+
length= 1 terabyte (2^40 bytes), time= 18619 seconds
16602+
no anomalies in 401 test result(s)
16603+
16604+
rng=RNG_stdin, seed=unknown
16605+
length= 2 terabytes (2^41 bytes), time= 39032 seconds
16606+
no anomalies in 413 test result(s)
16607+
```
16608+
16609+
16610+
-------------------------------------------------------------------------------
16611+
16612+
```txt
16613+
# ./tmp/rnd -p M128 | RNG_test stdin
16614+
RNG_test using PractRand version 0.95
16615+
RNG = RNG_stdin, seed = unknown
16616+
test set = core, folding = standard(unknown format)
16617+
16618+
rng=RNG_stdin, seed=unknown
16619+
length= 256 megabytes (2^28 bytes), time= 2.4 seconds
16620+
no anomalies in 217 test result(s)
16621+
16622+
rng=RNG_stdin, seed=unknown
16623+
length= 512 megabytes (2^29 bytes), time= 5.0 seconds
16624+
no anomalies in 232 test result(s)
16625+
16626+
rng=RNG_stdin, seed=unknown
16627+
length= 1 gigabyte (2^30 bytes), time= 10.1 seconds
16628+
no anomalies in 251 test result(s)
16629+
16630+
rng=RNG_stdin, seed=unknown
16631+
length= 2 gigabytes (2^31 bytes), time= 20.6 seconds
16632+
no anomalies in 269 test result(s)
16633+
16634+
rng=RNG_stdin, seed=unknown
16635+
length= 4 gigabytes (2^32 bytes), time= 41.2 seconds
16636+
no anomalies in 283 test result(s)
16637+
16638+
rng=RNG_stdin, seed=unknown
16639+
length= 8 gigabytes (2^33 bytes), time= 83.9 seconds
16640+
no anomalies in 300 test result(s)
16641+
16642+
rng=RNG_stdin, seed=unknown
16643+
length= 16 gigabytes (2^34 bytes), time= 167 seconds
16644+
no anomalies in 315 test result(s)
16645+
16646+
rng=RNG_stdin, seed=unknown
16647+
length= 32 gigabytes (2^35 bytes), time= 330 seconds
16648+
no anomalies in 328 test result(s)
16649+
16650+
rng=RNG_stdin, seed=unknown
16651+
length= 64 gigabytes (2^36 bytes), time= 665 seconds
16652+
no anomalies in 344 test result(s)
16653+
16654+
rng=RNG_stdin, seed=unknown
16655+
length= 128 gigabytes (2^37 bytes), time= 1329 seconds
16656+
no anomalies in 359 test result(s)
16657+
16658+
rng=RNG_stdin, seed=unknown
16659+
length= 256 gigabytes (2^38 bytes), time= 2630 seconds
16660+
no anomalies in 372 test result(s)
16661+
16662+
rng=RNG_stdin, seed=unknown
16663+
length= 512 gigabytes (2^39 bytes), time= 6846 seconds
16664+
no anomalies in 387 test result(s)
16665+
16666+
rng=RNG_stdin, seed=unknown
16667+
length= 1 terabyte (2^40 bytes), time= 18633 seconds
16668+
no anomalies in 401 test result(s)
16669+
16670+
rng=RNG_stdin, seed=unknown
16671+
length= 2 terabytes (2^41 bytes), time= 39061 seconds
16672+
no anomalies in 413 test result(s)
16673+
```
16674+
1634516675
-------------------------------------------------------------------------------
1634616676

1634716677
-------------------------------------------------------------------------------

0 commit comments

Comments
 (0)