Skip to content
Merged
Show file tree
Hide file tree
Changes from 68 commits
Commits
Show all changes
131 commits
Select commit Hold shift + click to select a range
9eb5f1c
Add memory utils
ernestognw Sep 4, 2024
2d397f4
Fix tests upgradeable
ernestognw Sep 4, 2024
2a0fb7e
Add docs
ernestognw Sep 5, 2024
a7e61c3
Make use of the library
ernestognw Sep 5, 2024
1aae8bb
Update docs/modules/ROOT/pages/utilities.adoc
ernestognw Oct 9, 2024
1b2679a
Merge branch 'master' into utils/memory
Amxx Mar 6, 2025
d514606
fix tests
Amxx Mar 6, 2025
14fa04e
Update contracts/utils/Memory.sol
ernestognw May 7, 2025
d0d55fc
Update contracts/utils/Memory.sol
arr00 May 7, 2025
7b3cb66
Add RLP library
ernestognw May 10, 2025
95149f8
Add TrieProof library
ernestognw May 11, 2025
ad5d4ac
up
ernestognw May 11, 2025
18540ef
Add docs
ernestognw May 11, 2025
163f27c
Workaround stack too deep
ernestognw May 24, 2025
c484289
Add Changesets
ernestognw May 29, 2025
e0d4790
Add more changesets
ernestognw Jun 7, 2025
a6f9053
Add FV and fuzz tests
ernestognw Jun 7, 2025
e2b5e4c
Merge branch 'master' into feature/rlp
ernestognw Jun 7, 2025
203d1a2
up
ernestognw Jun 7, 2025
48eabc1
docs
ernestognw Jun 7, 2025
63ced95
up pragma
ernestognw Jun 7, 2025
f342756
Add missing Bytes test
ernestognw Jun 7, 2025
23dba37
Add unit tests
ernestognw Jun 7, 2025
0cacca2
up pragma
ernestognw Jun 7, 2025
831e8ab
Move TrieProof
ernestognw Jun 7, 2025
5da111f
Fix countLeadingZeroes
ernestognw Jun 8, 2025
ba2293e
nits
ernestognw Jun 8, 2025
9409bc6
Improve
ernestognw Jun 8, 2025
e740dac
Fix
ernestognw Jun 8, 2025
0332ffe
Add Memory.sol library
ernestognw Jun 8, 2025
608e3cd
Merge branch 'master' into utils/memory
ernestognw Jun 8, 2025
ac92bb4
up
ernestognw Jun 8, 2025
6094bb7
Merge branch 'master' into utils/memory
ernestognw Jun 8, 2025
6bb96d5
WIP: Add more Memory functions
ernestognw Jun 8, 2025
860e5a8
up
ernestognw Jun 8, 2025
ecdb768
revert
ernestognw Jun 8, 2025
95907aa
Update docs
ernestognw Jun 8, 2025
124ccee
Nit
ernestognw Jun 8, 2025
c3237df
Finish fuzz tests and FV
ernestognw Jun 9, 2025
27f0a9b
up
ernestognw Jun 9, 2025
282ce39
up
ernestognw Jun 9, 2025
bdd2cf1
Add operations to Math.sol
ernestognw Jun 9, 2025
42c79f1
Add new equal, nibbles and countLeadingZeroes functions
ernestognw Jun 9, 2025
5754ab8
Rename countLeadingZeroes to clz
ernestognw Jun 9, 2025
44f0e14
up
ernestognw Jun 9, 2025
05c73bd
Pragma changes
ernestognw Jun 9, 2025
3a6fbf6
up
ernestognw Jun 9, 2025
e67e8b4
up
ernestognw Jul 4, 2025
3385718
Rename to in Math library and update corresponding tests for consis…
ernestognw Jul 9, 2025
40d7922
Update return types of reverseBits functions to match their respectiv…
ernestognw Jul 9, 2025
89860bc
Refactor reverseBits functions in to use fixed-size byte types
ernestognw Jul 9, 2025
9b58730
Test nits
ernestognw Jul 9, 2025
77ffa8c
Simplify
ernestognw Jul 9, 2025
ce91c80
up
ernestognw Jul 9, 2025
b3e3add
Move reverse functions to Bytes.sol
ernestognw Jul 9, 2025
2f3107c
Move Bytes.t.sol
ernestognw Jul 9, 2025
4383e01
Merge branch 'master' into feat/bytes-rlp
ernestognw Jul 9, 2025
5a44b11
up
ernestognw Jul 9, 2025
d6db2d7
Document
ernestognw Jul 9, 2025
3847050
Remove extra functions
ernestognw Jul 9, 2025
4fd1947
Update docs
ernestognw Jul 9, 2025
c4e0375
up
ernestognw Jul 9, 2025
acb14cb
Merge branch 'utils/memory' into feature/rlp
ernestognw Jul 9, 2025
2208006
Merge branch 'feat/math-reverse-bits' into feature/rlp
ernestognw Jul 9, 2025
13f4d8f
Merge branch 'feat/bytes-rlp' into feature/rlp
ernestognw Jul 9, 2025
502a520
Merge branch 'master' into feature/rlp
ernestognw Jul 12, 2025
aab9274
Merge branch 'master' into feature/rlp
Amxx Jul 15, 2025
aa26e48
up
Amxx Jul 15, 2025
948f0a1
Merge branch 'master' into feature/rlp
ernestognw Jul 31, 2025
d4bfb8b
Fix compilation
ernestognw Jul 31, 2025
138de7f
Remove dangling clz
ernestognw Jul 31, 2025
5efeb37
Make nibbles function private
ernestognw Jul 31, 2025
00ff228
Remove nibbles test
ernestognw Jul 31, 2025
c58c7fd
Remove TrieProof library
ernestognw Jul 31, 2025
d1aa944
Add some tests
ernestognw Aug 1, 2025
0925577
Merge branch 'master' into feature/rlp
ernestognw Aug 26, 2025
590b7d7
Use Bytes.concat
ernestognw Aug 26, 2025
8928039
Use Math.ternary
ernestognw Aug 27, 2025
bb027df
refactor encoding to reduce memory allocation
Amxx Sep 4, 2025
ec4a5c1
refactor tests to use ethers.encodeRlp as a reference
Amxx Sep 4, 2025
b75dc7c
fix comments
Amxx Sep 4, 2025
fc208c8
use mstore8 to avoid shift
Amxx Sep 4, 2025
fe0597c
decoding works
Amxx Sep 6, 2025
106764e
Add Memory.slice and refactor RLP read/decode
Amxx Sep 6, 2025
b55b1db
testing
Amxx Sep 6, 2025
e490d79
inspired by
Amxx Sep 6, 2025
bde8251
remove unecessary imports
Amxx Sep 6, 2025
8b6ea8c
read bool and string
Amxx Sep 7, 2025
dba642d
Bytes.Accumulator & RLP.encoder
Amxx Sep 8, 2025
5e3ce7c
reorder
Amxx Sep 8, 2025
4d39206
Merge branch 'master' into feature/rlp
Amxx Sep 10, 2025
902b222
document Bytes.accumulator
Amxx Sep 10, 2025
b301b06
pragma fix
Amxx Sep 10, 2025
1b112b2
mock stateless
Amxx Sep 10, 2025
9c0e924
slither
Amxx Sep 10, 2025
c48e623
Memory slice doc
Amxx Sep 10, 2025
b10ae8c
Update enums.js
Amxx Sep 10, 2025
d411318
avoid load error
Amxx Sep 10, 2025
df528af
fuzz test Memory slices
Amxx Sep 10, 2025
da61a5e
coverage
Amxx Sep 10, 2025
fadc8ac
Merge branch 'master' into feature/rlp
Amxx Sep 15, 2025
283ce7c
remove unused code
Amxx Sep 15, 2025
5561d52
Merge branch 'feature/rlp' of https://github.com/ernestognw/openzeppe…
Amxx Sep 15, 2025
33b57b5
Move Bytes.Accumulator to a dedicated file in utils/structs
Amxx Sep 15, 2025
8b86b1b
add changeset entry
Amxx Sep 15, 2025
785ae84
Update test/utils/Bytes.t.sol
Amxx Sep 15, 2025
452d4b8
Update .changeset/wise-webs-fly.md
Amxx Sep 15, 2025
00af6a4
pragma
Amxx Sep 16, 2025
27dd119
update solidity-coverage
Amxx Sep 16, 2025
93f4d4b
add slice support to Accumulator
Amxx Sep 16, 2025
0e503b4
revert solidity-coverage update
Amxx Sep 16, 2025
9cfaceb
Update .changeset/modern-moments-raise.md
Amxx Sep 17, 2025
07f5db8
move fuzzing test
Amxx Sep 17, 2025
8534802
zero out-of-slice bytes when doing load(Slice,uint256)
Amxx Sep 17, 2025
c3df360
doc
Amxx Sep 17, 2025
8c98d4a
dec -> hex
Amxx Sep 17, 2025
6efa28d
up
ernestognw Sep 17, 2025
7478616
Reorder private functions
ernestognw Sep 18, 2025
fb62ca3
Docs
ernestognw Sep 18, 2025
698221f
Add to CHANGELOG
ernestognw Sep 18, 2025
5fa4c3e
Merge branch 'master' into feature/rlp
ernestognw Sep 18, 2025
2bd9cc4
Delete .changeset/lovely-cooks-add.md
ernestognw Sep 18, 2025
fefd039
Uncomment invalid tests
ernestognw Sep 18, 2025
f2e5c81
up
ernestognw Sep 18, 2025
ff4c153
dynamic allocation of space for lists
Amxx Sep 18, 2025
dd242c2
use a single custom error
Amxx Sep 18, 2025
46914e9
Update contracts/utils/RLP.sol
Amxx Sep 18, 2025
fdcb15f
Memory pointer equality
Amxx Sep 18, 2025
0c5d508
test panic code of out-of-bound slice operations
Amxx Sep 19, 2025
42dd3d3
Avoid Accumulators NatSpec to show up on docs sidebar
ernestognw Sep 19, 2025
371595d
Pointer forward
Amxx Sep 19, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/khaki-hats-leave.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'openzeppelin-solidity': minor
---

`Bytes`: Add a `nibbles` function to split each byte into two nibbles.
5 changes: 5 additions & 0 deletions .changeset/lovely-cooks-add.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'openzeppelin-solidity': minor
---

`RLP`: Add library for Ethereum's Recursive Length Prefix encoding/decoding.
5 changes: 5 additions & 0 deletions .changeset/shaky-phones-mix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'openzeppelin-solidity': minor
---

`TrieProof`: Add library for verifying Ethereum Merkle-Patricia trie inclusion proofs.
5 changes: 5 additions & 0 deletions .changeset/ten-steaks-try.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'openzeppelin-solidity': minor
---

`Bytes`: Add an `equal` function to compare byte buffers.
5 changes: 5 additions & 0 deletions .changeset/whole-cats-find.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'openzeppelin-solidity': minor
---

`Bytes`: Add a `clz` function to count the leading zero bytes in a `uint256` value.
22 changes: 22 additions & 0 deletions contracts/utils/Bytes.sol
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,28 @@ library Bytes {
return result;
}

/// @dev Split each byte in `value` into two nibbles (4 bits each).
function nibbles(bytes memory value) internal pure returns (bytes memory) {
uint256 length = value.length;
bytes memory nibbles_ = new bytes(length * 2);
for (uint256 i = 0; i < length; i++) {
(nibbles_[i * 2], nibbles_[i * 2 + 1]) = (value[i] & 0xf0, value[i] & 0x0f);
}
return nibbles_;
}
Copy link

@0xClandestine 0xClandestine Jul 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The toNibbles function converts a bytes array like 0xABCD into a nibble-expanded format like 0xA00BC00D. The existing implementation does this by iterating linearly over the input, which works correctly but is extremely inefficient.

My approach processes the input in parallel, reducing time complexity from O(n) to O(n / 32). This is inspired by bit-hacking techniques for interleaving bits, as described in the following resources:

https://stackoverflow.com/questions/38881877/bit-hack-expanding-bits
https://graphics.stanford.edu/~seander/bithacks.html#InterleaveBMN

    // 0xABCD -> 0xA0B0C0D0
    function expandNibbles(uint128 x) internal pure returns (uint256) {
        uint256 y = uint256(x);
        y = (y | (y << 64)) & 0x0000000000000000FFFFFFFFFFFFFFFF0000000000000000FFFFFFFFFFFFFFFF;
        y = (y | (y << 32)) & 0x00000000FFFFFFFF00000000FFFFFFFF00000000FFFFFFFF00000000FFFFFFFF;
        y = (y | (y << 16)) & 0x0000FFFF0000FFFF0000FFFF0000FFFF0000FFFF0000FFFF0000FFFF0000FFFF;
        y = (y | (y << 8)) & 0x00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF;
        y = (y | (y << 4)) & 0x0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F;
        return y;
    }
    
    // Puts nibbles in their respective height (high/low). 
    // 0xA0B0C0D0 -> 0xA00BC00D
    function toNibbles(uint128 x) internal pure returns (uint256) {
        uint256 y = expandNibbles(x);
        uint256 high = y & 0x00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF;
        uint256 low = y & 0xFF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00;
        high = high >> 4;
        return high | low;
    }

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bringing it all together:

function toNibbles(bytes memory input) internal pure returns (bytes memory output) {
    /// @solidity memory-safe-assembly
    assembly {
        output := mload(0x40)
        mstore(output, mul(mload(input), 2))
        mstore(0x40, add(output, add(0x20, mul(mload(input), 2))))
        
        for { let i := 0 } lt(i, mload(input)) { i := add(i, 16) } {
            let x := shr(128, mload(add(add(input, 0x20), i)))
            x := and(or(x, shl(64, x)), 0x0000000000000000FFFFFFFFFFFFFFFF0000000000000000FFFFFFFFFFFFFFFF)
            x := and(or(x, shl(32, x)), 0x00000000FFFFFFFF00000000FFFFFFFF00000000FFFFFFFF00000000FFFFFFFF)
            x := and(or(x, shl(16, x)), 0x0000FFFF0000FFFF0000FFFF0000FFFF0000FFFF0000FFFF0000FFFF0000FFFF)
            x := and(or(x, shl(8, x)),  0x00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF00FF)
            let mask := 0x0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F0F
            x := and(or(x, shl(4, x)),  mask)
            x := or(shl(4, and(shr(4, x), mask)), and(x, mask))
            mstore(add(add(output, 0x20), mul(i, 2)), x)
        }
    }
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can probably be simplified further by adjusting the first 4 masks, working on that now.


/**
* @dev Returns true if the two byte buffers are equal.
*/
function equal(bytes memory a, bytes memory b) internal pure returns (bool) {
return a.length == b.length && keccak256(a) == keccak256(b);
}

/// @dev Counts the number of leading zeros in a uint256.
function clz(uint256 x) internal pure returns (uint256) {
return Math.ternary(x == 0, 32, 31 - Math.log256(x));
}

/**
* @dev Moves the content of `buffer`, from `start` (included) to the end of `buffer` to the start of that buffer.
*
Expand Down
3 changes: 3 additions & 0 deletions contracts/utils/README.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ Miscellaneous contracts and libraries containing utility functions you can use t
* {InteroperableAddress}: Library for formatting and parsing ERC-7930 interoperable addresses.
* {Blockhash}: A library for accessing historical block hashes beyond the standard 256 block limit utilizing EIP-2935's historical blockhash functionality.
* {Time}: A library that provides helpers for manipulating time-related objects, including a `Delay` type.
* {RLP}: Library for encoding and decoding data in Ethereum's Recursive Length Prefix format.

[NOTE]
====
Expand Down Expand Up @@ -143,3 +144,5 @@ Ethereum contracts have no native concept of an interface, so applications must
{{Blockhash}}

{{Time}}

{{RLP}}
Loading
Loading