Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -6,59 +6,53 @@ Commit hash: TBD (link)
## Files to Audit
Note: Paths relative to `aztec-packages/barretenberg/cpp/src/barretenberg`

### Core Merkle Tree Implementation
1. `crypto/merkle_tree/merkle_tree.hpp`
2. `crypto/merkle_tree/memory_tree.hpp`
3. `crypto/merkle_tree/hash.hpp`
4. `crypto/merkle_tree/hash_path.hpp`
5. `crypto/merkle_tree/types.hpp`
6. `crypto/merkle_tree/index.hpp`
7. `crypto/merkle_tree/response.hpp`
8. `crypto/merkle_tree/signal.hpp`

### Storage Implementations
9. `crypto/merkle_tree/memory_store.hpp`
10. `crypto/merkle_tree/lmdb_store/lmdb_tree_store.cpp`
11. `crypto/merkle_tree/lmdb_store/lmdb_tree_store.hpp`

### Node Store
12. `crypto/merkle_tree/node_store/array_store.hpp`
13. `crypto/merkle_tree/node_store/cached_content_addressed_tree_store.hpp`
14. `crypto/merkle_tree/node_store/content_addressed_cache.hpp`
15. `crypto/merkle_tree/node_store/tree_meta.hpp`
### Core Types and Utilities
1. `crypto/merkle_tree/types.hpp`
2. `crypto/merkle_tree/hash.hpp`
3. `crypto/merkle_tree/hash_path.hpp`
4. `crypto/merkle_tree/response.hpp`
5. `crypto/merkle_tree/signal.hpp`

### Storage Layer
6. `crypto/merkle_tree/lmdb_store/lmdb_tree_store.hpp`
7. `crypto/merkle_tree/lmdb_store/lmdb_tree_store.cpp`

### Node Store (Cache + Metadata)
8. `crypto/merkle_tree/node_store/cached_content_addressed_tree_store.hpp`
9. `crypto/merkle_tree/node_store/content_addressed_cache.hpp`
10. `crypto/merkle_tree/node_store/tree_meta.hpp`

### Append-Only Tree
16. `crypto/merkle_tree/append_only_tree/content_addressed_append_only_tree.hpp`
11. `crypto/merkle_tree/append_only_tree/content_addressed_append_only_tree.hpp`

### Indexed Tree
17. `crypto/merkle_tree/indexed_tree/content_addressed_indexed_tree.hpp`
18. `crypto/merkle_tree/indexed_tree/indexed_leaf.hpp`
19. `crypto/merkle_tree/indexed_tree/fixtures.hpp`

### Nullifier Tree
20. `crypto/merkle_tree/nullifier_tree/nullifier_tree.cpp`
21. `crypto/merkle_tree/nullifier_tree/nullifier_tree.hpp`
22. `crypto/merkle_tree/nullifier_tree/nullifier_memory_tree.hpp`
23. `crypto/merkle_tree/nullifier_tree/nullifier_leaf.hpp`

### Test Fixtures
24. `crypto/merkle_tree/fixtures.hpp`
25. `crypto/merkle_tree/test_fixtures.hpp`
12. `crypto/merkle_tree/indexed_tree/content_addressed_indexed_tree.hpp`
13. `crypto/merkle_tree/indexed_tree/indexed_leaf.hpp`

## Summary of Module

The `merkle_tree` module provides a comprehensive implementation of Merkle tree data structures for the Aztec protocol, including standard Merkle trees, append-only trees, indexed trees, and nullifier trees. The module supports multiple hash policies (Pedersen hash and Poseidon2) and provides both in-memory and persistent LMDB-backed storage options. The core implementation includes content-addressed storage with caching, allowing efficient tree operations and proof generation. The append-only tree implementation is optimized for sequential insertions, while the indexed tree maintains an ordered set of leaves with nullifier functionality. The nullifier tree is a specialized indexed tree used for tracking spent notes in the Aztec protocol. The module includes utilities for computing Merkle paths, tree roots, and managing tree metadata across multiple storage backends.
The `merkle_tree` module implements the persistent Merkle tree infrastructure for the Aztec protocol's world state. It provides two tree variants: append-only trees (used for NOTE_HASH_TREE, L1_TO_L2_MESSAGE_TREE, and ARCHIVE) and indexed trees (used for NULLIFIER_TREE and PUBLIC_DATA_TREE). Both use Poseidon2 hashing.

The storage architecture is two-layered: `ContentAddressedCachedTreeStore` wraps an in-memory cache (`ContentAddressedCache`) over a persistent LMDB backend (`LMDBTreeStore`). All tree operations are asynchronous, using thread pools and callback-based completion. The system supports block-level history, forking, checkpointing, and rollback for the world state managed by `world_state/`.

## Test Files
1. `crypto/merkle_tree/merkle_tree.test.cpp`
2. `crypto/merkle_tree/memory_tree.test.cpp`
3. `crypto/merkle_tree/fixtures.test.cpp`
4. `crypto/merkle_tree/lmdb_store/lmdb_tree_store.test.cpp`
5. `crypto/merkle_tree/node_store/content_addressed_cache.test.cpp`
6. `crypto/merkle_tree/append_only_tree/content_addressed_append_only_tree.test.cpp`
7. `crypto/merkle_tree/indexed_tree/content_addressed_indexed_tree.test.cpp`
8. `crypto/merkle_tree/nullifier_tree/nullifier_tree.test.cpp`
9. `crypto/merkle_tree/nullifier_tree/nullifier_memory_tree.test.cpp`
1. `crypto/merkle_tree/memory_tree.test.cpp`
2. `crypto/merkle_tree/fixtures.test.cpp`
3. `crypto/merkle_tree/lmdb_store/lmdb_tree_store.test.cpp`
4. `crypto/merkle_tree/node_store/content_addressed_cache.test.cpp`
5. `crypto/merkle_tree/append_only_tree/content_addressed_append_only_tree.test.cpp`
6. `crypto/merkle_tree/indexed_tree/content_addressed_indexed_tree.test.cpp`
7. `crypto/merkle_tree/nullifier_tree/nullifier_memory_tree.test.cpp`

### Test Infrastructure
Non-production code used for testing and benchmarking the production files above:

- `crypto/merkle_tree/memory_tree.hpp` — Simple in-memory tree. Also used by vm2/simulation as a lightweight tree for AVM transaction execution.
- `crypto/merkle_tree/fixtures.hpp` — Test utility functions (random directories, values, thread pools).
- `crypto/merkle_tree/test_fixtures.hpp` — GTest assertion wrappers for tree state verification.
- `crypto/merkle_tree/node_store/array_store.hpp` — Lightweight in-memory store substitute for LMDB in tests/benchmarks.
- `crypto/merkle_tree/nullifier_tree/nullifier_memory_tree.hpp` — Reference implementation of an indexed nullifier tree, used for differential testing of `ContentAddressedIndexedTree`.
- `crypto/merkle_tree/nullifier_tree/nullifier_leaf.hpp` — Leaf types used by `nullifier_memory_tree.hpp`.

## Security Mechanisms
None identified.
1 change: 0 additions & 1 deletion barretenberg/cpp/src/barretenberg/benchmark/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ add_subdirectory(chonk_bench)
add_subdirectory(pippenger_bench)
add_subdirectory(relations_bench)
add_subdirectory(poseidon2_bench)
add_subdirectory(merkle_tree_bench)
add_subdirectory(indexed_tree_bench)
add_subdirectory(append_only_tree_bench)
add_subdirectory(ultra_bench)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ using namespace bb::crypto::merkle_tree;
namespace {
using StoreType = ContentAddressedCachedTreeStore<bb::fr>;

using Pedersen = ContentAddressedAppendOnlyTree<StoreType, PedersenHashPolicy>;
using Poseidon2 = ContentAddressedAppendOnlyTree<StoreType, Poseidon2HashPolicy>;

const size_t TREE_DEPTH = 32;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ using namespace bb::crypto::merkle_tree;
using StoreType = ContentAddressedCachedTreeStore<NullifierLeafValue>;

using Poseidon2 = ContentAddressedIndexedTree<StoreType, Poseidon2HashPolicy>;
using Pedersen = ContentAddressedIndexedTree<StoreType, PedersenHashPolicy>;

const size_t TREE_DEPTH = 40;
const size_t MAX_BATCH_SIZE = 64;
Expand Down

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// === AUDIT STATUS ===
// internal: { status: Planned, auditors: [Raju], commit: }
// internal: { status: Planned, auditors: [], commit: }
// external_1: { status: not started, auditors: [], commit: }
// external_2: { status: not started, auditors: [], commit: }
// =====================
Expand Down
11 changes: 1 addition & 10 deletions barretenberg/cpp/src/barretenberg/crypto/merkle_tree/hash.hpp
Original file line number Diff line number Diff line change
@@ -1,24 +1,15 @@
// === AUDIT STATUS ===
// internal: { status: Planned, auditors: [Raju], commit: }
// internal: { status: Planned, auditors: [], commit: }
// external_1: { status: not started, auditors: [], commit: }
// external_2: { status: not started, auditors: [], commit: }
// =====================

#pragma once
#include "barretenberg/crypto/pedersen_hash/pedersen.hpp"
#include "barretenberg/crypto/poseidon2/poseidon2.hpp"
#include <vector>

namespace bb::crypto::merkle_tree {

struct PedersenHashPolicy {
static fr hash(const std::vector<fr>& inputs) { return crypto::pedersen_hash::hash(inputs); }

static fr hash_pair(const fr& lhs, const fr& rhs) { return hash(std::vector<fr>({ lhs, rhs })); }

static fr zero_hash() { return fr::zero(); }
};

struct Poseidon2HashPolicy {
static fr hash(const std::vector<fr>& inputs)
{
Expand Down
Original file line number Diff line number Diff line change
@@ -1,56 +1,23 @@
// === AUDIT STATUS ===
// internal: { status: Planned, auditors: [Raju], commit: }
// internal: { status: Planned, auditors: [], commit: }
// external_1: { status: not started, auditors: [], commit: }
// external_2: { status: not started, auditors: [], commit: }
// =====================

#pragma once
#include "barretenberg/stdlib/primitives/field/field.hpp"
#include "hash.hpp"
#include <algorithm>
#include <vector>

namespace bb::crypto::merkle_tree {

using fr_hash_path = std::vector<std::pair<fr, fr>>;
using fr_sibling_path = std::vector<fr>;
template <typename Ctx> using hash_path = std::vector<std::pair<bb::stdlib::field_t<Ctx>, bb::stdlib::field_t<Ctx>>>;

inline fr_hash_path get_random_hash_path(size_t const& tree_depth)
{
fr_hash_path path;
for (size_t i = 0; i < tree_depth; ++i) {
path.push_back(std::make_pair(fr::random_element(), fr::random_element()));
}
return path;
}

template <typename Ctx> inline hash_path<Ctx> create_witness_hash_path(Ctx& ctx, fr_hash_path const& input)
{
hash_path<Ctx> result;
std::transform(input.begin(), input.end(), std::back_inserter(result), [&](auto const& v) {
return std::make_pair(bb::stdlib::field_t(bb::stdlib::witness_t(&ctx, v.first)),
bb::stdlib::field_t(bb::stdlib::witness_t(&ctx, v.second)));
});
return result;
}

} // namespace bb::crypto::merkle_tree

// We add to std namespace as fr_hash_path is actually a std::vector, and this is the only way
// to achieve effective ADL.
namespace std {
template <typename Ctx>
inline std::ostream& operator<<(std::ostream& os, bb::crypto::merkle_tree::hash_path<Ctx> const& path)
{
os << "[\n";
for (size_t i = 0; i < path.size(); ++i) {
os << " (" << i << ": " << path[i].first << ", " << path[i].second << ")\n";
}
os << "]\n";
return os;
}

inline std::ostream& operator<<(std::ostream& os, bb::crypto::merkle_tree::fr_hash_path const& path)
{
os << "[\n";
Expand Down
12 changes: 0 additions & 12 deletions barretenberg/cpp/src/barretenberg/crypto/merkle_tree/index.hpp

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// === AUDIT STATUS ===
// internal: { status: Planned, auditors: [Raju], commit: }
// internal: { status: Planned, auditors: [], commit: }
// external_1: { status: not started, auditors: [], commit: }
// external_2: { status: not started, auditors: [], commit: }
// =====================
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
#include "../node_store/array_store.hpp"
#include "../nullifier_tree/nullifier_memory_tree.hpp"
#include "../test_fixtures.hpp"
#include "./fixtures.hpp"
#include "barretenberg/common/streams.hpp"
#include "barretenberg/common/test.hpp"
#include "barretenberg/common/thread_pool.hpp"
Expand Down Expand Up @@ -38,6 +37,22 @@ using PublicDataTreeType = ContentAddressedIndexedTree<PublicDataStore, Poseidon
using CompletionCallback = TreeType::AddCompletionCallbackWithWitness;
using SequentialCompletionCallback = TreeType::AddSequentiallyCompletionCallbackWithWitness;

using IndexedNullifierLeafType = IndexedLeaf<NullifierLeafValue>;
using IndexedPublicDataLeafType = IndexedLeaf<PublicDataLeafValue>;

inline IndexedNullifierLeafType create_indexed_nullifier_leaf(const fr& value, index_t nextIndex, const fr& nextValue)
{
return IndexedNullifierLeafType{ NullifierLeafValue(value), nextIndex, nextValue };
}

inline IndexedPublicDataLeafType create_indexed_public_data_leaf(const fr& slot,
const fr& value,
index_t nextIndex,
const fr& nextValue)
{
return IndexedPublicDataLeafType{ PublicDataLeafValue(slot, value), nextIndex, nextValue };
}

class PersistedContentAddressedIndexedTreeTest : public testing::Test {
protected:
void SetUp() override
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// === AUDIT STATUS ===
// internal: { status: Planned, auditors: [Raju], commit: }
// internal: { status: Planned, auditors: [], commit: }
// external_1: { status: not started, auditors: [], commit: }
// external_2: { status: not started, auditors: [], commit: }
// =====================
Expand Down Expand Up @@ -67,9 +67,9 @@ struct NullifierLeafValue {

static NullifierLeafValue padding(index_t i) { return { i }; }

static std::string name() { return "NullifierLeafValue"; };

size_t hash() const noexcept { return std::hash<fr>{}(nullifier); }

static std::string name() { return "NullifierLeafValue"; };
};

struct PublicDataLeafValue {
Expand Down Expand Up @@ -129,9 +129,9 @@ struct PublicDataLeafValue {

static PublicDataLeafValue padding(index_t i) { return { i, fr::zero() }; }

static std::string name() { return "PublicDataLeafValue"; };

size_t hash() const noexcept { return utils::hash_as_tuple(value, slot); }

static std::string name() { return "PublicDataLeafValue"; };
};

template <typename LeafType> struct IndexedLeaf {
Expand Down
Loading
Loading