Skip to content

Expose commitment-tree primitives instead of monolithic witness generation #23

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
p0mvn merged 1 commit into from
Apr 10, 2026
Merged

Expose commitment-tree primitives instead of monolithic witness generation #23

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions zcash_client_sqlite/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,12 @@ workspace.
### Added
- `WalletDb::get_orchard_notes_at_snapshot` returns Orchard notes received
and unspent as of a given height, for governance voting snapshots.
- `WalletDb::generate_orchard_witnesses_at_frontier` generates Merkle
witnesses at a historical frontier using an ephemeral in-memory tree.
- `WalletDb::conn()` provides access to the underlying connection handle.
- `wallet::commitment_tree::create_orchard_tree_tables` creates the
Orchard commitment-tree schema in a given connection, enabling
construction of an ephemeral in-memory `SqliteShardStore`.
- `wallet::commitment_tree::SqliteShardStore::from_connection` is now
publicly accessible.

## [0.19.5] - 2026-03-10

Expand Down
40 changes: 7 additions & 33 deletions zcash_client_sqlite/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -401,6 +401,11 @@ impl Borrow<rusqlite::Connection> for SqlTransaction<'_> {
}

impl<C, P, CL, R> WalletDb<C, P, CL, R> {
/// Returns a reference to the underlying connection handle.
pub fn conn(&self) -> &C {
&self.conn
}
Comment on lines +404 to +407
Copy link
Copy Markdown
Author

@p0mvn p0mvn Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: new public method #2


/// Returns the network parameters that this walletdb instance is bound to.
pub fn params(&self) -> &P {
&self.params
Expand Down Expand Up @@ -2313,8 +2318,8 @@ impl<P: consensus::Parameters, CL, R> WalletCommitmentTrees
// --- Governance-specific methods ---
//
// These methods support Zcash shielded voting and are not part of the
// general-purpose wallet traits. They provide backward-looking queries
// (historical snapshot) and witness generation at an external frontier.
// general-purpose wallet traits. They provide a backward-looking snapshot
// query for historical note selection.

#[cfg(feature = "orchard")]
impl<C: Borrow<rusqlite::Connection>, P: consensus::Parameters, CL, R> WalletDb<C, P, CL, R> {
Expand All @@ -2337,37 +2342,6 @@ impl<C: Borrow<rusqlite::Connection>, P: consensus::Parameters, CL, R> WalletDb<
snapshot_height,
)
}

/// Generate Orchard Merkle witnesses at a historical frontier.
///
/// Copies the wallet's Orchard shard data into an ephemeral in-memory
/// database, inserts the provided frontier as a checkpoint, and generates
/// a witness for each of the given note positions.
///
/// The wallet DB is strictly read-only — shard data is copied, not modified.
pub fn generate_orchard_witnesses_at_frontier(
&self,
note_positions: &[Position],
frontier: incrementalmerkletree::frontier::NonEmptyFrontier<
orchard::tree::MerkleHashOrchard,
>,
checkpoint_height: BlockHeight,
) -> Result<
Vec<
incrementalmerkletree::MerklePath<
orchard::tree::MerkleHashOrchard,
{ orchard::NOTE_COMMITMENT_TREE_DEPTH as u8 },
>,
>,
SqliteClientError,
> {
wallet::commitment_tree::generate_orchard_witnesses_at_frontier(
self.conn.borrow(),
note_positions,
frontier,
checkpoint_height,
)
}
}

/// A handle for the SQLite block source.
Expand Down
289 changes: 16 additions & 273 deletions zcash_client_sqlite/src/wallet/commitment_tree.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,10 +103,7 @@ pub struct SqliteShardStore<C, H, const SHARD_HEIGHT: u8> {
impl<C, H, const SHARD_HEIGHT: u8> SqliteShardStore<C, H, SHARD_HEIGHT> {
const SHARD_ROOT_LEVEL: Level = Level::new(SHARD_HEIGHT);

pub(crate) fn from_connection(
conn: C,
table_prefix: &'static str,
) -> Result<Self, rusqlite::Error> {
pub fn from_connection(conn: C, table_prefix: &'static str) -> Result<Self, rusqlite::Error> {
Ok(SqliteShardStore {
conn,
table_prefix,
Expand Down Expand Up @@ -1140,179 +1137,23 @@ pub(crate) fn check_witnesses(
Ok(scan_ranges)
}

/// Generate Orchard Merkle witnesses at a historical frontier.
/// Creates the Orchard commitment-tree tables in the given connection.
///
/// Copies the wallet's Orchard shard data into an ephemeral in-memory database,
/// inserts the provided frontier (from lightwalletd) as a checkpoint, and
/// generates a witness for each of the given note positions.
///
/// The wallet DB is strictly read-only — shard data is copied, not modified.
///
/// This is used by governance voting: the wallet tree may have advanced past
/// the snapshot height, but we need witnesses anchored at the snapshot frontier.
/// This enables constructing an ephemeral in-memory [`SqliteShardStore`] for
/// operations like witness generation at a historical frontier, without
/// requiring the full wallet schema.
#[cfg(feature = "orchard")]
pub fn generate_orchard_witnesses_at_frontier(
conn: &rusqlite::Connection,
note_positions: &[Position],
frontier: incrementalmerkletree::frontier::NonEmptyFrontier<orchard::tree::MerkleHashOrchard>,
checkpoint_height: BlockHeight,
) -> Result<
Vec<
incrementalmerkletree::MerklePath<
orchard::tree::MerkleHashOrchard,
{ orchard::NOTE_COMMITMENT_TREE_DEPTH as u8 },
>,
>,
SqliteClientError,
> {
use incrementalmerkletree::Marking;
use shardtree::ShardTree;
use shardtree::store::Checkpoint;
use zcash_client_backend::data_api::ORCHARD_SHARD_HEIGHT;

let frontier_position = frontier.position();

// Create in-memory DB with the tree schema
let mem_conn = rusqlite::Connection::open_in_memory().map_err(SqliteClientError::DbError)?;

mem_conn
.execute_batch(
"CREATE TABLE orchard_tree_shards (
shard_index INTEGER PRIMARY KEY,
subtree_end_height INTEGER,
root_hash BLOB,
shard_data BLOB,
contains_marked INTEGER,
CONSTRAINT root_unique UNIQUE (root_hash)
);
CREATE TABLE orchard_tree_cap (
cap_id INTEGER PRIMARY KEY,
cap_data BLOB NOT NULL
);
CREATE TABLE orchard_tree_checkpoints (
checkpoint_id INTEGER PRIMARY KEY,
position INTEGER
);
CREATE TABLE orchard_tree_checkpoint_marks_removed (
checkpoint_id INTEGER NOT NULL,
mark_removed_position INTEGER NOT NULL,
FOREIGN KEY (checkpoint_id) REFERENCES orchard_tree_checkpoints(checkpoint_id)
ON DELETE CASCADE,
CONSTRAINT spend_position_unique UNIQUE (checkpoint_id, mark_removed_position)
);",
)
.map_err(SqliteClientError::DbError)?;

// Copy shard data from wallet into in-memory DB
{
use rusqlite::types::Value;

let mut stmt = conn
.prepare(
"SELECT shard_index, subtree_end_height, root_hash, shard_data, contains_marked
FROM orchard_tree_shards",
)
.map_err(SqliteClientError::DbError)?;
let mut rows = stmt.query([]).map_err(SqliteClientError::DbError)?;
while let Some(row) = rows.next().map_err(SqliteClientError::DbError)? {
mem_conn
.execute(
"INSERT INTO orchard_tree_shards
(shard_index, subtree_end_height, root_hash, shard_data, contains_marked)
VALUES (?1, ?2, ?3, ?4, ?5)",
rusqlite::params![
row.get::<_, Value>(0)?,
row.get::<_, Value>(1)?,
row.get::<_, Value>(2)?,
row.get::<_, Value>(3)?,
row.get::<_, Value>(4)?,
],
)
.map_err(SqliteClientError::DbError)?;
}
}

// Copy cap data
{
use rusqlite::types::Value;

let mut stmt = conn
.prepare("SELECT cap_id, cap_data FROM orchard_tree_cap")
.map_err(SqliteClientError::DbError)?;
let mut rows = stmt.query([]).map_err(SqliteClientError::DbError)?;
while let Some(row) = rows.next().map_err(SqliteClientError::DbError)? {
mem_conn
.execute(
"INSERT INTO orchard_tree_cap (cap_id, cap_data) VALUES (?1, ?2)",
rusqlite::params![row.get::<_, Value>(0)?, row.get::<_, Value>(1)?,],
)
.map_err(SqliteClientError::DbError)?;
}
}

// Build ShardTree from in-memory store
let tx = mem_conn
.unchecked_transaction()
.map_err(SqliteClientError::DbError)?;

let store = SqliteShardStore::<
_,
orchard::tree::MerkleHashOrchard,
ORCHARD_SHARD_HEIGHT,
>::from_connection(&tx, "orchard")
.map_err(SqliteClientError::DbError)?;

let mut tree =
ShardTree::<_, { orchard::NOTE_COMMITMENT_TREE_DEPTH as u8 }, ORCHARD_SHARD_HEIGHT>::new(
store, 100,
);

// Insert frontier + checkpoint
tree.insert_frontier_nodes(
frontier,
Retention::Checkpoint {
id: checkpoint_height,
marking: Marking::None,
},
)
.map_err(|e| {
SqliteClientError::CorruptedData(format!("failed to insert frontier nodes: {}", e))
})?;

tree.store_mut()
.add_checkpoint(
checkpoint_height,
Checkpoint::at_position(frontier_position),
)
.map_err(|e| {
SqliteClientError::CorruptedData(format!("failed to add checkpoint: {}", e))
})?;

// Generate witness per note position
let mut witnesses = Vec::with_capacity(note_positions.len());
for &pos in note_positions {
let merkle_path = tree
.witness_at_checkpoint_id(pos, &checkpoint_height)
.map_err(|e| {
SqliteClientError::CorruptedData(format!(
"failed to generate witness for position {}: {} \
(wallet may need to sync through snapshot height)",
u64::from(pos),
e
))
})?
.ok_or_else(|| {
SqliteClientError::CorruptedData(format!(
"no witness available for position {} \
(wallet missing shard data — sync through snapshot height)",
u64::from(pos)
))
})?;

witnesses.push(merkle_path);
}

Ok(witnesses)
pub fn create_orchard_tree_tables(conn: &rusqlite::Connection) -> Result<(), rusqlite::Error> {
Comment on lines 1145 to +1146
Copy link
Copy Markdown
Author

@p0mvn p0mvn Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: new public function #1

use super::db::{
TABLE_ORCHARD_TREE_CAP, TABLE_ORCHARD_TREE_CHECKPOINT_MARKS_REMOVED,
TABLE_ORCHARD_TREE_CHECKPOINTS, TABLE_ORCHARD_TREE_SHARDS,
};
conn.execute_batch(&format!(
"{TABLE_ORCHARD_TREE_SHARDS};\
{TABLE_ORCHARD_TREE_CAP};\
{TABLE_ORCHARD_TREE_CHECKPOINTS};\
{TABLE_ORCHARD_TREE_CHECKPOINT_MARKS_REMOVED};",
))
}

#[cfg(test)]
Expand Down Expand Up @@ -1401,11 +1242,6 @@ mod tests {
super::check_rewind_remove_mark(new_tree::<OrchardPoolTester>);
}

#[test]
fn witnesses_at_frontier() {
super::witnesses_at_frontier()
}

#[test]
fn put_shard_roots() {
super::put_shard_roots::<OrchardPoolTester>()
Expand Down Expand Up @@ -1522,97 +1358,4 @@ mod tests {
]
);
}

/// Test that `generate_orchard_witnesses_at_frontier` produces valid
/// witnesses when given a frontier extracted from an earlier tree state.
#[cfg(feature = "orchard")]
fn witnesses_at_frontier() {
use ::orchard::tree::MerkleHashOrchard;
use incrementalmerkletree::frontier::Frontier;
use rand::SeedableRng;
use rand_chacha::ChaChaRng;
use zcash_client_backend::data_api::ORCHARD_SHARD_HEIGHT;

let data_file = NamedTempFile::new().unwrap();
let mut db_data = WalletDb::for_path(
data_file.path(),
Network::TestNetwork,
test_clock(),
test_rng(),
)
.unwrap();
data_file.keep().unwrap();

WalletMigrator::new().init_or_migrate(&mut db_data).unwrap();

let mut rng = ChaChaRng::seed_from_u64(0);

// Build a tree with some leaves, marking one as our note.
// We track the frontier separately to capture the tree state at a
// snapshot height, while the wallet DB holds the persistent shard data.
let mut frontier_tree: Frontier<MerkleHashOrchard, 32> = Frontier::empty();
let snapshot_height = BlockHeight::from(100);
let note_position;
let note_leaf;

{
let tx = db_data.conn.transaction().unwrap();
let store =
SqliteShardStore::<_, MerkleHashOrchard, ORCHARD_SHARD_HEIGHT>::from_connection(
&tx, "orchard",
)
.unwrap();
let mut tree = ShardTree::<
_,
{ ::orchard::NOTE_COMMITMENT_TREE_DEPTH as u8 },
ORCHARD_SHARD_HEIGHT,
>::new(store, 100);

// Insert 5 leaves. Mark leaf 2 as our note.
let mut saved_leaf = None;
for i in 0u64..5 {
let leaf = MerkleHashOrchard::random(&mut rng);
let retention = if i == 2 {
saved_leaf = Some(leaf);
Retention::Marked
} else {
Retention::Ephemeral
};
tree.append(leaf, retention).unwrap();
frontier_tree.append(leaf);
}
note_position = Position::from(2);
note_leaf = saved_leaf.unwrap();

// Advance the tree past the snapshot height, simulating the
// wallet continuing to sync after the voting snapshot.
tree.checkpoint(snapshot_height).unwrap();
for _ in 0..5 {
let leaf = MerkleHashOrchard::random(&mut rng);
tree.append(leaf, Retention::Ephemeral).unwrap();
}
tree.checkpoint(BlockHeight::from(200)).unwrap();

tx.commit().unwrap();
}

// The frontier_tree captured the state after 5 leaves.
let expected_root = frontier_tree.root();
let frontier = frontier_tree.take().expect("frontier is non-empty");

// Generate witness using the function under test
let witnesses = super::generate_orchard_witnesses_at_frontier(
&db_data.conn,
&[note_position],
frontier,
snapshot_height,
)
.expect("witness generation should succeed");

assert_eq!(witnesses.len(), 1);

// The witness, combined with the note's leaf hash, must recompute
// to the tree root at the snapshot height.
assert_eq!(witnesses[0].root(note_leaf), expected_root);
}
}
Loading