Skip to content

Remove set_orchard and narrow serialize_from visibility#3

Merged
greg0x merged 1271 commits into
valargroup/pczt-governance-extensions-0.11from
subtree/librustzcash/2026-03-12
Mar 12, 2026
Merged

Remove set_orchard and narrow serialize_from visibility#3
greg0x merged 1271 commits into
valargroup/pczt-governance-extensions-0.11from
subtree/librustzcash/2026-03-12

Conversation

@greg0x
Copy link
Copy Markdown

@greg0x greg0x commented Mar 12, 2026

Clean up PCZT APIs that are no longer needed now that governance PCZT construction uses Creator::build_from_parts (see valargroup/zcash_voting#1).

  • Remove Pczt::set_orchard() — was only used by librustvoting to manually inject an orchard bundle after creating an empty PCZT shell. No longer needed since build_from_parts accepts the bundle directly.
  • Narrow orchard::Bundle::serialize_from from pub to pub(crate) — only used internally by the creator role, no external callers.

greg0x and others added 30 commits February 27, 2026 19:46
The ceremony already handles non-acking validators gracefully: they
get stripped from the round, and the ceremony completes with the 1/3
ack threshold. x/slashing handles the dominant failure mode (offline
validators missing blocks). The edge case of a validator online but
with broken ceremony logic is operationally noticeable and doesn't
block ceremonies.

Removing ceremony miss tracking eliminates the vote module's dependency
on x/slashing keeper, deletes ~500 lines of code (counter logic, jail
logic, SlashingKeeper interface, mocks, tests), and shrinks the audit
surface. x/slashing remains wired in the app for standard block-miss
jailing.
With up to 100 validator vote servers that can go down at any time,
the iOS app had no resilience at the HTTP layer. A single dead server
would block share delegation for 120s per attempt (the ZKP-length
timeout), turning a 30-minute CDN stale window into multi-minute
voting hangs.

Adds a ServerHealthTracker actor with per-server circuit breakers
(closed → open after 3 failures → halfOpen after 30s cooldown).
Share POSTs now use a 5s fast timeout with immediate failover to
another healthy server. Parallel health probes run at startup and
every 60s in the background. If all servers are marked unhealthy,
the tracker falls back to the full list so voting is never blocked.

Net effect: a dead server costs ~5-10s instead of ~6 minutes.
The proto definition was removed in a prior commit but buf generate
was not re-run, leaving stale MsgUnjailValidator types in tx.pb.go.
The Cosmos interface registry panicked at startup because the message
had no cosmos.msg.v1.signer annotation.
Moves DOMAIN_VC, vote_commitment_hash (out-of-circuit), and a new
vote_commitment_poseidon gadget into circuit/vote_commitment.rs,
mirroring the van_integrity pattern. ZKP #2 (cond 12) and ZKP #3
(cond 2) now both call the shared gadget instead of duplicating
the 5-input Poseidon block inline. Resolves the TODO in vote_proof/circuit.rs.

Made-with: Cursor
…ad-slashing-module-for-jailing

Unify jailing under standard x/slashing module
…-tolerance

iOS: client-side circuit breaker for vote server health
extract vote commitment hash into shared circuit gadget
Reshapes the share commitment mux gate layout from 4×10 to 4×9 by
shifting sel[9] and comm[0..1] into row 1 of the 9-column grid.
Poseidon and Merkle gadgets are unchanged — they only use columns 0–8.

Made-with: Cursor
reduce share reveal circuit from 10 to 9 advice columns
vote_commitment already commits to voting_round_id as one of its
Poseidon inputs, so including it again in the nullifier hash was
redundant. Removing it simplifies the nullifier to 5 inputs while
preserving identical security: cross-round replay is prevented
transitively through vote_commitment.

Also adds Config::assign_constant helper to de-duplicate the
assign_advice_from_constant boilerplate (used by cond2 domain_vc
and cond5 domain_tag), and collapses the Poseidon init+hash into
a single chained expression.

Made-with: Cursor
c1_x = (g^r).x already carries the full ElGamal randomness needed to
make the nullifier unlinkable to its vote commitment without knowledge of
the ciphertext. c2_x is redundant for that purpose.

Dropping it reduces the Poseidon hash from ConstantLength<5> (3
permutations at rate-2) to ConstantLength<4> (2 permutations), saving
one permutation in the circuit.

Updates share_nullifier_hash signature, Condition 5 in-circuit layout,
module/function doc comments, builder call site, and tests.

Made-with: Cursor
K=11 (2,048 rows) is sufficient for the share reveal circuit which uses
~1,592 rows. This reduces IPA params size, proof size, and proving time
(~5-15s vs ~30-60s in release mode).

Regenerated all fixtures, SDK circuits, and mobile FFI xcframework.

Made-with: Cursor
Lower share reveal circuit K from 14 to 11
The identical Merkle conditional swap gate + Poseidon path synthesis
loop was duplicated across three circuits (vote_proof condition 1,
share_reveal condition 1, delegation/IMT condition 13). Extract into
`circuit::poseidon_merkle` with a reusable `MerkleSwapGate` and a
generic `synthesize_poseidon_merkle_path<DEPTH>()` function.

Also removes the duplicated local `assign_free_advice` helpers from
vote_proof and share_reveal in favour of the existing
`orchard::circuit::gadget::assign_free_advice`.

Net: ~340 lines of duplicated circuit logic replaced by a single
~90-line shared gadget.

Made-with: Cursor
7 tests using a minimal depth-4 test circuit with MockProver:
- position_zero_valid, position_nonzero_valid, all_right_child_valid
- wrong_root_fails, wrong_leaf_fails, wrong_sibling_fails,
  wrong_position_fails

Made-with: Cursor
extract shared Poseidon Merkle tree gadget
- Skip remaining bundles now shows a confirmation alert with locked-in/giving-up
  ZEC amounts instead of inline warning text
- After signing a bundle, return to idle state instead of auto-showing QR for
  the next bundle — user taps "Confirm with Keystone" to start each bundle
- Fix progress bar after skip: use signed bundle count as divisor and delete
  skipped bundles from DB so proof_generated works correctly on resume
- Recalculate votingWeight after skip to reflect only signed bundles' weight
  on the governance page
- Filter wallet notes by account (seed fingerprint + account index) so Keystone
  and hotkey accounts show only their own notes and eligible balance
- Navigate Keystone users directly to delegation signing screen instead of
  briefly flashing the proposal list
Keystone multi-bundle UX improvements and account-scoped note filtering
Drop raw tier data after YPIR construction — the YPIR server copies
everything into its own db_buf_aligned during YServer::new(), so keeping
the source bytes alive wastes ~6 GB. OwnedTierState now takes &[u8]
instead of Vec<u8> and no longer retains the data.

Also clear hint_0 Vec<u64> from OfflinePrecomputedValues after extracting
the hint bytes (hint_0 is unused during online query answering), and use
Bytes instead of Vec<u8> for HTTP-served data to avoid cloning ~112 MB
per hint request.

Row debug endpoints (/tier1/row/:idx, /tier2/row/:idx) now read from
disk instead of indexing into in-memory data.

Same fixes applied to the standalone pir-server binary, which previously
used Box::leak for data that was never needed again.
…ization

Reduce PIR server memory by ~5 GB
Replace the fixed-interval ticker in the helper processor with
exponentially-distributed random sleep intervals so that share
submissions form a Poisson process, preventing an observer from
correlating submission patterns with share readiness.

- Processor.Run() now samples inter-wake-up times from Exp(1/meanInterval)
  using crypto/rand instead of time.NewTicker
- Default MaxConcurrentProofs changed from 2 to 1 (sequential processing)
- Concurrency test updated to assert sequential execution

Made-with: Cursor
Poisson-distributed share processing for timing privacy
- Add Layer 3 intra-batch jitter (exponential, half the inter-cycle mean)
  so multiple shares becoming ready in the same cycle are not submitted
  as a burst. Deadline bypass skips jitter when <60s remain.
- Extract shared exponentialDelay() sampler from processor for reuse.
- Remove delay_seconds from share-scheduled log to prevent log-based
  timing correlation.
- Fix loadShare to include vote_end_time so the deadline bypass in
  processBatch has the data it needs.
- Change ProcessInterval default from 5s to 30s to match documentation.
- Add QueuedShare.VoteEndTime field.
- Add sdk/internal/README.md documenting the three-layer delay model.

Made-with: Cursor
czarcas7ic and others added 5 commits March 9, 2026 23:06
Remove dead MeanDelay config and simplify share submission delay
The fork at valargroup/librustzcash (branch valargroup/pczt-governance-extensions-0.11)
is the source of truth with governance wallet methods. Replacing the vendored copy
with a git subtree.
set_orchard is no longer needed now that governance PCZT
construction goes through Creator::build_from_parts.
serialize_from is only used internally by the creator role.
greg0x pushed a commit that referenced this pull request Mar 12, 2026
greg0x pushed a commit that referenced this pull request Mar 12, 2026
greg0x added a commit that referenced this pull request Mar 12, 2026
The raw block height (#3,235,467) isn't meaningful to most users. Show
the snapshot date instead, matching the "Ends" column format for visual
consistency. Block height is still accessible via tap → popover.

Bumps modules Package.swift platform to iOS 16.6 to match the
secant-mainnet deployment target, which unlocks presentationCompactAdaptation.

Also removes the orange active-proposal highlight from the list view —
it was visually confusing since the first card always appeared selected.
greg0x added a commit that referenced this pull request Mar 12, 2026
Background tasks that complete the share relay lifecycle:

TreeSync wraps TreeClient with periodic sync from the chain (or mock
tree) via spawn_blocking, since HttpTreeSyncApi uses blocking reqwest.
Positions are marked as shares arrive so witnesses survive pruning.

Share nullifier uses the same Poseidon (P128Pow5T3 over Pallas Fp) as
the vote commitment tree, with a two-level hash chain to pack 4 logical
inputs (domain tag, vote_commitment, share_index, enc_share_hash) into
arity-2 calls.

The processor loop picks shares past their scheduled delay, syncs the
tree, generates VC Merkle witness, derives nullifier, and POSTs
MsgRevealShare to the chain. ZKP #3 proof is mocked (placeholder bytes)
until the Halo2 circuit exists — the pipeline is wired to plug it in.
greg0x pushed a commit that referenced this pull request Mar 12, 2026
Wire real ZKP #3 (share reveal) proofs into the E2E voting flow
and fix the Go/Rust El Gamal generator mismatch so auto-tally
BSGS decryption succeeds end-to-end.

- Add orchard/src/share_reveal/ circuit, builder, and prover (K=14)
- Expose share_reveal via sdk/circuits FFI for on-chain verification
- Update Go PallasGenerator() to use SpendAuthG (matching ZKP #2 circuit)
- Add Go ZKP #3 proof verification in sdk/crypto/zkp
- E2E test submits 2 of 4 real share reveals, waits for auto-tally,
  and asserts exact decrypted total_value = 7,500,000
- Wire helper-server share-reveal processing and nullifier tracking
- Regenerate FFI bindings (xcframework + Swift)
- Remove dead mock reveal_share_payload and unused encrypt_share
greg0x pushed a commit that referenced this pull request Mar 12, 2026
Real ZKP #3 share reveal + SpendAuthG El Gamal alignment
greg0x added a commit that referenced this pull request Mar 12, 2026
After rebasing onto main (which includes PR zcash#71's ZKP #3 circuit),
the e2e tests need to:

- Reconstruct enc_c1_x/enc_c2_x from EncryptedShareOutput compressed
  bytes instead of using the old VoteProofBundle fields
- Use the updated reveal_share_payload signature (7 args with
  nullifier + proof)
- Add real ZKP #3 proof generation in the librustvoting e2e test
greg0x added a commit that referenced this pull request Mar 12, 2026
Summarizes the current state of the 22-commit branch: what works
(Keystone path mostly wired), what doesn't (non-Keystone path skips
delegation entirely), remaining work prioritized, key files, and
architecture notes (ZKP #3 runs on helper server, not mobile).
greg0x added a commit that referenced this pull request Mar 12, 2026
Rewrote voting_flow_librustvoting.rs to exercise the complete path:
delegation (ZKP #1) → cast-vote (ZKP #2) → helper server (ZKP #3)
→ tally accumulation → auto-tally finalization → result verification.

Previously the test generated ZKP #3 inline and submitted reveal-share
directly to the chain. Now it sends share payloads to the helper
server, which handles tree sync, witness generation, ZKP #3 proof,
and chain submission — matching the production Zashi flow.

Key fixes: use the chain's EA public key (from ~/.zallyd/ea.pk)
instead of generating a random keypair, which was preventing
auto-tally decryption. Reduced the default vote window from 480s
to 180s since the helper server processes shares much faster than
inline ZKP #3 generation.
greg0x added a commit that referenced this pull request Mar 12, 2026
The voting_flow_librustvoting test now sends shares through the
helper server instead of generating ZKP #3 inline. Without this,
the test fails at step 10 with a connection refused error.

The helper server is built, started in the background with fast
delays (1-3s), and health-checked before the test runs. Its log
is uploaded as an artifact on failure for debugging.
greg0x added a commit that referenced this pull request Mar 12, 2026
Shares were being sent directly to the chain with a mock proof. Now they
go to the helper server which generates real ZKP #3 proofs and submits
reveal-share TXs on the voter's behalf.

- Add allEncShares field to SharePayload (needed for ZKP #3 witness)
- Rewrite delegateShares to POST to helper server /api/v1/shares
- Add postHelperJSON helper with configurable helperServerURL
- Drop anchorHeight param from delegateShares (not needed by helper)
- SDK: auto-fill ea_pk from filesystem in create-voting-session handler
- Update handoff docs to reflect completed status
greg0x pushed a commit that referenced this pull request Mar 12, 2026
Rewrite the standalone Rust helper server as a Go package
(sdk/internal/helper/) running inside the zallyd binary. This
eliminates the separate deployment artifact and the HTTP tree sync
layer — the helper now reads commitment tree leaves directly from
the vote keeper's KV store.

Components:
- Composite Rust FFI function (zally_generate_share_reveal) for
  ZKP #3 proof generation, called from Go via CGo
- SQLite-backed share queue with crypto/rand delays for temporal
  unlinkability, crash recovery, and exponential backoff retries
- HTTP API (POST /api/v1/shares, GET /api/v1/status) with optional
  token auth, body size limits, and idempotent enqueue semantics
- Background processor with bounded concurrency via errgroup
- Integration via Cosmos SDK PostSetup hook with build-tag-gated
  halo2 support
- Deploy health checks for both chain API and helper endpoints
- Delete the standalone helper-server/ Rust crate
greg0x pushed a commit that referenced this pull request Mar 12, 2026
- Add sentinel injection (k*2^250 for k=0..16) to pir-export to satisfy
  circuit gap-width constraint (#3)
- Change Tier 2 empty-leaf padding from Fp::zero() to -Fp::one() so
  trailing entries sort after real leaves, fixing binary search (#2)
- Make TierServer::answer_query() return Result with input validation
  (length checks, alignment) instead of panicking on malformed
  requests; handlers return HTTP 400 on error (#1)
- Replace unwrap/assert with fallible returns in pir-client and
  Tier0Data::from_bytes (#4)
greg0x pushed a commit that referenced this pull request Mar 12, 2026
Replace deterministic shares_hash = Poseidon(c1_0_x, c2_0_x, ..., c1_4_x, c2_4_x)
with per-share blinded commitments:
  share_comm_i = Poseidon(blind_i, c1_i_x, c2_i_x)
  shares_hash  = Poseidon(share_comm_0, ..., share_comm_4)

This prevents observers from recomputing shares_hash from on-chain encrypted
shares and linking them back to a specific vote commitment, fixing a voter
privacy leak.

Changes span the full stack:
- Circuits (orchard): ZKP #2 and ZKP #3 condition 3/10 use ConstantLength<3>
  per-share hashes + ConstantLength<5> final hash instead of ConstantLength<10>
- Rust lib (librustvoting): share_blinds plumbed through types and builders
- C FFI (sdk/circuits): share_blinds_ptr/len params on share reveal function
- Go helper (sdk): wire types, API validation, store schema, processor, CGo
- UniFFI bridge + iOS: shareBlinds/shareBlindFactors through FFI and Swift
greg0x pushed a commit that referenced this pull request Mar 12, 2026
…#3

Move the two-level Poseidon shares-hash computation (per-share blinded
commitments + outer hash) from vote_proof/circuit.rs into a new
shared_primitives::shares_hash module, and replace the inlined version
in share_reveal/circuit.rs with calls to the same gadget.

Co-authored-by: Cursor <cursoragent@cursor.com>
greg0x pushed a commit that referenced this pull request Mar 12, 2026
…gadget

refactor(orchard): extract shares-hash gadget shared by ZKP #2 and ZKP #3
greg0x pushed a commit that referenced this pull request Mar 12, 2026
Bind the vote commitment to its voting round by including
voting_round_id in the Poseidon hash:

  vote_commitment = Poseidon(DOMAIN_VC, voting_round_id,
                             shares_hash, proposal_id, vote_decision)

This prevents cross-round replay where an attacker reuses a ZKP #3
proof from round A against round B, injecting shares encrypted under
the wrong election key. The change updates both ZKP #2 (condition 12)
and ZKP #3 (condition 2) circuits, out-of-circuit helpers, builders,
FFI, tests, and documentation.
@greg0x greg0x changed the title sync: monorepo changes 2026-03-12 Remove set_orchard and narrow serialize_from visibility Mar 12, 2026
@greg0x greg0x merged commit 505b85f into valargroup/pczt-governance-extensions-0.11 Mar 12, 2026
greg0x added a commit that referenced this pull request Mar 12, 2026
External governance protocols (shielded voting) need to:

- Replace the Orchard bundle in a PCZT after constructing a custom
  governance action (Pczt::set_orchard)
- Read back the spend_auth_sig after a hardware wallet signs the PCZT,
  so the signature can be threaded into a ZK delegation proof
- Serialize an orchard::pczt::Bundle into the PCZT wire format from
  outside the crate (Bundle::serialize_from)
- Construct an ephemeral SqliteShardStore from a raw connection to build
  Merkle witnesses without going through WalletDb
  (SqliteShardStore::from_connection)

Remove set_orchard and narrow serialize_from visibility (#3)

Clean up PCZT APIs that are no longer needed now that governance PCZT construction uses `Creator::build_from_parts` (see valargroup/zcash_voting#1).

- Remove `Pczt::set_orchard()` — was only used by librustvoting to manually inject an orchard bundle after creating an empty PCZT shell. No longer needed since `build_from_parts` accepts the bundle directly.
- Narrow `orchard::Bundle::serialize_from` from `pub` to `pub(crate)` — only used internally by the creator role, no external callers.
greg0x added a commit that referenced this pull request Mar 13, 2026
External governance protocols (shielded voting) need to:

- Replace the Orchard bundle in a PCZT after constructing a custom
  governance action (Pczt::set_orchard)
- Read back the spend_auth_sig after a hardware wallet signs the PCZT,
  so the signature can be threaded into a ZK delegation proof
- Serialize an orchard::pczt::Bundle into the PCZT wire format from
  outside the crate (Bundle::serialize_from)
- Construct an ephemeral SqliteShardStore from a raw connection to build
  Merkle witnesses without going through WalletDb
  (SqliteShardStore::from_connection)

Remove set_orchard and narrow serialize_from visibility (#3)

Clean up PCZT APIs that are no longer needed now that governance PCZT construction uses `Creator::build_from_parts` (see valargroup/zcash_voting#1).

- Remove `Pczt::set_orchard()` — was only used by librustvoting to manually inject an orchard bundle after creating an empty PCZT shell. No longer needed since `build_from_parts` accepts the bundle directly.
- Narrow `orchard::Bundle::serialize_from` from `pub` to `pub(crate)` — only used internally by the creator role, no external callers.
greg0x added a commit that referenced this pull request Mar 13, 2026
External governance protocols (shielded voting) need to:

- Replace the Orchard bundle in a PCZT after constructing a custom
  governance action (Pczt::set_orchard)
- Read back the spend_auth_sig after a hardware wallet signs the PCZT,
  so the signature can be threaded into a ZK delegation proof
- Serialize an orchard::pczt::Bundle into the PCZT wire format from
  outside the crate (Bundle::serialize_from)
- Construct an ephemeral SqliteShardStore from a raw connection to build
  Merkle witnesses without going through WalletDb
  (SqliteShardStore::from_connection)

Remove set_orchard and narrow serialize_from visibility (#3)

Clean up PCZT APIs that are no longer needed now that governance PCZT construction uses `Creator::build_from_parts` (see valargroup/zcash_voting#1).

- Remove `Pczt::set_orchard()` — was only used by librustvoting to manually inject an orchard bundle after creating an empty PCZT shell. No longer needed since `build_from_parts` accepts the bundle directly.
- Narrow `orchard::Bundle::serialize_from` from `pub` to `pub(crate)` — only used internally by the creator role, no external callers.
@greg0x greg0x deleted the subtree/librustzcash/2026-03-12 branch March 13, 2026 20:44
p0mvn pushed a commit that referenced this pull request Apr 3, 2026
…14868de..23f0768ea

23f0768ea Release lightwallet-protocol v0.4.0
41156c767 Merge pull request #11 from zcash/feature/get_mempool_tx_pools
7c130e883 Add `lightwalletProtocolVersion` field to `LightdInfo` struct.
edbb726d7 Apply suggestion from code review
38fddd73b Apply suggestions from code review
0250f2720 Add pool type filtering to `GetMempoolTx` argument.
54ccaadd5 Change semantics of pool-based pruning of compact transactions from "may prune" to "must prune".
b0667ec99 Merge pull request #9 from zcash/2025-11-doc-TransparentAddressBlockFilter
f3fea7bd4 doc: TransparentAddressBlockFilter doesn't include mempool
a67dd323a Merge pull request #8 from zcash/2025-11-lightdinfo-upgrade-info
11da4b7e3 add next upgrade info to LightdInfo structure (GetLightdInfo)
42cd8f720 Transparent data docs update (#7)
c0cf957ac Merge pull request #5 from zcash/2025-11-comments
912fc3609 Minor clarification in GetBlockRange documentation.
6b03f2cce Documentation (comments) only
d978256a2 Merge pull request #1 from zcash/compact_tx_transparent
7eeb82e7c Merge pull request #4 from zcash/add_changelog
a95359dc9 Apply suggestions from code review
592b637a8 Add transparent data to the `CompactBlock` format.
9d1fb2c41 Add a CHANGELOG.md that documents the evolution of the light client protocol.
180717dfa Merge pull request #3 from zcash/merge_librustzcash_history
450bd4181 Merge the history of the .proto files from `librustzcash` for complete history preservation.
a4859d11d Move protobuf files into place for use in `zcash/lightwallet-protocol`
2e66cdd9e Update zcash_client_backend/proto/service.proto
eda012519 fix comment
f838d10ad Add gRPC LightdInfo Donation Address
db12c0415 Merge pull request zcash#1473 from nuttycom/wallet/enrichment_queue
698feba96 Apply suggestions from code review
20ce57ab3 zcash_client_backend: Add `block_height` argument to `decrypt_and_store_transaction`
a6dea1da8 Merge pull request zcash#1482 from zancas/doc_tweak
4d2d45fc9 fix incorrect doc-comment
e826f4740 update CompactBlock doc-comment, to cover non-Sapling shielded notes, and addresses
e9a6c00bf Various documentation improvements
988bc7214 Merge pull request zcash#872 from nuttycom/feature/pre_dag_sync-suggest_scan_ranges
58d07d469 Implement `suggest_scan_ranges` and `update_chain_tip`
a9222b338 Address comments from code review.
e20310857 Rename proto::compact::{BlockMetadata => ChainMetadata}
ac63418c5 Reorganize Sapling and Orchard note commitment tree sizes in CompactBlock.
0fdca14f1 zcash_client_backend: Add note commitment tree sizes to `CompactBlock` serialization.
2a0c2b8b7 zcash_client_backend: Add gRPC bindings behind feature flag
1342f0480 zcash_client_backend: Address compact_formats.proto comments
68aa4e01b zcash_client_backend: Bring in latest `compact_formats.proto`
e712eb1bc Add prevHash field to CompactBlock
440384c3e Build protobufs for compact formats

git-subtree-dir: zcash_client_backend/lightwallet-protocol
git-subtree-split: 23f0768ea4471b63285f3c0e9b6fbb361674aa2b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants