Skip to content

[TieredStorage] Refactor TieredStorage::new_readonly() code path#195

Merged
yhchiang-sol merged 2 commits intoanza-xyz:masterfrom
yhchiang-sol:ts-open-refactor
Mar 20, 2024
Merged

[TieredStorage] Refactor TieredStorage::new_readonly() code path#195
yhchiang-sol merged 2 commits intoanza-xyz:masterfrom
yhchiang-sol:ts-open-refactor

Conversation

@yhchiang-sol
Copy link
Copy Markdown

@yhchiang-sol yhchiang-sol commented Mar 11, 2024 •
edited
Loading

Problem

The TieredStorage::new_readonly() function currently has the following
problems:

  • It opens the file without checking the magic number before checking and loading the footer.
  • It opens the file twice: first to load the footer, then open again by the reader.

Summary of Changes

This PR refactors TieredStorage::new_readonly() so that it first performs all
checks inside the constructor of TieredReadableFile. The TieredReadableFile
instance is then passed to the proper reader (currently HotStorageReader)
when all checks are passed.

Test Plan

  • Added a new test to check MagicNumberMismatch.
  • Existing tiered-storage tests

@yhchiang-sol yhchiang-sol mentioned this pull request Mar 11, 2024
Merged
@yhchiang-sol yhchiang-sol marked this pull request as draft March 11, 2024 23:30
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 12, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 96.66667% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 81.9%. Comparing base (184ba6c) to head (daf43bf).

Additional details and impacted files 
@@           Coverage Diff           @@
##           master     #195   +/-   ##
=======================================
  Coverage    81.9%    81.9%           
=======================================
  Files         837      837           
  Lines      226898   226932   +34     
=======================================
+ Hits       185871   185953   +82     
+ Misses      41027    40979   -48

@yhchiang-sol yhchiang-sol marked this pull request as ready for review March 12, 2024 02:57
brooksprumo
brooksprumo reviewed Mar 12, 2024
View reviewed changes
Comment thread accounts-db/src/tiered_storage/hot.rs Outdated
Comment thread accounts-db/src/tiered_storage/file.rs Outdated
Comment thread accounts-db/src/tiered_storage/file.rs Outdated
Comment thread accounts-db/src/tiered_storage/file.rs
@yhchiang-sol
Copy link
Copy Markdown
Author

yhchiang-sol commented Mar 12, 2024

The test failure doesn't seem related to the PR. Will rebase after the PR is fully reviewed.


git diff --check | 5m 11s
-- | --
  | Checking remote `origin` for updates to target branch `master`
  | ++git diff origin/master --check --oneline
  | .github/workflows/autolock_bot_PR.txt:26: trailing whitespace.
  | +
  | .github/workflows/autolock_bot_closed_issue.txt:26: trailing whitespace.
  | +
  | .github/workflows/solana-action.yml.txt:1: trailing whitespace.
  | +name : minimal
  | .github/workflows/solana-action.yml.txt:33: trailing whitespace.
  | +
  | .github/workflows/solana-action.yml.txt:41: trailing whitespace.
  | +
  | .github/workflows/solana-action.yml.txt:60: trailing whitespace.
  | +
  | .github/workflows/solana-action.yml.txt:64: trailing whitespace.
  | +      fail-fast: false
  | .github/workflows/solana-action.yml.txt:79: trailing whitespace.
  | +      - name: release artifact
  | .github/workflows/solana-action.yml.txt:87: trailing whitespace.
  | +#            choco install openssl
  | .github/workflows/solana-action.yml.txt:92: trailing whitespace.
  | +        uses: actions/checkout@v2
  | .github/workflows/solana-action.yml.txt:104: trailing whitespace.
  | +
  | .github/workflows/solana-action.yml.txt:113: trailing whitespace.
  | :# Received cancellation signal, interrupting
  | >^^^ +++

@yhchiang-sol
Copy link
Copy Markdown
Author

yhchiang-sol commented Mar 13, 2024

Rebase to address the unrelated CI failure.

brooksprumo
brooksprumo reviewed Mar 13, 2024
View reviewed changes
Comment thread accounts-db/src/tiered_storage/file.rs Outdated
Comment thread accounts-db/src/tiered_storage/file.rs Outdated
Comment thread accounts-db/src/tiered_storage/file.rs Outdated
@yhchiang-sol
Copy link
Copy Markdown
Author

yhchiang-sol commented Mar 20, 2024

Squash and rebase to include recent tiered-storage PRs.

@yhchiang-sol yhchiang-sol mentioned this pull request Mar 20, 2024
Closed
brooksprumo
brooksprumo reviewed Mar 20, 2024
View reviewed changes
Copy link
Copy Markdown

@brooksprumo brooksprumo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, just some nits

Comment thread accounts-db/src/tiered_storage/file.rs Outdated
Comment thread accounts-db/src/tiered_storage/file.rs Outdated
Comment thread accounts-db/src/tiered_storage/file.rs Outdated
Comment thread accounts-db/src/tiered_storage/footer.rs
Comment on lines 119 to 121
// This field is persisted in the storage but not in this struct.
// The number should match FOOTER_MAGIC_NUMBER.
// The number should match FILE_MAGIC_NUMBER.
// pub magic_number: u64,
Copy link
Copy Markdown

@brooksprumo brooksprumo Mar 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO the magic number is not part of the footer. And thus this part can be removed. I know this'll cause other necessary changes, so no need to change anything in this PR. (And it's pretty low priority, so no need to address this immediately.)

Copy link
Copy Markdown
Author

@yhchiang-sol yhchiang-sol Mar 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Magic number is not part of the footer as it is commented out. The comment is to point out that there is one additional field after the footer (and the magic_number: u64 is also commented out). Maybe we can find a better place for this comment.

Copy link
Copy Markdown

@brooksprumo brooksprumo Mar 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not part of the struct, but it is part of FOOTER_TAIL_SIZE, so we sometimes consider the magic number as part of the footer.

Copy link
Copy Markdown
Author

@yhchiang-sol yhchiang-sol Mar 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not part of the struct, but it is part of FOOTER_TAIL_SIZE, so we sometimes consider the magic number as part of the footer.

Aghh, got it. The concept of FOOTER_TAIL_SIZE is that the format of the last FOOTER_TAIL_SIZE bytes in any tiered-storage files will not change so that even if we have new versions of the footer that introduces new fields, we can still safely check the magic number and the format version. The tail also has a field describing the size of the footer.

brooksprumo
brooksprumo approved these changes Mar 20, 2024
View reviewed changes
Copy link
Copy Markdown

@brooksprumo brooksprumo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

Comment thread accounts-db/src/tiered_storage/footer.rs
Comment on lines 119 to 121
// This field is persisted in the storage but not in this struct.
// The number should match FOOTER_MAGIC_NUMBER.
// The number should match FILE_MAGIC_NUMBER.
// pub magic_number: u64,
Copy link
Copy Markdown

@brooksprumo brooksprumo Mar 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not part of the struct, but it is part of FOOTER_TAIL_SIZE, so we sometimes consider the magic number as part of the footer.

@yhchiang-sol yhchiang-sol merged commit 0e932c7 into anza-xyz:master Mar 20, 2024
@yhchiang-sol yhchiang-sol deleted the ts-open-refactor branch March 20, 2024 17:39
codebender828 pushed a commit to codebender828/agave that referenced this pull request Oct 3, 2024
@yhchiang-sol @willhickey
[TieredStorage] Refactor TieredStorage::new_readonly() code path (anz…
e97d359 
…a-xyz#195)

#### Problem
The TieredStorage::new_readonly() function currently has the following
problems:

* It opens the file without checking the magic number before checking and loading the footer.
* It opens the file twice: first to load the footer, then open again by the reader.

#### Summary of Changes
This PR refactors TieredStorage::new_readonly() so that it first performs all
checks inside the constructor of TieredReadableFile.  The TieredReadableFile
instance is then passed to the proper reader (currently HotStorageReader)
when all checks are passed.

#### Test Plan
* Added a new test to check MagicNumberMismatch.
* Existing tiered-storage tests
apfitzge pushed a commit to apfitzge/agave that referenced this pull request Aug 26, 2025
@buffalu
Jito Patch
4b70260 
only reroute if relayer connected (anza-xyz#123)
feat: add client tls config (anza-xyz#121)
remove extra val (anza-xyz#129)
fix clippy (anza-xyz#130)
copy all binaries to docker-output (anza-xyz#131)
Ledger tool halts at slot passed to create-snapshot (anza-xyz#118)
update program submodule (anza-xyz#133)
quick fix for tips and clearing old bundles (anza-xyz#135)
update submodule to new program (anza-xyz#136)
Improve stake-meta-generator usability (anza-xyz#134)
pinning submodule head (anza-xyz#140)
Use BundleAccountLocker when handling tip txs (anza-xyz#147)
Add metrics for relayer + block engine proxy (anza-xyz#149)
Build claim-mev in docker (anza-xyz#141)
Rework bundle receiving and add metrics (anza-xyz#152) (anza-xyz#154)
update submodule + dev files (anza-xyz#158)
Deterministically find tip amounts, add meta to stake info, and cleanup pubkey/strings in MEV tips (anza-xyz#159)
update jito-programs submodule (anza-xyz#160)
Separate MEV tip related workflow (anza-xyz#161)
Add block builder fee protos (anza-xyz#162)
fix jito programs (anza-xyz#163)
update submodule so autosnapshot exits out of ledger tool early (anza-xyz#164)
Pipe through block builder fee (anza-xyz#167)
pull in new snapshot code (anza-xyz#171)
block builder bug (anza-xyz#172)

Pull in new slack autosnapshot submodule (anza-xyz#174)

sort stake meta json and use int math (anza-xyz#176)

add accountsdb conn submod (anza-xyz#169)

Update tip distribution parameters (anza-xyz#177)

new submodules (anza-xyz#180)

Add buildkite link for jito CI (anza-xyz#183)

Fixed broken links to repositories (anza-xyz#184)

Changed from ssh to https transfer for clone

Seg/update submods (anza-xyz#187)

fix tests (anza-xyz#190)

rm geyser submod (anza-xyz#192)

rm dangling geyser references (anza-xyz#193)

fix syntax err (anza-xyz#195)

use deterministic req ids in batch calls (anza-xyz#199)

update jito-programs

revert cargo

update Cargo lock

update with path fix

fix cargo

update autosnapshot with block lookback (anza-xyz#201)

[JIT-460] When claiming mev tips, skip accounts that won't have min rent exempt amount after claiming (anza-xyz#203)

Add logging for sol balance desired (anza-xyz#205)

* add logging

* add logging

* update msg

* tweak vars

update submodule (anza-xyz#204)

use efficient data structures when calling batch_simulate_bundles (anza-xyz#206)

[JIT-504] Add low balance check in uploading merkle roots (anza-xyz#209)

add config to simulate on top of working bank (anza-xyz#211)

rm frozen bank check

simulate_bundle rpc bugfixes (anza-xyz#214)

rm frozen bank check in simulate_bundle rpc method

[JIT-519] Store ClaimStatus address in merkle-root-json (anza-xyz#210)

* add files

* switch to include bump

update submodule (anza-xyz#217)

add amount filter (anza-xyz#218)

update autosnapshot (anza-xyz#222)

Print TX error in Bundles (anza-xyz#223)

add new args to support single relayer and block-engine endpoints (anza-xyz#224)

point to new jito-programs submod and invoke updated init tda instruction (anza-xyz#228)

fix clippy errors (anza-xyz#230)

fix validator start scripts (anza-xyz#232)

Point README to gitbook (anza-xyz#237)

use packaged cargo bin to build (anza-xyz#239)

Add validator identity pubkey to StakeMeta (anza-xyz#226)

The vote account associated with a validator is not a permanent link, so log the validator identity as well.

bugfix: conditionally compile with debug flags (anza-xyz#240)

Seg/tip distributor master (anza-xyz#242)

* validate tree nodes

* fix unit tests

* pr feedback

* bump jito-programs submod

Simplify bootstrapping (anza-xyz#241)

* startup without precompile

* update spacing

* use release mode

* spacing

fix validation

rm validation skip

Account for block builder fee when generating excess tip balance (anza-xyz#247)

Improve docker caching

delay constructing claim mev txs (anza-xyz#253)

fix stake meta tests from bb fee (anza-xyz#254)

fix tests

Buffer bundles that exceed cost model (anza-xyz#225)

* buffer bundles that exceed cost model

clear qos failed bundles buffer if not leader soon (anza-xyz#260)

update Cargo.lock to correct solana versions in jito-programs submodule (anza-xyz#265)

fix simulate_bundle client and better error handling (anza-xyz#267)

update submod (anza-xyz#272)

Preallocate Bundle Cost (anza-xyz#238)

fix Dockerfile (anza-xyz#278)

Fix Tests (anza-xyz#279)

Fix Tests (anza-xyz#281)

* fix tests

update jito-programs submod (anza-xyz#282)

add reclaim rent workflow (anza-xyz#283)

update jito-programs submod

fix clippy errs

rm wrong assertion and swap out file write fn call (anza-xyz#292)

Remove security.md (anza-xyz#293)

demote frequent relayer_stage-stream_error to warn (anza-xyz#275)

account for case where TDA exists but not allocated (anza-xyz#295)

implement better retries for tip-distributor workflows (anza-xyz#297)

limit number of concurrent rpc calls (anza-xyz#298)

Discard Empty Packet Batches (anza-xyz#299)

Identity Hotswap (anza-xyz#290)

small fixes (anza-xyz#305)

Set backend config from admin rpc (anza-xyz#304)

Admin Shred Receiver Change (anza-xyz#306)

Seg/rm bundle UUID (anza-xyz#309)

Fix github workflow to recursively clone (anza-xyz#327)

Add recursive checkout for downstream-project-spl.yaml (anza-xyz#341)

Use cluster info functions for tpu (anza-xyz#345)

Use git rev-parse for git sha

Remove blacklisted tx from message_hash_to_transaction (anza-xyz#374)

Updates bootstrap and start scripts needed for local dev. (anza-xyz#384)

Remove Deprecated Cli Args (anza-xyz#387)

Master Rebase

improve simulate_bundle errors and response (anza-xyz#404)

derive Clone on accountoverrides (anza-xyz#416)

Add upsert to AccountOverrides (anza-xyz#419)

update jito-programs (anza-xyz#430)

[JIT-1661] Faster Autosnapshot (anza-xyz#436)

Reverts simulate_transaction result calls to upstream (anza-xyz#446)

Don't unlock accounts in TransactionBatches used during simulation (anza-xyz#449)

first pass at wiring up jito-plugin (anza-xyz#428)

[JIT-1713] Fix bundle's blockspace preallocation (anza-xyz#489)

[JIT-1708] Fix TOC TOU condition for relayer and block engine config (anza-xyz#491)

[JIT-1710] - Optimize Bundle Consumer Checks (anza-xyz#490)

Add Blockhash Metrics to Bundle Committer (anza-xyz#500)

add priority fee ix to mev-claim (anza-xyz#520)

Update Autosnapshot (anza-xyz#548)

Run MEV claims + reclaiming rent-exempt amounts in parallel. (anza-xyz#582)

Update CI (anza-xyz#584)
- Add recursive submodule checkouts.
- Re-add solana-secondary step

Add more release fixes (anza-xyz#585)

Fix more release urls (anza-xyz#588)

[JIT-1812] Fix blocking mutexs (anza-xyz#495)

 [JIT-1711] Compare the unprocessed transaction storage BundleStorage against a constant instead of VecDeque::capacity() (anza-xyz#587)

Automatically rebase Jito-Solana on a periodic basis. Send message on slack during any failures or success.

Fix periodic rebase anza-xyz#594

Fixes the following bugs in the periodic rebase:
Sends multiple messages on failure instead of one
Cancels entire job if one branch fails

Ignore buildkite curl errors for rebasing and try to keep curling until job times out (anza-xyz#597)

Sleep longer waiting for buildkite to start (anza-xyz#598)

correctly initialize account overrides (anza-xyz#595)

Fix: Ensure set contact info to UDP port instead of QUIC (anza-xyz#603)

Add fast replay branch to daily rebase (anza-xyz#607)

take a snapshot of all bundle accounts before sim (anza-xyz#13) (anza-xyz#615)

update jito-programs submodule

Add 2.0 to daily rebase (anza-xyz#626)

Export agave binaries during docker build (anza-xyz#627)

Buffer bundles that exceed processing time and make the allowed processing time longer (anza-xyz#611)

Publish releases to S3 and GCS (anza-xyz#633)

Rebase from different repos (anza-xyz#637)

Point SECURITY.md to immunefi (anza-xyz#671)

Loosen requirements on tip accounts touchable in BankingStage (anza-xyz#683)

Separate out broadcast + retransmit shredstream (anza-xyz#703)

Add packet flag for staked node (anza-xyz#705)

Add auto-rebase to v2.1 (anza-xyz#739)

Fix release github (anza-xyz#745)

Move block_cost_limit tracking to BankingStage in preparation for SIMD-0207 (anza-xyz#753)

Add precompile checks in BundleStage (anza-xyz#787)

Add auto-rebase to v2.2 (anza-xyz#818)

Add better error handling around missing transaction signatures for bundle id generation (anza-xyz#860)

Remove unwrap from authentication (anza-xyz#861)

Revert Jito-Solana WorkingBankEntry changes (anza-xyz#873)

BP anza-xyz#885: Add libclang to Dockerfile (anza-xyz#886)

Remove the tip distributor code (anza-xyz#888)

Rebase: Update anchor to not use deprecated crates
fuzzbuck pushed a commit to ample-sh/agave that referenced this pull request Nov 3, 2025
@buffalu
Jito Patch
9c1fdf0 
only reroute if relayer connected (anza-xyz#123)
feat: add client tls config (anza-xyz#121)
remove extra val (anza-xyz#129)
fix clippy (anza-xyz#130)
copy all binaries to docker-output (anza-xyz#131)
Ledger tool halts at slot passed to create-snapshot (anza-xyz#118)
update program submodule (anza-xyz#133)
quick fix for tips and clearing old bundles (anza-xyz#135)
update submodule to new program (anza-xyz#136)
Improve stake-meta-generator usability (anza-xyz#134)
pinning submodule head (anza-xyz#140)
Use BundleAccountLocker when handling tip txs (anza-xyz#147)
Add metrics for relayer + block engine proxy (anza-xyz#149)
Build claim-mev in docker (anza-xyz#141)
Rework bundle receiving and add metrics (anza-xyz#152) (anza-xyz#154)
update submodule + dev files (anza-xyz#158)
Deterministically find tip amounts, add meta to stake info, and cleanup pubkey/strings in MEV tips (anza-xyz#159)
update jito-programs submodule (anza-xyz#160)
Separate MEV tip related workflow (anza-xyz#161)
Add block builder fee protos (anza-xyz#162)
fix jito programs (anza-xyz#163)
update submodule so autosnapshot exits out of ledger tool early (anza-xyz#164)
Pipe through block builder fee (anza-xyz#167)
pull in new snapshot code (anza-xyz#171)
block builder bug (anza-xyz#172)

Pull in new slack autosnapshot submodule (anza-xyz#174)

sort stake meta json and use int math (anza-xyz#176)

add accountsdb conn submod (anza-xyz#169)

Update tip distribution parameters (anza-xyz#177)

new submodules (anza-xyz#180)

Add buildkite link for jito CI (anza-xyz#183)

Fixed broken links to repositories (anza-xyz#184)

Changed from ssh to https transfer for clone

Seg/update submods (anza-xyz#187)

fix tests (anza-xyz#190)

rm geyser submod (anza-xyz#192)

rm dangling geyser references (anza-xyz#193)

fix syntax err (anza-xyz#195)

use deterministic req ids in batch calls (anza-xyz#199)

update jito-programs

revert cargo

update Cargo lock

update with path fix

fix cargo

update autosnapshot with block lookback (anza-xyz#201)

[JIT-460] When claiming mev tips, skip accounts that won't have min rent exempt amount after claiming (anza-xyz#203)

Add logging for sol balance desired (anza-xyz#205)

* add logging

* add logging

* update msg

* tweak vars

update submodule (anza-xyz#204)

use efficient data structures when calling batch_simulate_bundles (anza-xyz#206)

[JIT-504] Add low balance check in uploading merkle roots (anza-xyz#209)

add config to simulate on top of working bank (anza-xyz#211)

rm frozen bank check

simulate_bundle rpc bugfixes (anza-xyz#214)

rm frozen bank check in simulate_bundle rpc method

[JIT-519] Store ClaimStatus address in merkle-root-json (anza-xyz#210)

* add files

* switch to include bump

update submodule (anza-xyz#217)

add amount filter (anza-xyz#218)

update autosnapshot (anza-xyz#222)

Print TX error in Bundles (anza-xyz#223)

add new args to support single relayer and block-engine endpoints (anza-xyz#224)

point to new jito-programs submod and invoke updated init tda instruction (anza-xyz#228)

fix clippy errors (anza-xyz#230)

fix validator start scripts (anza-xyz#232)

Point README to gitbook (anza-xyz#237)

use packaged cargo bin to build (anza-xyz#239)

Add validator identity pubkey to StakeMeta (anza-xyz#226)

The vote account associated with a validator is not a permanent link, so log the validator identity as well.

bugfix: conditionally compile with debug flags (anza-xyz#240)

Seg/tip distributor master (anza-xyz#242)

* validate tree nodes

* fix unit tests

* pr feedback

* bump jito-programs submod

Simplify bootstrapping (anza-xyz#241)

* startup without precompile

* update spacing

* use release mode

* spacing

fix validation

rm validation skip

Account for block builder fee when generating excess tip balance (anza-xyz#247)

Improve docker caching

delay constructing claim mev txs (anza-xyz#253)

fix stake meta tests from bb fee (anza-xyz#254)

fix tests

Buffer bundles that exceed cost model (anza-xyz#225)

* buffer bundles that exceed cost model

clear qos failed bundles buffer if not leader soon (anza-xyz#260)

update Cargo.lock to correct solana versions in jito-programs submodule (anza-xyz#265)

fix simulate_bundle client and better error handling (anza-xyz#267)

update submod (anza-xyz#272)

Preallocate Bundle Cost (anza-xyz#238)

fix Dockerfile (anza-xyz#278)

Fix Tests (anza-xyz#279)

Fix Tests (anza-xyz#281)

* fix tests

update jito-programs submod (anza-xyz#282)

add reclaim rent workflow (anza-xyz#283)

update jito-programs submod

fix clippy errs

rm wrong assertion and swap out file write fn call (anza-xyz#292)

Remove security.md (anza-xyz#293)

demote frequent relayer_stage-stream_error to warn (anza-xyz#275)

account for case where TDA exists but not allocated (anza-xyz#295)

implement better retries for tip-distributor workflows (anza-xyz#297)

limit number of concurrent rpc calls (anza-xyz#298)

Discard Empty Packet Batches (anza-xyz#299)

Identity Hotswap (anza-xyz#290)

small fixes (anza-xyz#305)

Set backend config from admin rpc (anza-xyz#304)

Admin Shred Receiver Change (anza-xyz#306)

Seg/rm bundle UUID (anza-xyz#309)

Fix github workflow to recursively clone (anza-xyz#327)

Add recursive checkout for downstream-project-spl.yaml (anza-xyz#341)

Use cluster info functions for tpu (anza-xyz#345)

Use git rev-parse for git sha

Remove blacklisted tx from message_hash_to_transaction (anza-xyz#374)

Updates bootstrap and start scripts needed for local dev. (anza-xyz#384)

Remove Deprecated Cli Args (anza-xyz#387)

Master Rebase

improve simulate_bundle errors and response (anza-xyz#404)

derive Clone on accountoverrides (anza-xyz#416)

Add upsert to AccountOverrides (anza-xyz#419)

update jito-programs (anza-xyz#430)

[JIT-1661] Faster Autosnapshot (anza-xyz#436)

Reverts simulate_transaction result calls to upstream (anza-xyz#446)

Don't unlock accounts in TransactionBatches used during simulation (anza-xyz#449)

first pass at wiring up jito-plugin (anza-xyz#428)

[JIT-1713] Fix bundle's blockspace preallocation (anza-xyz#489)

[JIT-1708] Fix TOC TOU condition for relayer and block engine config (anza-xyz#491)

[JIT-1710] - Optimize Bundle Consumer Checks (anza-xyz#490)

Add Blockhash Metrics to Bundle Committer (anza-xyz#500)

add priority fee ix to mev-claim (anza-xyz#520)

Update Autosnapshot (anza-xyz#548)

Run MEV claims + reclaiming rent-exempt amounts in parallel. (anza-xyz#582)

Update CI (anza-xyz#584)
- Add recursive submodule checkouts.
- Re-add solana-secondary step

Add more release fixes (anza-xyz#585)

Fix more release urls (anza-xyz#588)

[JIT-1812] Fix blocking mutexs (anza-xyz#495)

 [JIT-1711] Compare the unprocessed transaction storage BundleStorage against a constant instead of VecDeque::capacity() (anza-xyz#587)

Automatically rebase Jito-Solana on a periodic basis. Send message on slack during any failures or success.

Fix periodic rebase anza-xyz#594

Fixes the following bugs in the periodic rebase:
Sends multiple messages on failure instead of one
Cancels entire job if one branch fails

Ignore buildkite curl errors for rebasing and try to keep curling until job times out (anza-xyz#597)

Sleep longer waiting for buildkite to start (anza-xyz#598)

correctly initialize account overrides (anza-xyz#595)

Fix: Ensure set contact info to UDP port instead of QUIC (anza-xyz#603)

Add fast replay branch to daily rebase (anza-xyz#607)

take a snapshot of all bundle accounts before sim (anza-xyz#13) (anza-xyz#615)

update jito-programs submodule

Add 2.0 to daily rebase (anza-xyz#626)

Export agave binaries during docker build (anza-xyz#627)

Buffer bundles that exceed processing time and make the allowed processing time longer (anza-xyz#611)

Publish releases to S3 and GCS (anza-xyz#633)

Rebase from different repos (anza-xyz#637)

Point SECURITY.md to immunefi (anza-xyz#671)

Loosen requirements on tip accounts touchable in BankingStage (anza-xyz#683)

Separate out broadcast + retransmit shredstream (anza-xyz#703)

Add packet flag for staked node (anza-xyz#705)

Add auto-rebase to v2.1 (anza-xyz#739)

Fix release github (anza-xyz#745)

Move block_cost_limit tracking to BankingStage in preparation for SIMD-0207 (anza-xyz#753)

Add precompile checks in BundleStage (anza-xyz#787)

Add auto-rebase to v2.2 (anza-xyz#818)

Add better error handling around missing transaction signatures for bundle id generation (anza-xyz#860)

Remove unwrap from authentication (anza-xyz#861)

Revert Jito-Solana WorkingBankEntry changes (anza-xyz#873)

Add libclang to Dockerfile (anza-xyz#885)

Remove the tip distributor code (anza-xyz#888)

Rebase: Update anchor to not use deprecated crates

Add TLS webpki roots back in (anza-xyz#933)

Remove trusted relayer packets (anza-xyz#952)

Fix shred retransmit (anza-xyz#954)

[v3.0] Automatically use optimal Block Engine region (anza-xyz#983)

(Backport: anza-xyz#995) Disable autoconfig (anza-xyz#996)
fuzzbuck pushed a commit to ample-sh/agave that referenced this pull request Nov 3, 2025
@buffalu @fuzzbuck
Jito Patch
df24a99 
only reroute if relayer connected (anza-xyz#123)
feat: add client tls config (anza-xyz#121)
remove extra val (anza-xyz#129)
fix clippy (anza-xyz#130)
copy all binaries to docker-output (anza-xyz#131)
Ledger tool halts at slot passed to create-snapshot (anza-xyz#118)
update program submodule (anza-xyz#133)
quick fix for tips and clearing old bundles (anza-xyz#135)
update submodule to new program (anza-xyz#136)
Improve stake-meta-generator usability (anza-xyz#134)
pinning submodule head (anza-xyz#140)
Use BundleAccountLocker when handling tip txs (anza-xyz#147)
Add metrics for relayer + block engine proxy (anza-xyz#149)
Build claim-mev in docker (anza-xyz#141)
Rework bundle receiving and add metrics (anza-xyz#152) (anza-xyz#154)
update submodule + dev files (anza-xyz#158)
Deterministically find tip amounts, add meta to stake info, and cleanup pubkey/strings in MEV tips (anza-xyz#159)
update jito-programs submodule (anza-xyz#160)
Separate MEV tip related workflow (anza-xyz#161)
Add block builder fee protos (anza-xyz#162)
fix jito programs (anza-xyz#163)
update submodule so autosnapshot exits out of ledger tool early (anza-xyz#164)
Pipe through block builder fee (anza-xyz#167)
pull in new snapshot code (anza-xyz#171)
block builder bug (anza-xyz#172)

Pull in new slack autosnapshot submodule (anza-xyz#174)

sort stake meta json and use int math (anza-xyz#176)

add accountsdb conn submod (anza-xyz#169)

Update tip distribution parameters (anza-xyz#177)

new submodules (anza-xyz#180)

Add buildkite link for jito CI (anza-xyz#183)

Fixed broken links to repositories (anza-xyz#184)

Changed from ssh to https transfer for clone

Seg/update submods (anza-xyz#187)

fix tests (anza-xyz#190)

rm geyser submod (anza-xyz#192)

rm dangling geyser references (anza-xyz#193)

fix syntax err (anza-xyz#195)

use deterministic req ids in batch calls (anza-xyz#199)

update jito-programs

revert cargo

update Cargo lock

update with path fix

fix cargo

update autosnapshot with block lookback (anza-xyz#201)

[JIT-460] When claiming mev tips, skip accounts that won't have min rent exempt amount after claiming (anza-xyz#203)

Add logging for sol balance desired (anza-xyz#205)

* add logging

* add logging

* update msg

* tweak vars

update submodule (anza-xyz#204)

use efficient data structures when calling batch_simulate_bundles (anza-xyz#206)

[JIT-504] Add low balance check in uploading merkle roots (anza-xyz#209)

add config to simulate on top of working bank (anza-xyz#211)

rm frozen bank check

simulate_bundle rpc bugfixes (anza-xyz#214)

rm frozen bank check in simulate_bundle rpc method

[JIT-519] Store ClaimStatus address in merkle-root-json (anza-xyz#210)

* add files

* switch to include bump

update submodule (anza-xyz#217)

add amount filter (anza-xyz#218)

update autosnapshot (anza-xyz#222)

Print TX error in Bundles (anza-xyz#223)

add new args to support single relayer and block-engine endpoints (anza-xyz#224)

point to new jito-programs submod and invoke updated init tda instruction (anza-xyz#228)

fix clippy errors (anza-xyz#230)

fix validator start scripts (anza-xyz#232)

Point README to gitbook (anza-xyz#237)

use packaged cargo bin to build (anza-xyz#239)

Add validator identity pubkey to StakeMeta (anza-xyz#226)

The vote account associated with a validator is not a permanent link, so log the validator identity as well.

bugfix: conditionally compile with debug flags (anza-xyz#240)

Seg/tip distributor master (anza-xyz#242)

* validate tree nodes

* fix unit tests

* pr feedback

* bump jito-programs submod

Simplify bootstrapping (anza-xyz#241)

* startup without precompile

* update spacing

* use release mode

* spacing

fix validation

rm validation skip

Account for block builder fee when generating excess tip balance (anza-xyz#247)

Improve docker caching

delay constructing claim mev txs (anza-xyz#253)

fix stake meta tests from bb fee (anza-xyz#254)

fix tests

Buffer bundles that exceed cost model (anza-xyz#225)

* buffer bundles that exceed cost model

clear qos failed bundles buffer if not leader soon (anza-xyz#260)

update Cargo.lock to correct solana versions in jito-programs submodule (anza-xyz#265)

fix simulate_bundle client and better error handling (anza-xyz#267)

update submod (anza-xyz#272)

Preallocate Bundle Cost (anza-xyz#238)

fix Dockerfile (anza-xyz#278)

Fix Tests (anza-xyz#279)

Fix Tests (anza-xyz#281)

* fix tests

update jito-programs submod (anza-xyz#282)

add reclaim rent workflow (anza-xyz#283)

update jito-programs submod

fix clippy errs

rm wrong assertion and swap out file write fn call (anza-xyz#292)

Remove security.md (anza-xyz#293)

demote frequent relayer_stage-stream_error to warn (anza-xyz#275)

account for case where TDA exists but not allocated (anza-xyz#295)

implement better retries for tip-distributor workflows (anza-xyz#297)

limit number of concurrent rpc calls (anza-xyz#298)

Discard Empty Packet Batches (anza-xyz#299)

Identity Hotswap (anza-xyz#290)

small fixes (anza-xyz#305)

Set backend config from admin rpc (anza-xyz#304)

Admin Shred Receiver Change (anza-xyz#306)

Seg/rm bundle UUID (anza-xyz#309)

Fix github workflow to recursively clone (anza-xyz#327)

Add recursive checkout for downstream-project-spl.yaml (anza-xyz#341)

Use cluster info functions for tpu (anza-xyz#345)

Use git rev-parse for git sha

Remove blacklisted tx from message_hash_to_transaction (anza-xyz#374)

Updates bootstrap and start scripts needed for local dev. (anza-xyz#384)

Remove Deprecated Cli Args (anza-xyz#387)

Master Rebase

improve simulate_bundle errors and response (anza-xyz#404)

derive Clone on accountoverrides (anza-xyz#416)

Add upsert to AccountOverrides (anza-xyz#419)

update jito-programs (anza-xyz#430)

[JIT-1661] Faster Autosnapshot (anza-xyz#436)

Reverts simulate_transaction result calls to upstream (anza-xyz#446)

Don't unlock accounts in TransactionBatches used during simulation (anza-xyz#449)

first pass at wiring up jito-plugin (anza-xyz#428)

[JIT-1713] Fix bundle's blockspace preallocation (anza-xyz#489)

[JIT-1708] Fix TOC TOU condition for relayer and block engine config (anza-xyz#491)

[JIT-1710] - Optimize Bundle Consumer Checks (anza-xyz#490)

Add Blockhash Metrics to Bundle Committer (anza-xyz#500)

add priority fee ix to mev-claim (anza-xyz#520)

Update Autosnapshot (anza-xyz#548)

Run MEV claims + reclaiming rent-exempt amounts in parallel. (anza-xyz#582)

Update CI (anza-xyz#584)
- Add recursive submodule checkouts.
- Re-add solana-secondary step

Add more release fixes (anza-xyz#585)

Fix more release urls (anza-xyz#588)

[JIT-1812] Fix blocking mutexs (anza-xyz#495)

 [JIT-1711] Compare the unprocessed transaction storage BundleStorage against a constant instead of VecDeque::capacity() (anza-xyz#587)

Automatically rebase Jito-Solana on a periodic basis. Send message on slack during any failures or success.

Fix periodic rebase anza-xyz#594

Fixes the following bugs in the periodic rebase:
Sends multiple messages on failure instead of one
Cancels entire job if one branch fails

Ignore buildkite curl errors for rebasing and try to keep curling until job times out (anza-xyz#597)

Sleep longer waiting for buildkite to start (anza-xyz#598)

correctly initialize account overrides (anza-xyz#595)

Fix: Ensure set contact info to UDP port instead of QUIC (anza-xyz#603)

Add fast replay branch to daily rebase (anza-xyz#607)

take a snapshot of all bundle accounts before sim (anza-xyz#13) (anza-xyz#615)

update jito-programs submodule

Add 2.0 to daily rebase (anza-xyz#626)

Export agave binaries during docker build (anza-xyz#627)

Buffer bundles that exceed processing time and make the allowed processing time longer (anza-xyz#611)

Publish releases to S3 and GCS (anza-xyz#633)

Rebase from different repos (anza-xyz#637)

Point SECURITY.md to immunefi (anza-xyz#671)

Loosen requirements on tip accounts touchable in BankingStage (anza-xyz#683)

Separate out broadcast + retransmit shredstream (anza-xyz#703)

Add packet flag for staked node (anza-xyz#705)

Add auto-rebase to v2.1 (anza-xyz#739)

Fix release github (anza-xyz#745)

Move block_cost_limit tracking to BankingStage in preparation for SIMD-0207 (anza-xyz#753)

Add precompile checks in BundleStage (anza-xyz#787)

Add auto-rebase to v2.2 (anza-xyz#818)

Add better error handling around missing transaction signatures for bundle id generation (anza-xyz#860)

Remove unwrap from authentication (anza-xyz#861)

Revert Jito-Solana WorkingBankEntry changes (anza-xyz#873)

Add libclang to Dockerfile (anza-xyz#885)

Remove the tip distributor code (anza-xyz#888)

Rebase: Update anchor to not use deprecated crates

Add TLS webpki roots back in (anza-xyz#933)

Remove trusted relayer packets (anza-xyz#952)

Fix shred retransmit (anza-xyz#954)

[v3.0] Automatically use optimal Block Engine region (anza-xyz#983)

(Backport: anza-xyz#995) Disable autoconfig (anza-xyz#996)
fuzzbuck pushed a commit to ample-sh/agave that referenced this pull request Nov 27, 2025
@buffalu
Jito Patch
52d5d94 
only reroute if relayer connected (anza-xyz#123)
feat: add client tls config (anza-xyz#121)
remove extra val (anza-xyz#129)
fix clippy (anza-xyz#130)
copy all binaries to docker-output (anza-xyz#131)
Ledger tool halts at slot passed to create-snapshot (anza-xyz#118)
update program submodule (anza-xyz#133)
quick fix for tips and clearing old bundles (anza-xyz#135)
update submodule to new program (anza-xyz#136)
Improve stake-meta-generator usability (anza-xyz#134)
pinning submodule head (anza-xyz#140)
Use BundleAccountLocker when handling tip txs (anza-xyz#147)
Add metrics for relayer + block engine proxy (anza-xyz#149)
Build claim-mev in docker (anza-xyz#141)
Rework bundle receiving and add metrics (anza-xyz#152) (anza-xyz#154)
update submodule + dev files (anza-xyz#158)
Deterministically find tip amounts, add meta to stake info, and cleanup pubkey/strings in MEV tips (anza-xyz#159)
update jito-programs submodule (anza-xyz#160)
Separate MEV tip related workflow (anza-xyz#161)
Add block builder fee protos (anza-xyz#162)
fix jito programs (anza-xyz#163)
update submodule so autosnapshot exits out of ledger tool early (anza-xyz#164)
Pipe through block builder fee (anza-xyz#167)
pull in new snapshot code (anza-xyz#171)
block builder bug (anza-xyz#172)

Pull in new slack autosnapshot submodule (anza-xyz#174)

sort stake meta json and use int math (anza-xyz#176)

add accountsdb conn submod (anza-xyz#169)

Update tip distribution parameters (anza-xyz#177)

new submodules (anza-xyz#180)

Add buildkite link for jito CI (anza-xyz#183)

Fixed broken links to repositories (anza-xyz#184)

Changed from ssh to https transfer for clone

Seg/update submods (anza-xyz#187)

fix tests (anza-xyz#190)

rm geyser submod (anza-xyz#192)

rm dangling geyser references (anza-xyz#193)

fix syntax err (anza-xyz#195)

use deterministic req ids in batch calls (anza-xyz#199)

update jito-programs

revert cargo

update Cargo lock

update with path fix

fix cargo

update autosnapshot with block lookback (anza-xyz#201)

[JIT-460] When claiming mev tips, skip accounts that won't have min rent exempt amount after claiming (anza-xyz#203)

Add logging for sol balance desired (anza-xyz#205)

* add logging

* add logging

* update msg

* tweak vars

update submodule (anza-xyz#204)

use efficient data structures when calling batch_simulate_bundles (anza-xyz#206)

[JIT-504] Add low balance check in uploading merkle roots (anza-xyz#209)

add config to simulate on top of working bank (anza-xyz#211)

rm frozen bank check

simulate_bundle rpc bugfixes (anza-xyz#214)

rm frozen bank check in simulate_bundle rpc method

[JIT-519] Store ClaimStatus address in merkle-root-json (anza-xyz#210)

* add files

* switch to include bump

update submodule (anza-xyz#217)

add amount filter (anza-xyz#218)

update autosnapshot (anza-xyz#222)

Print TX error in Bundles (anza-xyz#223)

add new args to support single relayer and block-engine endpoints (anza-xyz#224)

point to new jito-programs submod and invoke updated init tda instruction (anza-xyz#228)

fix clippy errors (anza-xyz#230)

fix validator start scripts (anza-xyz#232)

Point README to gitbook (anza-xyz#237)

use packaged cargo bin to build (anza-xyz#239)

Add validator identity pubkey to StakeMeta (anza-xyz#226)

The vote account associated with a validator is not a permanent link, so log the validator identity as well.

bugfix: conditionally compile with debug flags (anza-xyz#240)

Seg/tip distributor master (anza-xyz#242)

* validate tree nodes

* fix unit tests

* pr feedback

* bump jito-programs submod

Simplify bootstrapping (anza-xyz#241)

* startup without precompile

* update spacing

* use release mode

* spacing

fix validation

rm validation skip

Account for block builder fee when generating excess tip balance (anza-xyz#247)

Improve docker caching

delay constructing claim mev txs (anza-xyz#253)

fix stake meta tests from bb fee (anza-xyz#254)

fix tests

Buffer bundles that exceed cost model (anza-xyz#225)

* buffer bundles that exceed cost model

clear qos failed bundles buffer if not leader soon (anza-xyz#260)

update Cargo.lock to correct solana versions in jito-programs submodule (anza-xyz#265)

fix simulate_bundle client and better error handling (anza-xyz#267)

update submod (anza-xyz#272)

Preallocate Bundle Cost (anza-xyz#238)

fix Dockerfile (anza-xyz#278)

Fix Tests (anza-xyz#279)

Fix Tests (anza-xyz#281)

* fix tests

update jito-programs submod (anza-xyz#282)

add reclaim rent workflow (anza-xyz#283)

update jito-programs submod

fix clippy errs

rm wrong assertion and swap out file write fn call (anza-xyz#292)

Remove security.md (anza-xyz#293)

demote frequent relayer_stage-stream_error to warn (anza-xyz#275)

account for case where TDA exists but not allocated (anza-xyz#295)

implement better retries for tip-distributor workflows (anza-xyz#297)

limit number of concurrent rpc calls (anza-xyz#298)

Discard Empty Packet Batches (anza-xyz#299)

Identity Hotswap (anza-xyz#290)

small fixes (anza-xyz#305)

Set backend config from admin rpc (anza-xyz#304)

Admin Shred Receiver Change (anza-xyz#306)

Seg/rm bundle UUID (anza-xyz#309)

Fix github workflow to recursively clone (anza-xyz#327)

Add recursive checkout for downstream-project-spl.yaml (anza-xyz#341)

Use cluster info functions for tpu (anza-xyz#345)

Use git rev-parse for git sha

Remove blacklisted tx from message_hash_to_transaction (anza-xyz#374)

Updates bootstrap and start scripts needed for local dev. (anza-xyz#384)

Remove Deprecated Cli Args (anza-xyz#387)

Master Rebase

improve simulate_bundle errors and response (anza-xyz#404)

derive Clone on accountoverrides (anza-xyz#416)

Add upsert to AccountOverrides (anza-xyz#419)

update jito-programs (anza-xyz#430)

[JIT-1661] Faster Autosnapshot (anza-xyz#436)

Reverts simulate_transaction result calls to upstream (anza-xyz#446)

Don't unlock accounts in TransactionBatches used during simulation (anza-xyz#449)

first pass at wiring up jito-plugin (anza-xyz#428)

[JIT-1713] Fix bundle's blockspace preallocation (anza-xyz#489)

[JIT-1708] Fix TOC TOU condition for relayer and block engine config (anza-xyz#491)

[JIT-1710] - Optimize Bundle Consumer Checks (anza-xyz#490)

Add Blockhash Metrics to Bundle Committer (anza-xyz#500)

add priority fee ix to mev-claim (anza-xyz#520)

Update Autosnapshot (anza-xyz#548)

Run MEV claims + reclaiming rent-exempt amounts in parallel. (anza-xyz#582)

Update CI (anza-xyz#584)
- Add recursive submodule checkouts.
- Re-add solana-secondary step

Add more release fixes (anza-xyz#585)

Fix more release urls (anza-xyz#588)

[JIT-1812] Fix blocking mutexs (anza-xyz#495)

 [JIT-1711] Compare the unprocessed transaction storage BundleStorage against a constant instead of VecDeque::capacity() (anza-xyz#587)

Automatically rebase Jito-Solana on a periodic basis. Send message on slack during any failures or success.

Fix periodic rebase anza-xyz#594

Fixes the following bugs in the periodic rebase:
Sends multiple messages on failure instead of one
Cancels entire job if one branch fails

Ignore buildkite curl errors for rebasing and try to keep curling until job times out (anza-xyz#597)

Sleep longer waiting for buildkite to start (anza-xyz#598)

correctly initialize account overrides (anza-xyz#595)

Fix: Ensure set contact info to UDP port instead of QUIC (anza-xyz#603)

Add fast replay branch to daily rebase (anza-xyz#607)

take a snapshot of all bundle accounts before sim (anza-xyz#13) (anza-xyz#615)

update jito-programs submodule

Add 2.0 to daily rebase (anza-xyz#626)

Export agave binaries during docker build (anza-xyz#627)

Buffer bundles that exceed processing time and make the allowed processing time longer (anza-xyz#611)

Publish releases to S3 and GCS (anza-xyz#633)

Rebase from different repos (anza-xyz#637)

Point SECURITY.md to immunefi (anza-xyz#671)

Loosen requirements on tip accounts touchable in BankingStage (anza-xyz#683)

Separate out broadcast + retransmit shredstream (anza-xyz#703)

Add packet flag for staked node (anza-xyz#705)

Add auto-rebase to v2.1 (anza-xyz#739)

Fix release github (anza-xyz#745)

Move block_cost_limit tracking to BankingStage in preparation for SIMD-0207 (anza-xyz#753)

Add precompile checks in BundleStage (anza-xyz#787)

Add auto-rebase to v2.2 (anza-xyz#818)

Add better error handling around missing transaction signatures for bundle id generation (anza-xyz#860)

Remove unwrap from authentication (anza-xyz#861)

Revert Jito-Solana WorkingBankEntry changes (anza-xyz#873)

Add libclang to Dockerfile (anza-xyz#885)

Remove the tip distributor code (anza-xyz#888)

Rebase: Update anchor to not use deprecated crates

Add TLS webpki roots back in (anza-xyz#933)

Remove trusted relayer packets (anza-xyz#952)

Fix shred retransmit (anza-xyz#954)

[v3.0] Automatically use optimal Block Engine region (anza-xyz#983)

(Backport: anza-xyz#995) Disable autoconfig (anza-xyz#996)
OliverNChalk pushed a commit to OliverNChalk/agave that referenced this pull request Dec 1, 2025
@buffalu
Jito Patch
e791c55 
only reroute if relayer connected (anza-xyz#123)
feat: add client tls config (anza-xyz#121)
remove extra val (anza-xyz#129)
fix clippy (anza-xyz#130)
copy all binaries to docker-output (anza-xyz#131)
Ledger tool halts at slot passed to create-snapshot (anza-xyz#118)
update program submodule (anza-xyz#133)
quick fix for tips and clearing old bundles (anza-xyz#135)
update submodule to new program (anza-xyz#136)
Improve stake-meta-generator usability (anza-xyz#134)
pinning submodule head (anza-xyz#140)
Use BundleAccountLocker when handling tip txs (anza-xyz#147)
Add metrics for relayer + block engine proxy (anza-xyz#149)
Build claim-mev in docker (anza-xyz#141)
Rework bundle receiving and add metrics (anza-xyz#152) (anza-xyz#154)
update submodule + dev files (anza-xyz#158)
Deterministically find tip amounts, add meta to stake info, and cleanup pubkey/strings in MEV tips (anza-xyz#159)
update jito-programs submodule (anza-xyz#160)
Separate MEV tip related workflow (anza-xyz#161)
Add block builder fee protos (anza-xyz#162)
fix jito programs (anza-xyz#163)
update submodule so autosnapshot exits out of ledger tool early (anza-xyz#164)
Pipe through block builder fee (anza-xyz#167)
pull in new snapshot code (anza-xyz#171)
block builder bug (anza-xyz#172)

Pull in new slack autosnapshot submodule (anza-xyz#174)

sort stake meta json and use int math (anza-xyz#176)

add accountsdb conn submod (anza-xyz#169)

Update tip distribution parameters (anza-xyz#177)

new submodules (anza-xyz#180)

Add buildkite link for jito CI (anza-xyz#183)

Fixed broken links to repositories (anza-xyz#184)

Changed from ssh to https transfer for clone

Seg/update submods (anza-xyz#187)

fix tests (anza-xyz#190)

rm geyser submod (anza-xyz#192)

rm dangling geyser references (anza-xyz#193)

fix syntax err (anza-xyz#195)

use deterministic req ids in batch calls (anza-xyz#199)

update jito-programs

revert cargo

update Cargo lock

update with path fix

fix cargo

update autosnapshot with block lookback (anza-xyz#201)

[JIT-460] When claiming mev tips, skip accounts that won't have min rent exempt amount after claiming (anza-xyz#203)

Add logging for sol balance desired (anza-xyz#205)

* add logging

* add logging

* update msg

* tweak vars

update submodule (anza-xyz#204)

use efficient data structures when calling batch_simulate_bundles (anza-xyz#206)

[JIT-504] Add low balance check in uploading merkle roots (anza-xyz#209)

add config to simulate on top of working bank (anza-xyz#211)

rm frozen bank check

simulate_bundle rpc bugfixes (anza-xyz#214)

rm frozen bank check in simulate_bundle rpc method

[JIT-519] Store ClaimStatus address in merkle-root-json (anza-xyz#210)

* add files

* switch to include bump

update submodule (anza-xyz#217)

add amount filter (anza-xyz#218)

update autosnapshot (anza-xyz#222)

Print TX error in Bundles (anza-xyz#223)

add new args to support single relayer and block-engine endpoints (anza-xyz#224)

point to new jito-programs submod and invoke updated init tda instruction (anza-xyz#228)

fix clippy errors (anza-xyz#230)

fix validator start scripts (anza-xyz#232)

Point README to gitbook (anza-xyz#237)

use packaged cargo bin to build (anza-xyz#239)

Add validator identity pubkey to StakeMeta (anza-xyz#226)

The vote account associated with a validator is not a permanent link, so log the validator identity as well.

bugfix: conditionally compile with debug flags (anza-xyz#240)

Seg/tip distributor master (anza-xyz#242)

* validate tree nodes

* fix unit tests

* pr feedback

* bump jito-programs submod

Simplify bootstrapping (anza-xyz#241)

* startup without precompile

* update spacing

* use release mode

* spacing

fix validation

rm validation skip

Account for block builder fee when generating excess tip balance (anza-xyz#247)

Improve docker caching

delay constructing claim mev txs (anza-xyz#253)

fix stake meta tests from bb fee (anza-xyz#254)

fix tests

Buffer bundles that exceed cost model (anza-xyz#225)

* buffer bundles that exceed cost model

clear qos failed bundles buffer if not leader soon (anza-xyz#260)

update Cargo.lock to correct solana versions in jito-programs submodule (anza-xyz#265)

fix simulate_bundle client and better error handling (anza-xyz#267)

update submod (anza-xyz#272)

Preallocate Bundle Cost (anza-xyz#238)

fix Dockerfile (anza-xyz#278)

Fix Tests (anza-xyz#279)

Fix Tests (anza-xyz#281)

* fix tests

update jito-programs submod (anza-xyz#282)

add reclaim rent workflow (anza-xyz#283)

update jito-programs submod

fix clippy errs

rm wrong assertion and swap out file write fn call (anza-xyz#292)

Remove security.md (anza-xyz#293)

demote frequent relayer_stage-stream_error to warn (anza-xyz#275)

account for case where TDA exists but not allocated (anza-xyz#295)

implement better retries for tip-distributor workflows (anza-xyz#297)

limit number of concurrent rpc calls (anza-xyz#298)

Discard Empty Packet Batches (anza-xyz#299)

Identity Hotswap (anza-xyz#290)

small fixes (anza-xyz#305)

Set backend config from admin rpc (anza-xyz#304)

Admin Shred Receiver Change (anza-xyz#306)

Seg/rm bundle UUID (anza-xyz#309)

Fix github workflow to recursively clone (anza-xyz#327)

Add recursive checkout for downstream-project-spl.yaml (anza-xyz#341)

Use cluster info functions for tpu (anza-xyz#345)

Use git rev-parse for git sha

Remove blacklisted tx from message_hash_to_transaction (anza-xyz#374)

Updates bootstrap and start scripts needed for local dev. (anza-xyz#384)

Remove Deprecated Cli Args (anza-xyz#387)

Master Rebase

improve simulate_bundle errors and response (anza-xyz#404)

derive Clone on accountoverrides (anza-xyz#416)

Add upsert to AccountOverrides (anza-xyz#419)

update jito-programs (anza-xyz#430)

[JIT-1661] Faster Autosnapshot (anza-xyz#436)

Reverts simulate_transaction result calls to upstream (anza-xyz#446)

Don't unlock accounts in TransactionBatches used during simulation (anza-xyz#449)

first pass at wiring up jito-plugin (anza-xyz#428)

[JIT-1713] Fix bundle's blockspace preallocation (anza-xyz#489)

[JIT-1708] Fix TOC TOU condition for relayer and block engine config (anza-xyz#491)

[JIT-1710] - Optimize Bundle Consumer Checks (anza-xyz#490)

Add Blockhash Metrics to Bundle Committer (anza-xyz#500)

add priority fee ix to mev-claim (anza-xyz#520)

Update Autosnapshot (anza-xyz#548)

Run MEV claims + reclaiming rent-exempt amounts in parallel. (anza-xyz#582)

Update CI (anza-xyz#584)
- Add recursive submodule checkouts.
- Re-add solana-secondary step

Add more release fixes (anza-xyz#585)

Fix more release urls (anza-xyz#588)

[JIT-1812] Fix blocking mutexs (anza-xyz#495)

 [JIT-1711] Compare the unprocessed transaction storage BundleStorage against a constant instead of VecDeque::capacity() (anza-xyz#587)

Automatically rebase Jito-Solana on a periodic basis. Send message on slack during any failures or success.

Fix periodic rebase anza-xyz#594

Fixes the following bugs in the periodic rebase:
Sends multiple messages on failure instead of one
Cancels entire job if one branch fails

Ignore buildkite curl errors for rebasing and try to keep curling until job times out (anza-xyz#597)

Sleep longer waiting for buildkite to start (anza-xyz#598)

correctly initialize account overrides (anza-xyz#595)

Fix: Ensure set contact info to UDP port instead of QUIC (anza-xyz#603)

Add fast replay branch to daily rebase (anza-xyz#607)

take a snapshot of all bundle accounts before sim (anza-xyz#13) (anza-xyz#615)

update jito-programs submodule

Add 2.0 to daily rebase (anza-xyz#626)

Export agave binaries during docker build (anza-xyz#627)

Buffer bundles that exceed processing time and make the allowed processing time longer (anza-xyz#611)

Publish releases to S3 and GCS (anza-xyz#633)

Rebase from different repos (anza-xyz#637)

Point SECURITY.md to immunefi (anza-xyz#671)

Loosen requirements on tip accounts touchable in BankingStage (anza-xyz#683)

Separate out broadcast + retransmit shredstream (anza-xyz#703)

Add packet flag for staked node (anza-xyz#705)

Add auto-rebase to v2.1 (anza-xyz#739)

Fix release github (anza-xyz#745)

Move block_cost_limit tracking to BankingStage in preparation for SIMD-0207 (anza-xyz#753)

Add precompile checks in BundleStage (anza-xyz#787)

Add auto-rebase to v2.2 (anza-xyz#818)

Add better error handling around missing transaction signatures for bundle id generation (anza-xyz#860)

Remove unwrap from authentication (anza-xyz#861)

Revert Jito-Solana WorkingBankEntry changes (anza-xyz#873)

Add libclang to Dockerfile (anza-xyz#885)

Remove the tip distributor code (anza-xyz#888)

Rebase: Update anchor to not use deprecated crates

Add TLS webpki roots back in (anza-xyz#933)

Remove trusted relayer packets (anza-xyz#952)

Fix shred retransmit (anza-xyz#954)

Add daily v3.0 rebase (anza-xyz#972)

[Master] Automatically use optimal Block Engine region (anza-xyz#974)

Disable autoconfig (anza-xyz#995)

Make shredstream optional (anza-xyz#997)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@brooksprumo brooksprumo brooksprumo approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@yhchiang-sol @codecov-commenter @brooksprumo