[TieredStorage] Add AccountsFile::TieredStorage by yhchiang-sol · Pull Request #72 · anza-xyz/agave

yhchiang-sol · 2024-03-04T23:14:11Z

Problem

AccountsFile currently doesn't have an implementation for TieredStorage.
To enable AccountsDB tests for the TieredStorage, we need AccountsFile
to support TieredStorage.

Summary of Changes

This PR implements a AccountsFile::TieredStorage, a thin wrapper between
AccountsFile and TieredStorage.

yhchiang-sol · 2024-03-05T18:04:46Z

Putting it back to draft to understand more about the test failure.

brooksprumo · 2024-03-05T20:19:13Z

Please readd me as a reviewer when the PR is ready. Thanks!

codecov-commenter · 2024-03-12T00:30:33Z

Codecov Report

Attention: Patch coverage is 21.42857% with 22 lines in your changes are missing coverage. Please review.

Project coverage is 81.8%. Comparing base (b6d2237) to head (5b85d59).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##           master      #72     +/-   ##
=========================================
- Coverage    81.9%    81.8%   -0.1%     
=========================================
  Files         840      840             
  Lines      228058   228105     +47     
=========================================
+ Hits       186795   186815     +20     
- Misses      41263    41290     +27

yhchiang-sol · 2024-03-22T04:06:36Z

Rebase to address conflicts.

yhchiang-sol · 2024-03-22T20:53:04Z

Rebased and addressed comments.

yhchiang-sol · 2024-03-22T21:22:04Z

Rebased on top of master with #400

yhchiang-sol · 2024-03-22T22:06:28Z

Rebased on top of #401 that includes the capacity() API.

yhchiang-sol · 2024-03-23T00:40:48Z

Rebased on top of #401

yhchiang-sol · 2024-03-24T05:18:26Z

Rebased on top of master with #401

brooksprumo

…ed-storage (#418) #### Problem As #72 introduced AccountsFile::TieredStorage, it also performs file-type check when opening an accounts-file to determine whether it is a tiered-storage or an append-vec. But before tiered-storage is enabled, this opening check is unnecessary. #### Summary of Changes Remove the accounts-file type check code and simply assume everything is append-vec on AccountsFile::new_from_file().

mergify Bot added the need:merge-assist label Mar 4, 2024

yhchiang-sol mentioned this pull request Mar 4, 2024

[TieredStorage] Make TieredStorage's Offset compatible with AccountsDB's offset #74

Merged

yhchiang-sol requested a review from brooksprumo March 4, 2024 23:31

yhchiang-sol marked this pull request as draft March 5, 2024 18:04

brooksprumo removed their request for review March 5, 2024 20:18

yhchiang-sol force-pushed the ts-pr-accounts-file-hot branch 3 times, most recently from b76fd86 to f4f04f9 Compare March 11, 2024 23:02

yhchiang-sol mentioned this pull request Mar 11, 2024

[TieredStorage] Refactor TieredStorage::new_readonly() code path #195

Merged

yhchiang-sol force-pushed the ts-pr-accounts-file-hot branch 4 times, most recently from 37f4515 to 80b251e Compare March 15, 2024 17:36

yhchiang-sol mentioned this pull request Mar 20, 2024

Introduce AccountsFileProvider #334

Closed

yhchiang-sol force-pushed the ts-pr-accounts-file-hot branch from 80b251e to b5b3309 Compare March 20, 2024 17:51

yhchiang-sol marked this pull request as ready for review March 20, 2024 17:51

yhchiang-sol requested a review from brooksprumo March 20, 2024 17:51

yhchiang-sol mentioned this pull request Mar 20, 2024

Enable accountsdb_scan_account_storage_no_bank tests for hot storage #344

Merged

brooksprumo removed the need:merge-assist label Mar 20, 2024

yhchiang-sol commented Mar 20, 2024

View reviewed changes

Comment thread accounts-db/src/accounts_file.rs Outdated

yhchiang-sol changed the title ~~[TieredStorage] Add AccountsFile::TieredHot~~ [TieredStorage] Add AccountsFile::TieredStorage Mar 20, 2024

brooksprumo reviewed Mar 20, 2024

View reviewed changes

yhchiang-sol mentioned this pull request Mar 22, 2024

[TieredStorage] Add TieredStorage::write_accounts_to_hot_storage() API #382

Closed

yhchiang-sol force-pushed the ts-pr-accounts-file-hot branch from 51fe14a to 9ca4257 Compare March 22, 2024 04:06

yhchiang-sol force-pushed the ts-pr-accounts-file-hot branch 2 times, most recently from 9db160d to 40030e1 Compare March 22, 2024 08:45

yhchiang-sol commented Mar 22, 2024

View reviewed changes

Comment thread accounts-db/src/accounts_file.rs Outdated

Comment thread accounts-db/src/accounts_file.rs

yhchiang-sol requested a review from brooksprumo March 22, 2024 09:07

brooksprumo reviewed Mar 22, 2024

View reviewed changes

Comment thread accounts-db/src/accounts_file.rs

Comment thread accounts-db/src/accounts_file.rs Outdated

Comment thread accounts-db/src/accounts_file.rs

brooksprumo reviewed Mar 22, 2024

View reviewed changes

Comment thread accounts-db/src/accounts_file.rs Outdated

yhchiang-sol requested a review from brooksprumo March 22, 2024 19:03

brooksprumo reviewed Mar 22, 2024

View reviewed changes

Comment thread accounts-db/src/accounts_file.rs

Comment thread accounts-db/src/accounts_file.rs

yhchiang-sol force-pushed the ts-pr-accounts-file-hot branch from 732964a to 717476f Compare March 22, 2024 20:51

yhchiang-sol requested a review from brooksprumo March 22, 2024 20:53

yhchiang-sol mentioned this pull request Mar 22, 2024

[TieredStorage] Refactor file_size() code path #400

Merged

yhchiang-sol force-pushed the ts-pr-accounts-file-hot branch from 717476f to 0c7053d Compare March 22, 2024 21:21

yhchiang-sol force-pushed the ts-pr-accounts-file-hot branch from 0c7053d to 5f71d03 Compare March 22, 2024 22:05

brooksprumo reviewed Mar 22, 2024

View reviewed changes

Comment thread accounts-db/src/accounts_file.rs Outdated

Comment thread accounts-db/src/accounts_file.rs Outdated

yhchiang-sol force-pushed the ts-pr-accounts-file-hot branch from 5f71d03 to 8441131 Compare March 23, 2024 00:40

yhchiang-sol requested a review from brooksprumo March 23, 2024 00:43

brooksprumo reviewed Mar 23, 2024

View reviewed changes

Comment thread accounts-db/src/accounts_file.rs Outdated

Comment thread accounts-db/src/ancient_append_vecs.rs

yhchiang-sol force-pushed the ts-pr-accounts-file-hot branch from a33f52f to 6a825f1 Compare March 24, 2024 05:16

[TieredStorage] Add AccountsFile::TieredStorage

5b85d59

yhchiang-sol force-pushed the ts-pr-accounts-file-hot branch from 6a825f1 to 5b85d59 Compare March 24, 2024 05:17

yhchiang-sol requested a review from brooksprumo March 24, 2024 05:18

brooksprumo approved these changes Mar 24, 2024

View reviewed changes

yhchiang-sol merged commit a916edb into anza-xyz:master Mar 24, 2024

yhchiang-sol mentioned this pull request Mar 25, 2024

[TieredStorage] Disable accounts-file type check before enabling tiered-storage #418

Merged

Conversation

yhchiang-sol commented Mar 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Summary of Changes

Uh oh!

yhchiang-sol commented Mar 5, 2024

Uh oh!

brooksprumo commented Mar 5, 2024

Uh oh!

codecov-commenter commented Mar 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yhchiang-sol commented Mar 22, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yhchiang-sol commented Mar 22, 2024

Uh oh!

yhchiang-sol commented Mar 22, 2024

Uh oh!

yhchiang-sol commented Mar 22, 2024

Uh oh!

Uh oh!

Uh oh!

yhchiang-sol commented Mar 23, 2024

Uh oh!

Uh oh!

Uh oh!

yhchiang-sol commented Mar 24, 2024

Uh oh!

brooksprumo left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

yhchiang-sol commented Mar 4, 2024 •

edited

Loading

codecov-commenter commented Mar 12, 2024 •

edited

Loading