Skip to content
This repository was archived by the owner on Jan 22, 2025. It is now read-only.
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion runtime/src/accounts_cache.rs
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ impl SlotCacheInner {
pub fn get_cloned(&self, pubkey: &Pubkey) -> Option<CachedAccount> {
self.cache
.get(pubkey)
// Could return AccountNoData here
// 1) Maybe can eventually use a Cow to avoid a clone on every read
// 2) Popping is only safe if its guaranteed only replay/banking threads
// are reading from the AccountsDb
Expand Down Expand Up @@ -86,7 +87,7 @@ impl Deref for SlotCacheInner {

#[derive(Debug, Clone)]
pub struct CachedAccount {
pub account: Account,
pub account: Account, // would become AccountNoData
pub hash: Hash,
}

Expand Down
3 changes: 2 additions & 1 deletion runtime/src/accounts_db.rs
Original file line number Diff line number Diff line change
Expand Up @@ -276,6 +276,7 @@ impl<'a> LoadedAccount<'a> {
}
}

/* would become AccountNoData */
pub fn account(self) -> Account {
match self {
LoadedAccount::Stored(stored_account_meta) => stored_account_meta.clone_account(),
Expand Down Expand Up @@ -2195,7 +2196,7 @@ impl AccountsDb {
bank_hashes.insert(slot, new_hash_info);
}

pub fn load(&self, ancestors: &Ancestors, pubkey: &Pubkey) -> Option<(Account, Slot)> {
pub fn load(&self, ancestors: &Ancestors, pubkey: &Pubkey) -> Option<(Account /* would become AccountNoData */, Slot)> {
self.do_load(ancestors, pubkey, None)
}

Expand Down
6 changes: 3 additions & 3 deletions runtime/src/bank.rs
Original file line number Diff line number Diff line change
Expand Up @@ -111,9 +111,9 @@ impl ExecuteTimings {
type BankStatusCache = StatusCache<Result<()>>;
#[frozen_abi(digest = "3ZaEt781qwhfQSE4DZPBHhng2S6MuimchRjkR9ZWzDFs")]
pub type BankSlotDelta = SlotDelta<Result<()>>;
type TransactionAccountRefCells = Vec<Rc<RefCell<Account>>>;
type TransactionAccountDepRefCells = Vec<(Pubkey, RefCell<Account>)>;
type TransactionLoaderRefCells = Vec<Vec<(Pubkey, RefCell<Account>)>>;
type TransactionAccountRefCells = Vec<Rc<RefCell<Account>>>; /* would become AccountNoData */
type TransactionAccountDepRefCells = Vec<(Pubkey, RefCell<Account>)>; /* would become AccountNoData */
type TransactionLoaderRefCells = Vec<Vec<(Pubkey, RefCell<Account>)>>; /* would become AccountNoData */

// Eager rent collection repeats in cyclic manner.
// Each cycle is composed of <partiion_count> number of tiny pubkey subranges
Expand Down
2 changes: 1 addition & 1 deletion runtime/src/message_processor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ impl Executors {
pub struct PreAccount {
key: Pubkey,
is_writable: bool,
account: RefCell<Account>,
account: RefCell<Account>, /* would become AccountNoData */
}
impl PreAccount {
pub fn new(key: &Pubkey, account: &Account, is_writable: bool) -> Self {
Expand Down
32 changes: 31 additions & 1 deletion sdk/src/account.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::{clock::Epoch, pubkey::Pubkey};
use solana_program::{account_info::AccountInfo, sysvar::Sysvar};
use std::{cell::RefCell, cmp, fmt, rc::Rc};
use std::{cell::RefCell, cmp, fmt, rc::Rc, sync::Arc};

/// An Account with data that is stored on chain
#[repr(C)]
Expand All @@ -21,6 +21,36 @@ pub struct Account {
pub rent_epoch: Epoch,
}

#[derive(Clone, Default, Debug, PartialEq, Eq, AbiExample)]
pub struct AccountNoData {
/// lamports in the account
pub lamports: u64,
/// data held in this account
pub data: Arc<Vec<u8>>,
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sakridge , @carllin
I've been working on replay timing. We copy data around a lot. This is an attempt to greatly reduce copying data. I've prototyped this all out in a separate branch. Overall, I measure we're saving 30% of the time for a ledger verify of a recent mainnet-beta snapshot with a handful of changes that will each be discussed in their own pull request. This is one of the larger ones. It is intended to be consistent with comments in the code that we should return a ref or a cow from the cache. It looks like it needs to be an Arc vs Rc because we move accounts between threads frequently and Rc is no good. It looks like it needs to be an Arc vs Cow because of lifetime issues. I worked a change almost all the way through that attempted to add the right lifetimes to everything from the cache out and eventually ran into a few problems that looked fundamental. Perhaps this lifetime approach can work at some point.

Note this change doesn't 'work' all the way yet. In my prototype implementation, I made a lot of functions generic on the AnAccount trait. That allows Account and AccountNoData to coexist. It looks like they need to coexist because Account has to remain as is because it is an ABI? This is just a guess. If so, the internals of all our code would likely switch to generating and using AccountNoData instances, and we'd have to convert when we go to some api that requires an Account. This isn't different than cloning an Account today. This change is meant to give us a specific thing to discuss.

Some alternatives include the whole account being behind an Arc or the account being in an Arc AND the data being in an Arc. Updating the data and copying it gets quite expensive. Arc seems to give us the copy on write behavior I was imagining we'd want. Arc also frees us from the lifetime issues of Cow.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ryoqun Carl says you've worked on something similar.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't worry about my prior work around here. I was just experimenting like you. And nothing concrete.. I'm constantly interrupted by other things... xD
I'll look in depth this later.
@Lichtso This pr might be interesting because you're also working on similar area. ie: #15410

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Lichtso and I have communicated some on this topic. I am happy to collaborate more. The lifetime and copies of accounts with data from account load and caching to locking, serialization, vms, program execution, deserialization, verification, rollbacks, temporary storage, permanent storage, etc. are all in a circle of life. Sounds fun!

/// the program that owns this account. If executable, the program that loads this account.
pub owner: Pubkey,
/// this account's data contains a loaded program (and is now read-only)
pub executable: bool,
/// the epoch at which this account will next owe rent
pub rent_epoch: Epoch,
}

pub trait AnAccount: Default + Clone + Sized {
fn lamports(&self) -> u64;
fn set_lamports(&mut self, lamports: u64);
fn data(&self) -> &Vec<u8>;
fn set_data(&mut self, data: Vec<u8>);
fn owner(&self) -> &Pubkey;
fn set_owner(&mut self, owner: Pubkey);
fn executable(&self) -> bool;
fn rent_epoch(&self) -> Epoch;
fn set_rent_epoch(&mut self, epoch: Epoch);
fn clone_as_account_no_data(&self) -> AccountNoData;
fn clone_as_account(&self) -> Account;
fn from_account_no_data(item: AccountNoData) -> Self;
fn to_account_no_data(&mut self) -> AccountNoData;
}

impl fmt::Debug for Account {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
let data_len = cmp::min(64, self.data.len());
Expand Down
2 changes: 1 addition & 1 deletion sdk/src/keyed_account.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ pub struct KeyedAccount<'a> {
is_signer: bool, // Transaction was signed by this account's key
is_writable: bool,
key: &'a Pubkey,
pub account: &'a RefCell<Account>,
pub account: &'a RefCell<Account>, /* would become AccountNoData */
}

impl<'a> KeyedAccount<'a> {
Expand Down