Skip to content
This repository was archived by the owner on Jan 22, 2025. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions geyser-plugin-interface/src/geyser_plugin_interface.rs
Original file line number Diff line number Diff line change
Expand Up @@ -71,13 +71,47 @@ pub struct ReplicaAccountInfoV2<'a> {
pub txn_signature: Option<&'a Signature>,
}

#[derive(Debug, Clone, PartialEq, Eq)]
/// Information about an account being updated
/// (extended with reference to transaction doing this update)
pub struct ReplicaAccountInfoV3<'a> {
/// The Pubkey for the account
pub pubkey: &'a [u8],

/// The lamports for the account
pub lamports: u64,

/// The Pubkey of the owner program account
pub owner: &'a [u8],

/// This account's data contains a loaded program (and is now read-only)
pub executable: bool,

/// The epoch at which this account will next owe rent
pub rent_epoch: u64,

/// The data held in this account.
pub data: &'a [u8],

/// A global monotonically increasing atomic number, which can be used
/// to tell the order of the account update. For example, when an
/// account is updated in the same slot multiple times, the update
/// with higher write_version should supersede the one with lower
/// write_version.
pub write_version: u64,

/// Reference to transaction causing this account modification
pub txn: Option<&'a SanitizedTransaction>,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can SanitizedTransaction change under our feet and resulting in missing a version update?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how can it be possible. update_account notification happens syncronously after transaction processing in a single thread, reference is immutable. If such change is possible then it definitely happens regardless plugin behaviour.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant update in different validator versions. I don't have a good solution either -- duplicating the object is an overkill and will likely impact performance. In the end the plugin has to compile with the validator and test to make sure it works.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But there's no cloning here. It all depends on behaviour of plugin. In my implementation of plugin there's no cloning at all. This field is used to proceed filtration at the very beginning of account_update handler and not passed any further.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plugin than pass only signature of transaction to account table. And it will be enough to match accounts and transactions in different tables in database

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you are talking about compatibility between different versions of validator (in case when SenitizedTransaction changed internally) then of course you should always have plugin built against corresponding validator libraries. But it is the same as for TransactionReplicaInfo - we have the same reference to SanitizedTransaction inside. Of course better way for compatibility is to have special structure which clones internals of transaction. But this approach is very inefficient in terms of performance. So I think this implementation of ReplicaAccountInfo doent not bring any additional drawbacks other than already exists in plugin engine design.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we faced such issues when moving from 1.14.10 to 1.14.12 (if i'm not mistaken with numbers) - SanitizedTransaction changed it's message field type.

}

/// A wrapper to future-proof ReplicaAccountInfo handling.
/// If there were a change to the structure of ReplicaAccountInfo,
/// there would be new enum entry for the newer version, forcing
/// plugin implementations to handle the change.
pub enum ReplicaAccountInfoVersions<'a> {
V0_0_1(&'a ReplicaAccountInfo<'a>),
V0_0_2(&'a ReplicaAccountInfoV2<'a>),
V0_0_3(&'a ReplicaAccountInfoV3<'a>),
}

/// Information about a transaction
Expand Down
26 changes: 13 additions & 13 deletions geyser-plugin-manager/src/accounts_update_notifier.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ use {
crate::geyser_plugin_manager::GeyserPluginManager,
log::*,
solana_geyser_plugin_interface::geyser_plugin_interface::{
ReplicaAccountInfoV2, ReplicaAccountInfoVersions,
ReplicaAccountInfoV3, ReplicaAccountInfoVersions,
},
solana_measure::measure::Measure,
solana_metrics::*,
Expand All @@ -15,7 +15,7 @@ use {
account::{AccountSharedData, ReadableAccount},
clock::Slot,
pubkey::Pubkey,
signature::Signature,
transaction::SanitizedTransaction,
},
std::sync::{Arc, RwLock},
};
Expand All @@ -29,12 +29,12 @@ impl AccountsUpdateNotifierInterface for AccountsUpdateNotifierImpl {
&self,
slot: Slot,
account: &AccountSharedData,
txn_signature: &Option<&Signature>,
txn: &Option<&SanitizedTransaction>,
pubkey: &Pubkey,
write_version: u64,
) {
if let Some(account_info) =
self.accountinfo_from_shared_account_data(account, txn_signature, pubkey, write_version)
self.accountinfo_from_shared_account_data(account, txn, pubkey, write_version)
{
self.notify_plugins_of_account_update(account_info, slot, false);
}
Expand Down Expand Up @@ -107,41 +107,41 @@ impl AccountsUpdateNotifierImpl {
fn accountinfo_from_shared_account_data<'a>(
&self,
account: &'a AccountSharedData,
txn_signature: &'a Option<&'a Signature>,
txn: &'a Option<&'a SanitizedTransaction>,
pubkey: &'a Pubkey,
write_version: u64,
) -> Option<ReplicaAccountInfoV2<'a>> {
Some(ReplicaAccountInfoV2 {
) -> Option<ReplicaAccountInfoV3<'a>> {
Some(ReplicaAccountInfoV3 {
pubkey: pubkey.as_ref(),
lamports: account.lamports(),
owner: account.owner().as_ref(),
executable: account.executable(),
rent_epoch: account.rent_epoch(),
data: account.data(),
write_version,
txn_signature: *txn_signature,
txn: *txn,
})
}

fn accountinfo_from_stored_account_meta<'a>(
&self,
stored_account_meta: &'a StoredAccountMeta,
) -> Option<ReplicaAccountInfoV2<'a>> {
Some(ReplicaAccountInfoV2 {
) -> Option<ReplicaAccountInfoV3<'a>> {
Some(ReplicaAccountInfoV3 {
pubkey: stored_account_meta.pubkey().as_ref(),
lamports: stored_account_meta.account_meta.lamports,
owner: stored_account_meta.account_meta.owner.as_ref(),
executable: stored_account_meta.account_meta.executable,
rent_epoch: stored_account_meta.account_meta.rent_epoch,
data: stored_account_meta.data,
write_version: stored_account_meta.meta.write_version_obsolete,
txn_signature: None,
txn: None,
})
}

fn notify_plugins_of_account_update(
&self,
account: ReplicaAccountInfoV2,
account: ReplicaAccountInfoV3,
slot: Slot,
is_startup: bool,
) {
Expand All @@ -154,7 +154,7 @@ impl AccountsUpdateNotifierImpl {
for plugin in plugin_manager.plugins.iter_mut() {
let mut measure = Measure::start("geyser-plugin-update-account");
match plugin.update_account(
ReplicaAccountInfoVersions::V0_0_2(&account),
ReplicaAccountInfoVersions::V0_0_3(&account),
slot,
is_startup,
) {
Expand Down
29 changes: 11 additions & 18 deletions runtime/src/accounts.rs
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ use {
},
pubkey::Pubkey,
saturating_add_assign,
signature::Signature,
slot_hashes::SlotHashes,
system_program,
sysvar::{self, epoch_schedule::EpochSchedule, instructions::construct_instructions_data},
Expand Down Expand Up @@ -1203,7 +1202,7 @@ impl Accounts {
lamports_per_signature: u64,
include_slot_in_hash: IncludeSlotInHash,
) {
let (accounts_to_store, txn_signatures) = self.collect_accounts_to_store(
let (accounts_to_store, transactions) = self.collect_accounts_to_store(
txs,
res,
loaded,
Expand All @@ -1213,7 +1212,7 @@ impl Accounts {
);
self.accounts_db.store_cached(
(slot, &accounts_to_store[..], include_slot_in_hash),
Some(&txn_signatures),
Some(&transactions),
);
}

Expand All @@ -1240,10 +1239,10 @@ impl Accounts {
lamports_per_signature: u64,
) -> (
Vec<(&'a Pubkey, &'a AccountSharedData)>,
Vec<Option<&'a Signature>>,
Vec<Option<&'a SanitizedTransaction>>,
) {
let mut accounts = Vec::with_capacity(load_results.len());
let mut signatures = Vec::with_capacity(load_results.len());
let mut transactions = Vec::with_capacity(load_results.len());
for (i, ((tx_load_result, nonce), tx)) in load_results.iter_mut().zip(txs).enumerate() {
if tx_load_result.is_err() {
// Don't store any accounts if tx failed to load
Expand Down Expand Up @@ -1293,12 +1292,12 @@ impl Accounts {
if execution_status.is_ok() || is_nonce_account || is_fee_payer {
// Add to the accounts to store
accounts.push((&*address, &*account));
signatures.push(Some(tx.signature()));
transactions.push(Some(tx));
}
}
}
}
(accounts, signatures)
(accounts, transactions)
}
}

Expand Down Expand Up @@ -2874,7 +2873,6 @@ mod tests {
(message.account_keys[1], account2.clone()),
];
let tx0 = new_sanitized_tx(&[&keypair0], message, Hash::default());
let tx0_sign = *tx0.signature();

let instructions = vec![CompiledInstruction::new(2, &(), vec![0, 1])];
let message = Message::new_with_compiled_instructions(
Expand All @@ -2890,7 +2888,6 @@ mod tests {
(message.account_keys[1], account2),
];
let tx1 = new_sanitized_tx(&[&keypair1], message, Hash::default());
let tx1_sign = *tx1.signature();

let loaded0 = (
Ok(LoadedTransaction {
Expand Down Expand Up @@ -2927,9 +2924,9 @@ mod tests {
.unwrap()
.insert_new_readonly(&pubkey);
}
let txs = vec![tx0, tx1];
let txs = vec![tx0.clone(), tx1.clone()];
let execution_results = vec![new_execution_result(Ok(()), None); 2];
let (collected_accounts, txn_signatures) = accounts.collect_accounts_to_store(
let (collected_accounts, transactions) = accounts.collect_accounts_to_store(
&txs,
&execution_results,
loaded.as_mut_slice(),
Expand All @@ -2945,13 +2942,9 @@ mod tests {
.iter()
.any(|(pubkey, _account)| *pubkey == &keypair1.pubkey()));

assert_eq!(txn_signatures.len(), 2);
assert!(txn_signatures
.iter()
.any(|signature| signature.unwrap().to_string().eq(&tx0_sign.to_string())));
assert!(txn_signatures
.iter()
.any(|signature| signature.unwrap().to_string().eq(&tx1_sign.to_string())));
assert_eq!(transactions.len(), 2);
assert!(transactions.iter().any(|txn| txn.unwrap().eq(&tx0)));
assert!(transactions.iter().any(|txn| txn.unwrap().eq(&tx1)));

// Ensure readonly_lock reflects lock
assert_eq!(
Expand Down
46 changes: 20 additions & 26 deletions runtime/src/accounts_db.rs
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,8 @@ use {
pubkey::Pubkey,
rent::Rent,
saturating_add_assign,
signature::Signature,
timing::AtomicInterval,
transaction::SanitizedTransaction,
},
std::{
borrow::{Borrow, Cow},
Expand Down Expand Up @@ -6571,16 +6571,16 @@ impl AccountsDb {
&self,
slot: Slot,
accounts_and_meta_to_store: &impl StorableAccounts<'b, T>,
txn_signatures_iter: Box<dyn std::iter::Iterator<Item = &Option<&Signature>> + 'a>,
txn_iter: Box<dyn std::iter::Iterator<Item = &Option<&SanitizedTransaction>> + 'a>,
include_slot_in_hash: IncludeSlotInHash,
mut write_version_producer: P,
) -> Vec<AccountInfo>
where
P: Iterator<Item = u64>,
{
txn_signatures_iter
txn_iter
.enumerate()
.map(|(i, signature)| {
.map(|(i, txn)| {
let account = accounts_and_meta_to_store
.account_default_if_zero_lamport(i)
.map(|account| account.to_account_shared_data())
Expand All @@ -6594,7 +6594,7 @@ impl AccountsDb {
self.notify_account_at_accounts_update(
slot,
&account,
signature,
txn,
accounts_and_meta_to_store.pubkey(i),
&mut write_version_producer,
);
Expand Down Expand Up @@ -6630,7 +6630,7 @@ impl AccountsDb {
hashes: Option<Vec<impl Borrow<Hash>>>,
mut write_version_producer: P,
store_to: &StoreTo,
txn_signatures: Option<&[Option<&'a Signature>]>,
transactions: Option<&[Option<&'a SanitizedTransaction>]>,
) -> Vec<AccountInfo> {
let mut calc_stored_meta_time = Measure::start("calc_stored_meta");
let slot = accounts.target_slot();
Expand All @@ -6645,19 +6645,19 @@ impl AccountsDb {

match store_to {
StoreTo::Cache => {
let signature_iter: Box<dyn std::iter::Iterator<Item = &Option<&Signature>>> =
match txn_signatures {
Some(txn_signatures) => {
assert_eq!(txn_signatures.len(), accounts.len());
Box::new(txn_signatures.iter())
let txn_iter: Box<dyn std::iter::Iterator<Item = &Option<&SanitizedTransaction>>> =
match transactions {
Some(transactions) => {
assert_eq!(transactions.len(), accounts.len());
Box::new(transactions.iter())
}
None => Box::new(std::iter::repeat(&None).take(accounts.len())),
};

self.write_accounts_to_cache(
slot,
accounts,
signature_iter,
txn_iter,
accounts.include_slot_in_hash(),
write_version_producer,
)
Expand Down Expand Up @@ -8161,12 +8161,12 @@ impl AccountsDb {
pub fn store_cached<'a, T: ReadableAccount + Sync + ZeroLamport + 'a>(
&self,
accounts: impl StorableAccounts<'a, T>,
txn_signatures: Option<&'a [Option<&'a Signature>]>,
transactions: Option<&'a [Option<&'a SanitizedTransaction>]>,
) {
self.store(
accounts,
&StoreTo::Cache,
txn_signatures,
transactions,
StoreReclaims::Default,
);
}
Expand All @@ -8187,7 +8187,7 @@ impl AccountsDb {
&self,
accounts: impl StorableAccounts<'a, T>,
store_to: &StoreTo,
txn_signatures: Option<&'a [Option<&'a Signature>]>,
transactions: Option<&'a [Option<&'a SanitizedTransaction>]>,
reclaim: StoreReclaims,
) {
// If all transactions in a batch are errored,
Expand Down Expand Up @@ -8219,13 +8219,7 @@ impl AccountsDb {
}

// we use default hashes for now since the same account may be stored to the cache multiple times
self.store_accounts_unfrozen(
accounts,
None::<Vec<Hash>>,
store_to,
txn_signatures,
reclaim,
);
self.store_accounts_unfrozen(accounts, None::<Vec<Hash>>, store_to, transactions, reclaim);
self.report_store_timings();
}

Expand Down Expand Up @@ -8352,7 +8346,7 @@ impl AccountsDb {
accounts: impl StorableAccounts<'a, T>,
hashes: Option<Vec<impl Borrow<Hash>>>,
store_to: &StoreTo,
txn_signatures: Option<&'a [Option<&'a Signature>]>,
transactions: Option<&'a [Option<&'a SanitizedTransaction>]>,
reclaim: StoreReclaims,
) {
// This path comes from a store to a non-frozen slot.
Expand All @@ -8369,7 +8363,7 @@ impl AccountsDb {
None::<Box<dyn Iterator<Item = u64>>>,
store_to,
reset_accounts,
txn_signatures,
transactions,
reclaim,
);
}
Expand Down Expand Up @@ -8404,7 +8398,7 @@ impl AccountsDb {
write_version_producer: Option<Box<dyn Iterator<Item = u64>>>,
store_to: &StoreTo,
reset_accounts: bool,
txn_signatures: Option<&[Option<&Signature>]>,
transactions: Option<&[Option<&SanitizedTransaction>]>,
reclaim: StoreReclaims,
) -> StoreAccountsTiming {
let write_version_producer: Box<dyn Iterator<Item = u64>> = write_version_producer
Expand All @@ -8426,7 +8420,7 @@ impl AccountsDb {
hashes,
write_version_producer,
store_to,
txn_signatures,
transactions,
);
store_accounts_time.stop();
self.stats
Expand Down
Loading