Skip to content
This repository was archived by the owner on Jan 22, 2025. It is now read-only.

Simple cap by including sysvars and native programs#13884

Merged
ryoqun merged 11 commits intosolana-labs:masterfrom
ryoqun:simpler-capitalization
Dec 14, 2020
Merged

Simple cap by including sysvars and native programs#13884
ryoqun merged 11 commits intosolana-labs:masterfrom
ryoqun:simpler-capitalization

Conversation

@ryoqun
Copy link
Copy Markdown
Contributor

@ryoqun ryoqun commented Dec 1, 2020

Problem

Simplify capitalization calculation by removing special casing.

Summary of Changes

Adjust cap while adding/updating sysvar and native program accounts correctly to simplify cap calculation.

Also, this fixes a regression bug and a new bug introduced at #13403. In this sense, this pr is a follow up to #13403, #11750. (EDIT: this was done by separate pr)

@ryoqun ryoqun added the noCI Suppress CI on this Pull Request label Dec 1, 2020
@ryoqun ryoqun force-pushed the simpler-capitalization branch from d7e608b to f11a8df Compare December 3, 2020 14:16
@ryoqun ryoqun added CI Pull Request is ready to enter CI v1.4 and removed noCI Suppress CI on this Pull Request labels Dec 3, 2020
@solana-grimes solana-grimes removed the CI Pull Request is ready to enter CI label Dec 3, 2020
Comment thread runtime/src/bank.rs Outdated
Comment on lines +4449 to +4450
let existing_sysvar_account_count = 9;
let existing_native_program_account_count = 7;
Copy link
Copy Markdown
Contributor Author

@ryoqun ryoqun Dec 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if I killed the cluster, this would mean I even fail the elementary math class, I guess... ;)

Comment thread runtime/src/bank.rs
// flush the Stakes cache
account.lamports = 0;
self.store_account(&program_id, &account);
None
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a test justifying this new behavior

Copy link
Copy Markdown
Contributor Author

@ryoqun ryoqun Dec 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improved existing: https://github.com/solana-labs/solana/pull/13884/files#r535949045
Also wrote a new test case specifically for this: (test_add_native_program_squated_while_not_replacing)

@codecov
Copy link
Copy Markdown

codecov Bot commented Dec 3, 2020

Codecov Report

Merging #13884 (4582de3) into master (52c2cbd) will increase coverage by 0.0%.
The diff coverage is 97.5%.

@@           Coverage Diff            @@
##           master   #13884    +/-   ##
========================================
  Coverage    82.2%    82.2%            
========================================
  Files         384      384            
  Lines       94803    94965   +162     
========================================
+ Hits        77948    78103   +155     
- Misses      16855    16862     +7     

Copy link
Copy Markdown
Contributor

@mvines mvines left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read through this draft, looks good

Comment thread core/src/validator.rs
std::process::exit(1);

#[cfg(test)]
panic!("process::exit(1) is intercepted for friendly test failure...");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice idea

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks! Ideally, I want to get rid of all process:exit(1)s in lib code... But, I don't have enough bandwidth; this is a tiny 10-ish lines of hacky work-around for the current situation...

Comment thread core/src/validator.rs
ip_echo_server: solana_net_utils::IpEchoServer,
}

fn abort() -> ! {
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my til: !! (never type)

Comment thread runtime/src/bank.rs Outdated
Comment thread runtime/src/bank.rs
Comment thread runtime/src/bank.rs
assert!(bank.stakes.read().unwrap().vote_accounts().is_empty());
assert!(bank.stakes.read().unwrap().stake_delegations().is_empty());
assert_eq!(bank.calculate_capitalization(), bank.capitalization());
assert_eq!(
Copy link
Copy Markdown
Contributor Author

@ryoqun ryoqun Dec 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these additional assertions to a test written at #11750 exposes a regression introduced at #13403.

Comment thread runtime/src/bank.rs
Comment thread runtime/src/bank.rs
Comment thread runtime/src/bank.rs
})
},
|old, new| {
// creating new sysvar twice in a slot shouldn't increment capitalization twice
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(repost)

Very subtle bug in the original draft pr was caught here.

Combined with new use of inherit_specially_retained_account_balance in this (now-parameterized) test, this new code block illustrates why it was a bad idea to update bank.capitalization in inherit_specially_retained_account_balance (this was my original draft pr changes)...

@ryoqun
Copy link
Copy Markdown
Contributor Author

ryoqun commented Dec 10, 2020

Status update: well greatly interrupted by other things, but I've finally finished almost all work locally. This pr is just waiting to be rebased on #14042 after its merge.

@ryoqun ryoqun force-pushed the simpler-capitalization branch from 92a31b0 to 2b5a307 Compare December 12, 2020 17:05
@ryoqun ryoqun marked this pull request as ready for review December 12, 2020 17:07
Comment thread runtime/src/bank.rs
if let Some(account) = self.get_account(&feature_set::simpler_capitalization::id()) {
if let Some(feature) = feature::from_account(&account) {
if feature.activated_at == Some(0) {
return true;
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is pretty safe, not having any possibility of falsely activating this feature on testnet/mainnet-beta.

@ryoqun ryoqun force-pushed the simpler-capitalization branch from e069e6f to b6aa451 Compare December 12, 2020 17:57
Comment thread runtime/src/bank.rs
let mut existing_native_program_account_count = 4;

if self.get_account(&sysvar::rewards::id()).is_some() {
existing_sysvar_account_count += 1;
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heh, casual looking of coverage revealed I'm missing to test this branch... ;) b6aa451

image

Comment thread runtime/src/bank.rs
}
// updating sysvars (the fees sysvar in this case) now depends on feature activations in
// genesis_config.accounts above
self.update_fees();
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was a hard to debug thing... ;p

@ryoqun ryoqun requested a review from mvines December 12, 2020 18:44
@ryoqun
Copy link
Copy Markdown
Contributor Author

ryoqun commented Dec 12, 2020

@mvines hooray! i needed various supllemental prs for this (see referenced links...). But finally this pr is finally ready for serious review. I think this pr is pretty solid now compared from 9 days ago #13884 (review)

mvines
mvines previously approved these changes Dec 14, 2020
Copy link
Copy Markdown
Contributor

@mvines mvines left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. I think I'd prefer simpler_capitalization be called simple_capitalization as a global search and replace.

@ryoqun ryoqun changed the title Simpler cap by including sysvars and native programs Simple cap by including sysvars and native programs Dec 14, 2020
@mergify mergify Bot dismissed mvines’s stale review December 14, 2020 05:54

Pull request has been modified.

@ryoqun
Copy link
Copy Markdown
Contributor Author

ryoqun commented Dec 14, 2020

Looks good to me. I think I'd prefer simpler_capitalization be called simple_capitalization as a global search and replace.

Thanks for the review! Yeah, I've renamed: 31a3cd6

@ryoqun ryoqun merged commit de9ac43 into solana-labs:master Dec 14, 2020
@ryoqun
Copy link
Copy Markdown
Contributor Author

ryoqun commented Dec 14, 2020

After long and extensive testing, I finally merged this. :)

mergify Bot pushed a commit that referenced this pull request Dec 14, 2020
* Simpler cap by including sysvars and native programs

* Fix tests

* Add comment

* revert some unrelated code

* Update test_bank_update_sysvar_account for cap.

* Test cap. for add_native_program using new helper

* Improve the cap adjustment with new tests

* Fix typo...

* Adjust test for improved code coverage

* Rename simpler_capitalization => simple_capitalization

* More rename and bonus commenting

(cherry picked from commit de9ac43)
mergify Bot added a commit that referenced this pull request Dec 14, 2020
* Simpler cap by including sysvars and native programs

* Fix tests

* Add comment

* revert some unrelated code

* Update test_bank_update_sysvar_account for cap.

* Test cap. for add_native_program using new helper

* Improve the cap adjustment with new tests

* Fix typo...

* Adjust test for improved code coverage

* Rename simpler_capitalization => simple_capitalization

* More rename and bonus commenting

(cherry picked from commit de9ac43)

Co-authored-by: Ryo Onodera <ryoqun@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants