-
Notifications
You must be signed in to change notification settings - Fork 298
SIMD-0106: Epoch Rewards Partition Data PDA #106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
18 commits
Select commit
Hold shift + click to select a range
96e3868
SIMD-0106: Partitioned Epoch Rewards PDA
6db8f2e
Add feature
648e0c9
Add pseudocode for PDA contents
08c280b
Cleanup
9b8fe4a
Compatibility/feature clarifications
CriesofCarrots 0b95f9e
Make layouts specific
0943ed0
Improve wording
94f8ec3
Remove HasherKind storage
d62c62d
Make address definition more definitive and propose new program id
e821757
Update title
1598980
Fix stale program
7725c6c
Make rewards-calculation timing more specific
094bd7e
Expand write-lock detail
047ccf7
Clean up
d9fe0a3
Hasher type in prose, since no longer stored in account
5a409eb
Move write-lock paragraph into design
86d9876
Clarify timing relative to SIMD 0015
32aee73
Syscall distinction
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| --- | ||
| simd: '0106' | ||
| title: Epoch Rewards Partition Data PDA | ||
| authors: | ||
| - Tyera Eulberg | ||
| category: Standard | ||
| type: Core | ||
| status: Draft | ||
| created: 2024-01-17 | ||
| feature: https://github.com/solana-labs/solana/issues/32166 | ||
| extends: 0015 | ||
| --- | ||
|
|
||
| ## Summary | ||
|
|
||
| Partitioned epoch rewards (SIMD 0015) will split epoch rewards distribution | ||
| across multiple blocks at the start of an epoch. This extension of that SIMD | ||
| proposes storing data about the rewards calculation and partitions in a sysvar | ||
| account at a program-derived address (PDA), which can be queried by clients, or | ||
| indeed within the runtime, to performantly identify the partition for the | ||
| distribution to any particular address. | ||
|
|
||
| ## Motivation | ||
|
|
||
| When we move to partitioned epoch rewards as described in the original SIMD, the | ||
| only way to find the stake or voting rewards for a specific address will be to | ||
| iterate through blocks at the beginning of each epoch, inspecting all the | ||
| rewards. This is because the runtime does not persist information about how the | ||
| rewards were partitioned; in fact, it does not even persist how many blocks the | ||
| rewards distribution spans, so there is no way to predict how long it will take | ||
| or how far through the epoch to go in finding a particular address. | ||
|
|
||
| ## Alternatives Considered | ||
|
|
||
| An alternative to storing partition data in an on-chain account would be to | ||
| record the necessary data in the ledger in some fashion. This could be a | ||
| transaction that gets added to the block, a special RewardType, or metadata that | ||
| gets stored on the node and then duplicated to long-term storage, like Bigtable. | ||
| In fact, it is probably worthwhile to pursue this alternative as well, since it | ||
| will enable finding rewards without access to snapshots or the running chain. | ||
|
|
||
| ## New Terminology | ||
|
|
||
| None | ||
|
|
||
| ## Detailed Design | ||
|
|
||
| When partitioned rewards populates the temporary `EpochRewards` sysvar -- | ||
| defined in SIMD 0015 as at the start of the first block of the epoch, before any | ||
| transactions are processed -- the runtime must populate a PDA that stores the | ||
| partition data needed to recreate the hasher that returns the partition index | ||
| for any address. The hasher for v0 partitioned rewards is SipHash 1-3. The data | ||
| needed comprises: the number of partitions and parent blockhash. More | ||
| specifically: | ||
|
|
||
| ```rust | ||
| // Version wrapper to allow future updates | ||
| // Variant serialized as little-endian unsigned 32-bit integer | ||
| enum EpochRewardsPartitionDataVersion { | ||
| V0(PartitionData), | ||
| } | ||
|
|
||
| // Data about rewards partitions for a particular epoch | ||
| struct PartitionData { | ||
| num_partitions: u64, // little-endian unsigned 64-bit integer | ||
| parent_blockhash: Hash, // byte-array of length 32 | ||
| } | ||
| ``` | ||
|
|
||
| The address of this PDA will use some bytes -- to prevent griefing and namespace | ||
| the PDAs -- and the rewards distribution epoch number as a little-endian u64 as | ||
| seeds. Specifically: `[b"EpochRewards",b"PartitionData", &epoch.to_le_bytes()]`. | ||
| The owning program will be the Sysvar program id: | ||
| `Sysvar1111111111111111111111111111111111111`. | ||
|
|
||
| The partition-data PDA will not be available to on-chain programs via syscall. | ||
|
|
||
| Like traditional sysvars, the partitioned-rewards data PDAs should only be | ||
| loadable as read-only. SIMD 0105 defines a method for demoting sysvar write | ||
| locks, but depends on a list of addresses in code. If write-lock handling of | ||
| dynamically addressed sysvars like these PDAs seems needed in the future, a new | ||
| proposal should be introduced. | ||
|
|
||
| ## Impact | ||
|
|
||
| The change in this proposal does increase the number of "forever" accounts that | ||
| validators must store by one per epoch. However, the PDAs will be owned by the | ||
| Sysvar program, so could be adjusted or closed in the future by a feature-gated | ||
| change to that program. Meanwhile, the change greatly improves the post-SIMD | ||
| 0015 situation for clients trying to track stake or voting rewards, since they | ||
| can use the data in the PDA to pull the correct partition directly, instead of | ||
| scanning an unknown number of blocks. | ||
|
|
||
| ## Security Considerations | ||
|
|
||
| None | ||
|
|
||
| ## Backwards Compatibility | ||
|
|
||
| Runtime-population of an account each epoch is a consensus-breaking change and | ||
| must be activated with a feature gate. Since this is an extension of SIMD 0015 | ||
| and that feature gate has not yet been activated on any public clusters, it | ||
| should be gated by the same feature id. | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you also include the motivation for including this data on-chain? I personally think that storing it in the first blocks metadata would be great. We already use the metadata to record the list of rewards.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added. But yes, planning to do both. Will write up a separate SIMD for the metadata approach.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I reverted this one as well. Here's what I had written:
"If the partition data were stored in an on-chain account, it would be simple and
low-compute-cost for a client (with a snapshot or running node) or the runtime
itself to find the rewards for a particular address for any historical epoch
after partitioned epoch rewards are activated."
I do feel like this is already articulated in the summary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, how open are you to removing the PDA approach and just using the metadata? I know it's already implemented so we're at least slightly committed to that approach, so I won't push back too hard. But I generally prefer we limit the number of ways accounts can be written outside of the SVM because it opens up security issues and increases the amount of code that other protocol implementations need to write.
Getting specific reward data for an account is already impossible afaik from the runtime so I don't see much value in making the partition data accessible in an account. Since reward data is only in metadata, I think it makes sense for the partition data to also only be in metadata.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Somewhat opposed. I think it makes sense to have some sort of persistent record of the partitioned-rewards data on chain, but it's true I don't know of a specific in-program use-case for it. From the rpc perspective, it's also really nice to be able to figure out the rewards block before having to determine what data store to use to find that block.
@t-nelson , perhaps you'd care to chime in here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that very different from the rpc fetching the first block in the epoch to find out where the rewards are? If I recall correctly, there's an extra call getBlocks to figure out what the first block actually is, is that the part you're trying to improve?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. There's potentially quite a bit of latency in each long-term storage query, so it would be nice to avoid the extra call.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the main motivation is to save some latency overhead on step 1 of fetching rewards when step 2 requires high latency big table fetching most of the time anyways, it doesn't seem like this proposal is worth the runtime complexity.