Skip to content

chore(sequencer)!: update storage keys locations and values (ENG-898)#1616

Merged
Fraser999 merged 8 commits intomainfrom
fraser/update-storage-keys
Oct 9, 2024
Merged

chore(sequencer)!: update storage keys locations and values (ENG-898)#1616
Fraser999 merged 8 commits intomainfrom
fraser/update-storage-keys

Conversation

@Fraser999
Copy link
Contributor

Summary

The storage keys for all components were moved from the top of state_ext modules into new storage::keys modules. The key values were updated to a consistent style.

Background

We wanted the keys to be easy to find and to follow a consistent pattern in terms of their values and locations.

Changes

  • All key consts and functions were moved to <COMPONENT>/storage/keys.
  • All keys were given a prefix matching the component name followed by a /.
  • All keys now use full words (except for a few instances of well-know abbreviations like "init") and _ as word separators.
  • Existing helper structs in various state_ext modules for formatting keys were unified into a new struct AddressPrefixer in crate::storage.
  • The Asset functionality was moved from crate::storage_keys to crate::storage.

Testing

  • All keys have snapshot tests (including const keys - this might be considered overkill?)
  • All keys have tests ensuring they have a component name as prefix.
  • All prefix consts or functions which are used in state_ext modules are tested to ensure they are actually prefixes of the given keys.

Breaking Changelist

  • This breaks the on-disk format of both the verifiable and non-verifiable store.

Related Issues

Closes #1611.

@Fraser999 Fraser999 requested review from a team, SuperFluffy, joroshiba and noot as code owners October 2, 2024 22:49
@github-actions github-actions bot added the sequencer pertaining to the astria-sequencer crate label Oct 2, 2024
Copy link
Contributor

@SuperFluffy SuperFluffy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, thank you for going through all the keys and streamlining them.

I very much like that we have full (I think?) coverage of all keys and prefixes.

Since this is a breaking change and a rework of how our fees are structured, I think we should break with our previous key and make their types and (where applicable) byte-encoding consistent everywhere.

Some open points that I think should be addressed before merge:

  1. make the type of all keys consistently &str/String. Right now we have a string representation in some places, and raw byte slices in others. It's not clear why that's done. Yet especially for snapshot tests it's very valuable to have a human readable representation.
  2. Apart from ibc-prefixed assets (which are "ibc/<hex-encoded-32-bytes>", all key/prefix parts that involve raw bytes (like account) should be base64 encoded to save space.
  3. I don't have a strong opinion on only using the byte part of ibc-prefixed assets when constructing keys from them. I mildly lean toward encoding the full string (including "ibc/") and not just the bytes.

Copy link
Contributor

@SuperFluffy SuperFluffy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thank you.

I wonder if in this PR we should just change base64 encoding to the url safe alphabet everywhere (so that we don't have a different encoding in the storage keys and the normal display impl).

@Fraser999 Fraser999 enabled auto-merge October 9, 2024 09:49
@Fraser999 Fraser999 added this pull request to the merge queue Oct 9, 2024
Merged via the queue into main with commit bb6c435 Oct 9, 2024
@Fraser999 Fraser999 deleted the fraser/update-storage-keys branch October 9, 2024 10:03
@Fraser999
Copy link
Contributor Author

I wonder if in this PR we should just change base64 encoding to the url safe alphabet everywhere (so that we don't have a different encoding in the storage keys and the normal display impl).

Ticketed for follow-up at #1649.

steezeburger added a commit that referenced this pull request Oct 11, 2024
* main:
  feat(sequencer)!: enforce block ordering by transaction group  (#1618)
  fix(cli): ensure checkTx passes before waiting for inclusion (#1636)
  chore(sequencer)!: update storage keys locations and values (ENG-898) (#1616)
  chore: cargo audit warning (#1644)
  chore(proto, core)!: remove action suffix from all action types (#1630)
  feat(sequencer)!: add limit to total amount of transactions in parked  (#1638)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sequencer pertaining to the astria-sequencer crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update storage keys in sequencer

2 participants