refactor(metadata): migrate feature states to static metadata [part 5/12]

[glevco]

Depends on #1017

---

Motivation

Continuing with the static metadata refactor introduced in PRs #1013 and #1014, the next step is to move the feature_states attribute from metadata to static metadata. This PR implements this, finishing the static metadata refactor.

This results in very beneficial changes for Feature Activation, because now feature states are only calculated once when a block is received and then persisted in static metadata, meaning we don't have to deal with complications introduced by mutable states.

For instance, in order to get the state of a feature for a block before this PR, we would need to call feature_service.get_state() and it would either get it from the cache (in metadata) or recalculate it. Now, since an attribute is permanently available in static_metadata, retrieving feature states becomes as simple as accessing a property, which is guaranteed to always exist, incurring no performance penalties and preventing the need to inject the FeatureService singleton everywhere. In fact, FeatureService is no longer a singleton, and instead just a pure "calculator" (we may change its name in a separate PR).

This allowed us to simplify and clean up a lot of related code, and also naturally removed the FeatureService dependency from block verification — which was the final goal of the PRs in this sequence up to this point, for the Multiprocess Verification project.

Another great earned benefit is, now that feature_states are always guaranteed to be calculated for all blocks, we can change the algorithm to use parent blocks to retrieve the previous state, instead of previous boundary blocks. This also allowed us to remove the get_ancestor_at_height() method.

Acceptance Criteria

• Move the feature_states attribute from metadata to static metadata.
• Move some FeatureService convenience methods to static metadata.
• Remove FeatureService singleton as it's no longer needed and update usages.
• Update FeatureService so it no longer relies on stateful metadata, and instead becomes a pure function
• Optimize calculate_feature_state() by using parent blocks instead of previous boundary blocks.
• Remove feature activation related methods from Block.
• Update BitSignalingService to automatically enable support during MUST_SIGNAL via PubSub, instead of via FeatureService.
• Implement a migration to convert metadata into static metadata.
• Update tests accordingly.
• ATTENTION: This refactor means we now store feature states for all blocks, and not just boundary blocks. I checked it and after this migration, the database went from 8.2 to 8.3 GB. Still, there are known ways we could optimize metadata storage in future PRs.

Checklist

• If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged

[codecov]

Codecov Report

Attention: Patch coverage is 85.71429% with 16 lines in your changes missing coverage. Please review.

Project coverage is 84.44%. Comparing base (335d050) to head (db94b36).

Files with missing lines	Patch %	Lines
...ction/storage/migrations/migrate_feature_states.py	50.00%	14 Missing ⚠️
...on/storage/migrations/remove_first_nop_features.py	0.00%	1 Missing ⚠️
...n/storage/migrations/remove_second_nop_features.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1018      +/-   ##
==========================================
+ Coverage   84.42%   84.44%   +0.01%     
==========================================
  Files         317      318       +1     
  Lines       24303    24237      -66     
  Branches     3698     3687      -11     
==========================================
- Hits        20519    20467      -52     
+ Misses       3070     3050      -20     
- Partials      714      720       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

Bencher Report

Branch	refactor/metadata/migrate-feature-states-5
Testbed	ubuntu-22.04

Click to view all benchmark results

Benchmark	Latency	Benchmark Result nanoseconds (ns) (Result Δ%)	Lower Boundary nanoseconds (ns) (Limit %)	Upper Boundary nanoseconds (ns) (Limit %)
sync-v2 (up to 20000 blocks)	📈 view plot 🚷 view threshold	101,462,798,848.64 (-0.48%)	91,760,403,941.47 (90.44%)	112,151,604,817.35 (90.47%)

🐰 View full continuous benchmarking report in Bencher

[glevco]

I'm putting this PR on hold for now, and instead PR #1146 will revert related changes, keeping feature states in "normal" metadata, and not moving it to static metadata. See the new PR's description for motivation.