Skip to content

chore: improve pubsub#694

Merged
msbrogli merged 1 commit intomasterfrom
chore/improve-pubsub
Aug 29, 2023
Merged

chore: improve pubsub#694
msbrogli merged 1 commit intomasterfrom
chore/improve-pubsub

Conversation

@glevco
Copy link
Contributor

@glevco glevco commented Jul 11, 2023

Motivation

This PR resolves #552. There are 3 goals:

  1. Improve typing of code using Twisted reactors
  2. Change tests to use a MemoryReactorClock instead of a Clock, so it's compatible with the reactor used in production (that is, it implements the IReactorCore, IReactorTime, and IReactorTCP Twisted interfaces. Clock only implements IReactorTime)
  3. Remove PubSub support for reactors that don't implement IReactorCore

This was done by leveraging Python protocols to represent an intersection of Twisted interfaces, which allows us to better represent the reactor type. By doing this, we were able to change tests to use a reactor that is more similar to the one used in production, as previously we were using a Clock, that behaved differently than the production reactor during full node initialization. This also allows for simplifying PubSub, as it had a specifc implementation for dealing with Clock.

Acceptance Criteria

  • Create new ReactorCoreProtocol, ReactorTimeProtocol, and ReactorTCPProtocol classes stubbing the respective Twisted interfaces, IReactorCore, IReactorTime, and IReactorTCP
  • Create new ReactorProtocol that implements the 3 protocols above, effectively creating a type that represents an intersection of Twisted interfaces, and cast the Twisted reactor to that protocol (this resolves a XXX comment in hathor/util.py)
  • Update PubSub to improve its code, also removing support for "non-running" reactors (Clock)
  • Remove the ReactorThread enum as it's not used anymore
  • Change the Clock to MemoryReactorClock in tests (this resolves a XXX comment)
  • Change HeapClock to MemoryReactorHeapClock on simulator
  • Improve verified_cast() utils function and move it to its own file

Checklist

  • If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged

@glevco glevco self-assigned this Jul 11, 2023
@codecov
Copy link

codecov bot commented Jul 11, 2023

Codecov Report

Merging #694 (aa53857) into master (29d9ec3) will decrease coverage by 0.10%.
Report is 4 commits behind head on master.
The diff coverage is 95.72%.

❗ Current head aa53857 differs from pull request most recent head 3561317. Consider uploading reports for the commit 3561317 to get more accurate results

@@            Coverage Diff             @@
##           master     #694      +/-   ##
==========================================
- Coverage   84.70%   84.60%   -0.10%     
==========================================
  Files         252      258       +6     
  Lines       21801    21783      -18     
  Branches     2954     2943      -11     
==========================================
- Hits        18467    18430      -37     
- Misses       2692     2715      +23     
+ Partials      642      638       -4     
Files Changed Coverage Δ
hathor/debug_resources.py 58.41% <33.33%> (-0.35%) ⬇️
hathor/utils/zope.py 93.75% <93.75%> (ø)
hathor/builder/cli_builder.py 73.82% <100.00%> (-0.67%) ⬇️
hathor/p2p/sync_v1/agent.py 85.07% <100.00%> (+0.06%) ⬆️
hathor/p2p/sync_v2/streamers.py 88.48% <100.00%> (+0.07%) ⬆️
hathor/pubsub.py 96.00% <100.00%> (ø)
hathor/reactor/reactor.py 100.00% <100.00%> (ø)
hathor/reactor/reactor_core_protocol.py 100.00% <100.00%> (ø)
hathor/reactor/reactor_protocol.py 100.00% <100.00%> (ø)
hathor/reactor/reactor_tcp_protocol.py 100.00% <100.00%> (ø)
... and 5 more

... and 13 files with indirect coverage changes

@glevco glevco force-pushed the chore/peer-connecting-event branch from d1f4193 to 02a3548 Compare July 11, 2023 21:06
@glevco glevco force-pushed the chore/improve-pubsub branch from 516de72 to 3922d10 Compare July 11, 2023 21:07
@glevco glevco force-pushed the chore/peer-connecting-event branch from 02a3548 to 67e0848 Compare July 11, 2023 21:49
Base automatically changed from chore/peer-connecting-event to master July 11, 2023 22:08
@glevco glevco force-pushed the chore/improve-pubsub branch from 3425778 to 022e889 Compare July 11, 2023 22:21
@glevco glevco added enhancement New feature or request refactor labels Jul 12, 2023
@glevco glevco force-pushed the chore/improve-pubsub branch from 022e889 to a67a277 Compare July 12, 2023 17:01
@glevco glevco marked this pull request as ready for review July 12, 2023 17:05
@glevco glevco requested review from jansegre and msbrogli as code owners July 12, 2023 17:05
@glevco glevco force-pushed the chore/improve-pubsub branch 4 times, most recently from d80cc56 to dbbd3b6 Compare July 20, 2023 16:53
jansegre
jansegre previously approved these changes Jul 26, 2023
jansegre
jansegre previously approved these changes Jul 27, 2023
@glevco glevco force-pushed the chore/improve-pubsub branch from 4406f1b to 1ba3af5 Compare August 3, 2023 22:07
msbrogli
msbrogli previously approved these changes Aug 21, 2023
msbrogli
msbrogli previously approved these changes Aug 22, 2023
jansegre
jansegre previously approved these changes Aug 22, 2023
@glevco glevco force-pushed the chore/improve-pubsub branch 2 times, most recently from d7fa5a4 to b7fc247 Compare August 25, 2023 22:47
@glevco glevco dismissed stale reviews from jansegre and msbrogli via 3561317 August 28, 2023 20:31
@glevco glevco force-pushed the chore/improve-pubsub branch from b7fc247 to 3561317 Compare August 28, 2023 20:31
@jansegre jansegre force-pushed the chore/improve-pubsub branch from 3561317 to 78229b7 Compare August 29, 2023 20:09
@msbrogli msbrogli merged commit 78229b7 into master Aug 29, 2023
@msbrogli msbrogli deleted the chore/improve-pubsub branch August 29, 2023 21:54
This was referenced Aug 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request refactor

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Investigate different message handling order on PubSub for production and tests

3 participants