feat(consensus): Add new verifications for mempool only by msbrogli · Pull Request #1379 · HathorNetwork/hathor-core

msbrogli · 2025-08-22T16:29:40Z

Motivation

Add several verifications for mempool only (but one). As it impacts only the mempool, the policies can be more strict than the blockchain protocol itself.

Acceptance Criteria

Add BlockVerifier.verify_checkpoints() for all blocks.
Add NanoHeaderVerifier.verify_method_call() for nano transactions entering the mempool. This requires the blueprint to exist, the contract to have already been created, the method to exist (or a fallback method), and the call arguments to be parseable.
Add NanoHeaderVerifier.verify_seqnum() for nano transactions entering the mempool.
Add TransactionVerifier.verify_conflict() for transactions entering the mempool. This forbids transactions in the mempool to have a conflict with an already confirmed transaction.
Add TransactionVerifier.verify_token() for transactions entering the mempool. This limits the number of tokens, ensure that there is no duplicate token in the list, and ensure that all tokens are used at least once.
Add VertexVerifier.verify_old_timestamp() for all vertices entering the mempool. This limits how old these vertices can be.

Checklist

If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged

github-actions · 2025-08-22T16:38:35Z

Bencher Report

Branch	feat/mempool-verifications
Testbed	ubuntu-22.04

🚨 1 Alert

Benchmark	Measure Units	View	Benchmark Result (Result Δ%)	Upper Boundary (Limit %)
sync-v2 (up to 20000 blocks)	Latency minutes (m)	📈 plot 🚷 threshold 🚨 alert (🔔)	2.27 m (+32.98%) Baseline: 1.71 m	2.05 m (110.81%)

Click to view all benchmark results

Benchmark	Latency	Benchmark Result minutes (m) (Result Δ%)	Lower Boundary minutes (m) (Limit %)	Upper Boundary minutes (m) (Limit %)
sync-v2 (up to 20000 blocks)	📈 view plot 🚷 view threshold 🚨 view alert (🔔)	2.27 m (+32.98%) Baseline: 1.71 m	1.54 m (67.68%)	2.05 m (110.81%)

🐰 View full continuous benchmarking report in Bencher

glevco · 2025-09-15T19:29:10Z

hathor/verification/verification_service.py

+        if params.reject_too_old_vertices:
+            self.verifiers.vertex.verify_old_timestamp(vertex)


I think I prefer having the conditionals inside the methods. That is, just call them here and they'll be nops depending on the params.

The reason being, I think the verification service should just be a "dumb router", and it would be the method's responsibility to know when it should or not make the check, which makes sense because being mempool-only is something specific and internal for each method. Also, it makes sure that the correct conditionals are used whenever these methods are called, even if they're called outside the verification service.

Same for all other calls. What do you think?

I agree. Done!

glevco · 2025-09-15T19:33:35Z

hathor/verification/verification_service.py

I guess this solution is okay for now, but I think we should refactor it in the future to create a better separation of mempool-only checks and existing verification. What do you think?

glevco · 2025-09-15T19:34:35Z

hathor/verification/verification_service.py

Should we add all new mempool-only verifications to the consensus handling of reorgs, to remove invalid txs when the mempool changes?

Yes, I'll do it in another PR.

glevco · 2025-09-15T19:34:57Z

tests/tx/test_verification_mempool.py

Should we add all new verification methods to test_verification.py?

Yes, I'll do it in another PR.

glevco · 2025-09-15T19:44:37Z

hathor/verification/block_verifier.py

+            for cp in self._settings.CHECKPOINTS:
+                if cp.height == block_height and cp.hash != block.hash:
+                    raise CheckpointError(f'invalid checkpoint block (height={cp.height})')


Should we refactor the settings.CHECKPOINTS attribute to be a dict with height keys instead of a list, so we can replace this O(n) with O(1)? At the very least we could break out of the loop when the height is found.

I reduced the O(n) to O(1) but without changing the settings.CHECKPOINTS type.

glevco · 2025-09-15T19:45:12Z

hathor/verification/nano_header_verifier.py

+
+        nano_header = tx.get_nano_header()
+        if nano_header.is_creating_contract():
+            # TODO Confirm that the blueprint exists.


glevco · 2025-09-15T19:49:32Z

hathor/verification/transaction_verifier.py

+
+    def verify_standard_scripts(self, tx: Transaction) -> None:
+        """Verify that all outputs have standard scripts."""
+        pass


I decided to not add this verification now. I'll do it separately in the consensus simplify project.

msbrogli · 2025-09-17T15:58:11Z

hathor/verification/nano_header_verifier.py

+        block_storage = self._tx_storage.get_nc_block_storage(best_block)
+        seqnum = block_storage.get_address_seqnum(nano_header.nc_address)
+        diff = nano_header.nc_seqnum - seqnum
+        if diff < 0 or diff > MAX_SEQNUM_JUMP_SIZE:


This might prevent the same address from having "a lot of nano transactions" in the same block.

glevco · 2025-09-17T16:09:53Z

hathor/verification/nano_header_verifier.py

+    def verify_seqnum(self, tx: BaseTransaction) -> None:
+        assert tx.is_nano_contract()
+        assert isinstance(tx, Transaction)
+
+        nano_header = tx.get_nano_header()
+        best_block = self._tx_storage.get_best_block()
+        block_storage = self._tx_storage.get_nc_block_storage(best_block)
+        seqnum = block_storage.get_address_seqnum(nano_header.nc_address)
+        diff = nano_header.nc_seqnum - seqnum
+        if diff < 0 or diff > MAX_SEQNUM_JUMP_SIZE:
+            raise NCInvalidSeqnum(f'invalid seqnum (diff={diff})')


Should we iterate over all txs in the mempool instead of getting the best block storage?

hathor/dag_builder/artifacts.py

codecov · 2025-09-23T14:56:37Z

Codecov Report

❌ Patch coverage is 96.27329% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.83%. Comparing base (33a2888) to head (297959c).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
hathor/dag_builder/artifacts.py	77.77%	1 Missing and 1 partial ⚠️
hathor/verification/block_verifier.py	88.88%	1 Missing and 1 partial ⚠️
hathor/verification/nano_header_verifier.py	96.61%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1379      +/-   ##
==========================================
+ Coverage   85.74%   85.83%   +0.08%     
==========================================
  Files         431      431              
  Lines       33133    33278     +145     
  Branches     5196     5225      +29     
==========================================
+ Hits        28411    28564     +153     
+ Misses       3679     3673       -6     
+ Partials     1043     1041       -2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

msbrogli requested a review from jansegre as a code owner August 22, 2025 16:29

msbrogli self-assigned this Aug 22, 2025

msbrogli added this to Hathor Network Aug 22, 2025

github-project-automation bot moved this to Todo in Hathor Network Aug 22, 2025

msbrogli moved this from Todo to In Progress (WIP) in Hathor Network Aug 22, 2025

msbrogli force-pushed the feat/mempool-verifications branch from 80f6f7f to 6d0cfa0 Compare August 22, 2025 16:30

msbrogli force-pushed the feat/mempool-verifications branch from 6d0cfa0 to 5222b85 Compare August 22, 2025 20:05

msbrogli force-pushed the feat/mempool-verifications branch 5 times, most recently from 1930e49 to 3817185 Compare September 11, 2025 21:41

msbrogli force-pushed the feat/mempool-verifications branch 8 times, most recently from 0741c79 to 8d563c0 Compare September 12, 2025 20:44

glevco reviewed Sep 15, 2025

View reviewed changes

msbrogli force-pushed the feat/mempool-verifications branch from 8d563c0 to 897f6b4 Compare September 16, 2025 15:40

jansegre mentioned this pull request Sep 17, 2025

Release-candidate v0.66.0-rc.1 #1407

Merged

2 tasks

msbrogli commented Sep 17, 2025

View reviewed changes

glevco reviewed Sep 17, 2025

View reviewed changes

msbrogli force-pushed the feat/mempool-verifications branch from 897f6b4 to 3c9f5ed Compare September 18, 2025 16:42

msbrogli moved this from In Progress (WIP) to In Review (Done) in Hathor Network Sep 18, 2025

msbrogli force-pushed the feat/mempool-verifications branch 2 times, most recently from 17a16a8 to 7fc61e7 Compare September 19, 2025 14:44

msbrogli force-pushed the feat/mempool-verifications branch 3 times, most recently from fa7fd1d to 52dcc09 Compare September 22, 2025 20:33

jansegre reviewed Sep 23, 2025

View reviewed changes

hathor/dag_builder/artifacts.py Outdated Show resolved Hide resolved

msbrogli force-pushed the feat/mempool-verifications branch 2 times, most recently from 39aca90 to 62447bf Compare September 23, 2025 14:28

msbrogli force-pushed the feat/mempool-verifications branch 4 times, most recently from 74eeb43 to 3088105 Compare September 24, 2025 14:46

jansegre previously approved these changes Sep 24, 2025

View reviewed changes

msbrogli dismissed jansegre’s stale review via be64c1c September 25, 2025 15:40

msbrogli force-pushed the feat/mempool-verifications branch 2 times, most recently from be64c1c to e84308c Compare September 25, 2025 15:41

jansegre previously approved these changes Sep 25, 2025

View reviewed changes

msbrogli dismissed jansegre’s stale review via 5031493 September 25, 2025 17:42

msbrogli force-pushed the feat/mempool-verifications branch 2 times, most recently from 5031493 to eec4b15 Compare September 25, 2025 18:10

feat(consensus): Add new verifications for mempool only

297959c

msbrogli force-pushed the feat/mempool-verifications branch from eec4b15 to 297959c Compare September 25, 2025 18:10

jansegre approved these changes Sep 25, 2025

View reviewed changes

msbrogli merged commit 297959c into master Sep 25, 2025
6 of 8 checks passed

msbrogli deleted the feat/mempool-verifications branch September 25, 2025 19:29

github-project-automation bot moved this from In Review (Done) to Waiting to be deployed in Hathor Network Sep 25, 2025

This was referenced Sep 25, 2025

Release-candidate v0.66.0-rc.3 #1430

Merged

Release v0.66.0 #1427

Merged

jansegre moved this from Waiting to be deployed to Done in Hathor Network Sep 26, 2025

jansegre mentioned this pull request Oct 7, 2025

Release-candidate v0.67.0-rc.1 #1419

Merged

2 tasks

jansegre mentioned this pull request Oct 16, 2025

Release v0.67.0 #1473

Merged

2 tasks

		if params.reject_too_old_vertices:
		self.verifiers.vertex.verify_old_timestamp(vertex)

Conversation

msbrogli commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Acceptance Criteria

Checklist

Uh oh!

github-actions bot commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚨 1 Alert

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

msbrogli Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov bot commented Sep 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

msbrogli commented Aug 22, 2025 •

edited

Loading

github-actions bot commented Aug 22, 2025 •

edited

Loading

msbrogli Sep 19, 2025 •

edited

Loading

codecov bot commented Sep 23, 2025 •

edited

Loading