metrics: measure block assembly re-evaluation failures#6526
Merged
algorandskiy merged 15 commits intoalgorand:masterfrom Jan 28, 2026
Merged
metrics: measure block assembly re-evaluation failures#6526algorandskiy merged 15 commits intoalgorand:masterfrom
algorandskiy merged 15 commits intoalgorand:masterfrom
Conversation
cce
force-pushed
the
transaction-pool-reevaluation-metrics
branch
from
348c0b7 to
4dfd630
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6526 +/- ##
==========================================
- Coverage 47.83% 47.71% -0.12%
==========================================
Files 662 655 -7
Lines 87991 87906 -85
==========================================
- Hits 42089 41945 -144
- Misses 43126 43193 +67
+ Partials 2776 2768 -8 ☔ View full report in Codecov by Sentry. |
cce
force-pushed
the
transaction-pool-reevaluation-metrics
branch
from
4dfd630 to
229b273
Compare
Add always-enabled Prometheus metrics to track transaction groups that
were successfully added to the pool but later failed during block
assembly re-evaluation.
New reeval metric (TagCounter):
- algod_tx_pool_reeval_{TAG} where TAG is one of: fee, txn_dead,
too_large, groupid, txid_eval, lease_eval, eval
This change also refactors the error classification logic to be shared
between txHandler (initial remember) and transactionPool (re-evaluation):
- Moved tag constants to pools.TxPoolErrTag* for shared use
- Added pools.ClassifyTxPoolError() to classify BlockEvaluator errors
- Both txHandler and transactionPool now use the same classification
The reeval metrics are distinct from the existing txHandler metrics
(algod_transaction_messages_txpool_remember_err_{TAG}) which track
initial admission failures.
These metrics are always enabled (no config flag required) unlike the
existing telemetry events which require EnableAssembleStats.
cce
force-pushed
the
transaction-pool-reevaluation-metrics
branch
from
229b273 to
711d840
Compare
algorandskiy
previously approved these changes
Jan 14, 2026
Extend the error classification in txPoolErrors.go to distinguish
between different types of evaluation failures that were previously
all lumped into the generic "eval" bucket.
New error tags:
- not_well: TxnNotWellFormedError (malformed transaction)
- teal_err: logic.EvalError (TEAL runtime error)
- teal_reject: TEAL approval returned false
- min_balance: Account balance below minimum requirement
- overspend: Insufficient Algo funds
- asset_bal: Insufficient asset balance
These tags help operators understand the nature of transaction
failures on their relays:
- teal_reject indicates round-check MEV contracts failing at execution
- overspend indicates spray-and-pray attacks with unfunded accounts
- asset_bal distinguishes asset balance failures from Algo overspend
The classification uses type assertions for typed errors (EvalError,
TxnNotWellFormedError) and string matching for fmt.Errorf patterns
("rejected by ApprovalProgram", "balance below min", etc.).
- Document why cap/pending_eval excluded from reeval tags
- Make "insufficient" pattern more specific ("insufficient balance")
- Add TestClassifyByErrorMessage for message-based classification
Add metrics to track positive outcomes during re-evaluation: - algod_tx_pool_reeval_success: txn groups successfully re-evaluated - algod_tx_pool_reeval_committed: txn groups removed (already in block) This provides parity with txHandler's transactionMessagesRemember counter.
Replace type switch with errors.As calls in classifyUnwrappedError.
This properly traverses wrapped errors (e.g., from Remember's
fmt.Errorf("TransactionPool.Remember: %w", err)) to find the
underlying typed error.
Use TxPoolErrTags for both txPoolReevalCounter and transactionMessageTxPoolCheckCounter to ensure all tags that ClassifyTxPoolError can return are predeclared. This prevents schema drift where a counter receives an unregistered tag. Removed the separate TxPoolReevalErrTags since all counters now use the same complete set.
Move ClassifyTxPoolError, error tags, and metrics counters from txPoolErrors.go into errors.go alongside the error type declarations. Rename txPoolErrors_test.go to errors_test.go.
…reevaluation-metrics
algorandskiy
previously approved these changes
Jan 28, 2026
Exercise real evaluator/pool errors through RememberOne and re-evaluation, then verify ClassifyTxPoolError returns the correct tag. Covers fee escalation, lease re-eval, asset balance, TEAL reject, and TEAL runtime error paths. Consolidates min-balance classification into the existing TestSenderGoesBelowMinBalance test.
…reevaluation-metrics-coverage # Conflicts: # data/pools/transactionPool_test.go # data/txHandler.go
cce
force-pushed
the
transaction-pool-reevaluation-metrics
branch
from
bcb70c7 to
7a0b140
Compare
algorandskiy
approved these changes
Jan 28, 2026
jannotti
approved these changes
Jan 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds a new TagCounter metric to track transaction groups that were successfully added to the pool, but later failed during block assembly re-evaluation:
This change also refactors the error classification logic to be shared between txHandler (initial remember) and transactionPool (re-evaluation):
The reeval metrics are distinct from the existing txHandler metrics (algod_transaction_messages_txpool_remember_err_{TAG}) which track initial admission failures.