Skip to content

chore: added a composite index to tx_output to reduce query time#326

Merged
andreabadesso merged 1 commit intomasterfrom
chore/tx-output-composite-index
Dec 1, 2025
Merged

chore: added a composite index to tx_output to reduce query time#326
andreabadesso merged 1 commit intomasterfrom
chore/tx-output-composite-index

Conversation

@andreabadesso
Copy link
Copy Markdown
Collaborator

@andreabadesso andreabadesso commented Dec 1, 2025

Motivation

Problem

The following query was taking ~90 seconds to execute:

UPDATE `address_balance`
SET `unlocked_authorities` = (
    SELECT BIT_OR(`authorities`)
    FROM `tx_output`
    WHERE `address` = 'HH5As5aLtzFkcbmbXZmE65wSd22GqPWq2T'
      AND `token_id` = '00'
      AND `locked` = FALSE
      AND `spent_by` IS NULL
      AND `voided` = FALSE
)
WHERE `address` = 'HH5As5aLtzFkcbmbXZmE65wSd22GqPWq2T'
  AND `token_id` = '00'

Root Cause

MySQL was choosing tx_output_spent_by_idx, scanning ~1M rows (all unspent UTXOs) and filtering by address/token_id afterward.

Solution

Add a composite covering index:

CREATE INDEX idx_tx_output_utxo_lookup 
ON tx_output (address, token_id, spent_by, voided, locked, authorities);

Results

Metric Before After
Index used spent_by_idx idx_tx_output_utxo_lookup
Rows scanned 1,002,812 270,830
Filtered 6.25% 100%
Extra Using index condition; Using where Using index (covering)

Query time expected to drop from ~90s to milliseconds.

Index disk usage

On the current database with ~7.1M rows, the new index idx_tx_output_utxo_lookup uses ~1.26 GB.
At 10x scale (~71M rows), estimated size: ~12.6 GB.

Acceptance Criteria

  • We should check if the index already exist before creating it because the index was already created in mainnet-staging

Checklist

  • If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged
  • Make sure either the unit tests and/or the QA tests are capable of testing the new features
  • Make sure you do not include new dependencies in the project unless strictly necessary and do not include dev-dependencies as production ones. More dependencies increase the possibility of one of them being hijacked and affecting us.

@andreabadesso andreabadesso self-assigned this Dec 1, 2025
@andreabadesso andreabadesso added the enhancement New feature or request label Dec 1, 2025
@andreabadesso andreabadesso moved this from Todo to In Progress (Done) in Hathor Network Dec 1, 2025
@andreabadesso andreabadesso force-pushed the chore/tx-output-composite-index branch from 9b31f33 to 1d981e2 Compare December 1, 2025 14:17
@github-project-automation github-project-automation Bot moved this from In Progress (Done) to In Review (WIP) in Hathor Network Dec 1, 2025
@pedroferreira1 pedroferreira1 moved this from In Review (WIP) to In Review (Done) in Hathor Network Dec 1, 2025
@andreabadesso andreabadesso merged commit 1de8db0 into master Dec 1, 2025
1 check passed
@github-project-automation github-project-automation Bot moved this from In Review (Done) to Waiting to be deployed in Hathor Network Dec 1, 2025
@andreabadesso andreabadesso mentioned this pull request Jan 28, 2026
2 tasks
@andreabadesso andreabadesso moved this from Waiting to be deployed to Done in Hathor Network Jan 28, 2026
This was referenced Jan 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants