[DRAFT] WIP QML Load Snapshot Signet #424

D33r-Gee · 2024-10-11T17:00:10Z

Based on #421. For evaluation purposes only!

Context & Background

Bitcoin Core's initial block download (IBD) process can take several hours or days depending on hardware
and network connection. The assumeutxo feature allows fast bootstrapping of a validating node by:

Loading a UTXO snapshot from a known good state
Allowing immediate use of the node while validation happens in background
Maintaining similar security guarantees to traditional sync

Current State

The assumeutxo infrastructure exists in Core (see doc/design/assumeutxo.md)
Command line support exists via loadtxoutset RPC
No GUI integration exists yet

What This PR Adds

GUI Integration: Adds snapshot loading UI in the Connection Settings panel
Progress Feedback: Visual progress indicator during snapshot loading
Validation Status: Shows snapshot verification state

Key components modified:

SnapshotSettings QML component (src/qml/components/SnapshotSettings.qml)
- Provides UI/UX for snapshot loading
- Displays progress and verification status
- Manages snapshot file selection
Node Interface Integration (src/node/interfaces.cpp)
- Implements core snapshot loading functionality
- Handles snapshot validation and activation
- Generates a progress based on the amount of coins in the snapshot (this could be finessed)
- Manages chainstate transitions during loading
QML Model Layer Integration
- NodeModel: Tracks and exposes snapshot loading progress
- ChainModel: Manages snapshot activation state
- Provides bridge between core functionality and QML UI
Basic State Handling
- Success: Updates UI to show loaded state with snapshot details
- Failure: Currently only logged to debug.log

Implementation Details

The snapshot loading process follows these steps:

User selects snapshot file via GUI
Backend validates snapshot metadata
Progress is tracked via m_snapshot_progress in NodeModel
On completion:
- Success: Shows loaded state with snapshot info
- Failure: Currently only logged to debug.log

Testing Instructions

Start node
Navigate to Connection Settings

Load snapshot using provided signet test file:

magnet:?xt=urn:btih:9da986cb27b3980ea7fd06b21e199b148d486880&dn=utxo-signet-160000.dat

Verify (see screenshots below):
- Progress indicator fills
- "Snapshot Loaded" confirmation appears
- Block clock shows ~50% progress
- For errors, check debug.log for [snapshot] or [loadsnapshot] messages

Ubuntu 22.04 Screenshots

Future Work

Integration with onboarding flow
GUI error handling and recovery paths
User-friendly error messages in the UI

Security Considerations

Maintains assumeutxo's security model
Validates snapshot against hardcoded chain parameters
Background validation ensures full security

Notes

The chainparams.cpp changes are temporary for testing - upstream already has the signet snapshot parameters hardcoded.

Related documentation:

doc/design/assumeutxo.md for security model
contrib/devtools/utxo_snapshot.sh for snapshot creation

This is a work in progress, do not merge!

D33r-Gee · 2024-10-15T17:46:32Z

With cb409e3 rebased and got rid off the extra options_mode settings which are not relevant for this evaluation.

Also deleted the extra loadsnapshot.cpp file and instead added the load snapshot function in src/node/interfaces.cpp

yashrajd · 2024-10-16T14:05:33Z

Should I be getting a binary specific for this issue? I tried loading a signet snapshot into the one for #421 but was unable to...

D33r-Gee · 2024-10-16T16:28:05Z

Should I be getting a binary specific for this issue?

Yes please download the artifact for your system here

I tried loading a signet snapshot into the one for #421 but was unable to...

That's correct behaviour, 421 is UI only and doesn't not have backend wiring

D33r-Gee · 2024-10-16T19:59:27Z

With b77f7c1 minor tweaks to the src/node/interfaces.cpp snapshotload() function.

It mimics the rpc call loadtxoutset from bitcoin/bitcoin repository

Next will connect the loading to the progress bar

D33r-Gee · 2024-10-22T22:12:36Z

with c59e557 Integrated the UI with the C++ backend, implementing a progress callback function to dynamically update the loading progress based on the number of coins processed.

Touched src/validation.h and src/validation.cpp to extract the coin loading progress.

Next: will update the "Loaded Snapshot" stack with the real info from the snapshot

D33r-Gee · 2024-10-25T20:46:36Z

With e3f8c70 moved the load snapshot functionality from nodeModel to optionsModel since it make it easier to store the snapshot info and reduces redundant variable declarations.

jarolrod

With e3f8c70 moved the load snapshot functionality from nodeModel to optionsModel since it make it easier to store the snapshot info and reduces redundant variable declarations.

e3f8c70 is a regression because optionsmodel is not an appropriate place for this, and we are now violating the role of settings.json to store the state of a process.

NodeModel and AppMode are more appropriate places for this information. But the entire approach here is still a concept NACK as we are performing unecessary changes to validation and chainparams.

src/qml/models/options_model.cpp

jarolrod · 2024-10-26T22:34:28Z

src/qml/models/options_model.cpp

+    if (new_snapshot_load_completed != m_snapshot_load_completed) {
+        m_snapshot_load_completed = new_snapshot_load_completed;
+        if (m_onboarded) {
+            m_node.updateRwSetting("snapshotloadcompleted", new_snapshot_load_completed);


This would be polluting settings.json that stores read/write configuration options for core -> with a state of a process. We cannot do this.

Makes sense, although I have a question for when we would want to allow the user to load a snapshot during onboarding, could the snapshot file path be stored in the settings.json?

yashrajd · 2024-10-29T10:27:35Z

Ok I tested artefact#235 and functionality & UI seems to work.

Didn't see block clock so couldn't verify easily. Adding some UI feedback below:

fix list item width

fix list item width
fix icon size

blockHeight is accurate
the date in the copy above it seems right
didn't check hash accuracy
hash value is overflowing as you can see, I'd probably wrap it over 2-3 lines, perhaps add copy functionality

D33r-Gee · 2024-10-31T22:22:43Z

with 90dc0dd rebased over main and addressed @jarolrod concerns by reverting back to just a minimal integration of the loading snapshot functionality.

@yashrajd I will address the UI issues next

As a follow up will investigate how to access the snapshot verification progress

Also looking into how to make the snapshot data persist and be available for display (i.e. snapshot block height, date, and merkle hash)

D33r-Gee · 2024-10-31T22:30:40Z

But the entire approach here is still a concept NACK as we are performing unecessary changes to validation and chainparams.

For the chainparams the changes there are only temporary for evaluating and testing within this repo. I checked upstream and the signet snapshot has already been hardcoded so once our repo is synced the changes in chainparams.cpp can be discarded.

This introduce the UI flow to load a AssumeUTXO snapshot into the Bitcoin Core App. It modifies the connection seetings and adds a SnapshotSettings file, Icon, and modified profress bar. It has been rebased

D33r-Gee · 2024-11-01T18:46:48Z

with 85592f5 made the snapshot activation persist on node restart by making use of the interface with chain->hasAssumedValidChain

Also changed the green-check icon size to 30 pix

Next looking into how to extract the progress from the ActivateSnapshot process

D33r-Gee · 2024-11-04T23:38:00Z

with 955c65d modified the snapshotLoad() function in src/node/interfaces.cpp and the initializeSnapshot() function in nodemodel.cpp to extract progress information using regular expressions.

This design decision aimed to keep snapshot loading mechanisms self-contained.

However, it looks like the use of std::stod is triggering some lint failures... will look into how to address that.

Perhaps the way to go is to use the handler in src/node/interfaces_ui ... will explore that path then update

D33r-Gee · 2024-11-08T21:48:51Z

with 0eba513 reverted back to minimal snapshot load functionality.

Snapshot loading progress is still not were I want it to be so in the meantime replaced the progress bar with placeholder text.

Also updated the snapshot info display. once successfully loaded, and I'm using the hardcoded info in chainparams since if the snapshot has loaded successfully it means the info there has been validated.
This is make the info persistent and not rely on the snapshot.dat file (in case it gets deleted)

D33r-Gee · 2024-11-12T21:33:40Z

With cd2ec69 added a section to keep track of snapshot loading, it is not ideal so will look for better ways to achieve it.

Since the core functionality is there will ask for "pre-reviews" before un-drafting

src/qml/components/SnapshotSettings.qml

D33r-Gee · 2024-11-13T21:24:50Z

with 5d0756c updated SnapshotSettings.qml with comments documenting the loading process.

with 262bb03 updated nodemodel by adding the m_snapshot_loading bool so that the "Snapshot Loading" page persist if the user navigates away from it.

Next looking into how to finesse the snapshot loading progress...

Adds minimal wiring to connect QML GUI to loading a signet UTXO snapshot via the connection settings. Uses SnapshotSettings.qml to allow user interaction. Modifies src/interfaces/node.h, src/node/interfaces.cpp and chainparams.cpp (temporarily for signet snapshot testing) to implement snapshot loading functionality through the node model. Current limitations: - Not integrated with onboarding process - Requires manual navigation to connection settings after initial startup - Snapshot verification progress is working, could be improved Testing: 1. Start the node 2. Complete onboarding 3. Navigate to connection settings 4. Load snapshot from provided interface

D33r-Gee · 2024-11-13T23:03:09Z

with a6bede5 removed comments from SnapshotSettings.qml

D33r-Gee marked this pull request as draft October 11, 2024 17:11

D33r-Gee force-pushed the qml-load-snapshot-signet branch from cb409e3 to 670ca05 Compare October 15, 2024 17:43

D33r-Gee mentioned this pull request Oct 15, 2024

cpp: exposing AssumeUTXO functionality to the GUI (QML) via the Node interface bitcoin/bitcoin#31094

Closed

1 task

D33r-Gee force-pushed the qml-load-snapshot-signet branch from 670ca05 to b77f7c1 Compare October 16, 2024 19:53

D33r-Gee force-pushed the qml-load-snapshot-signet branch from b77f7c1 to c59e557 Compare October 22, 2024 21:57

D33r-Gee force-pushed the qml-load-snapshot-signet branch from c59e557 to e3f8c70 Compare October 25, 2024 20:42

jarolrod reviewed Oct 26, 2024

View reviewed changes

D33r-Gee force-pushed the qml-load-snapshot-signet branch from e3f8c70 to 90dc0dd Compare October 31, 2024 22:09

qml: Introduce UI Flow for Loading Snapshot

c97d2a1

This introduce the UI flow to load a AssumeUTXO snapshot into the Bitcoin Core App. It modifies the connection seetings and adds a SnapshotSettings file, Icon, and modified profress bar. It has been rebased

D33r-Gee force-pushed the qml-load-snapshot-signet branch from 90dc0dd to 85592f5 Compare November 1, 2024 18:41

D33r-Gee force-pushed the qml-load-snapshot-signet branch from 85592f5 to 955c65d Compare November 4, 2024 22:39

D33r-Gee force-pushed the qml-load-snapshot-signet branch from 955c65d to 0eba513 Compare November 8, 2024 18:35

D33r-Gee force-pushed the qml-load-snapshot-signet branch from 0eba513 to cd2ec69 Compare November 12, 2024 21:30

jarolrod reviewed Nov 12, 2024

View reviewed changes

src/qml/components/SnapshotSettings.qml Outdated Show resolved Hide resolved

jarolrod reviewed Nov 12, 2024

View reviewed changes

src/qml/components/SnapshotSettings.qml Show resolved Hide resolved

D33r-Gee force-pushed the qml-load-snapshot-signet branch 2 times, most recently from 5d0756c to 262bb03 Compare November 13, 2024 21:23

D33r-Gee force-pushed the qml-load-snapshot-signet branch from 262bb03 to a6bede5 Compare November 13, 2024 22:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DRAFT] WIP QML Load Snapshot Signet #424

[DRAFT] WIP QML Load Snapshot Signet #424

D33r-Gee commented Oct 11, 2024 •

edited

Loading

D33r-Gee commented Oct 15, 2024

yashrajd commented Oct 16, 2024

D33r-Gee commented Oct 16, 2024

D33r-Gee commented Oct 16, 2024 •

edited

Loading

D33r-Gee commented Oct 22, 2024 •

edited

Loading

D33r-Gee commented Oct 25, 2024

jarolrod left a comment

jarolrod Oct 26, 2024

D33r-Gee Oct 31, 2024

yashrajd commented Oct 29, 2024 •

edited

Loading

D33r-Gee commented Oct 31, 2024 •

edited

Loading

D33r-Gee commented Oct 31, 2024

D33r-Gee commented Nov 1, 2024

D33r-Gee commented Nov 4, 2024

D33r-Gee commented Nov 8, 2024

D33r-Gee commented Nov 12, 2024

D33r-Gee commented Nov 13, 2024

D33r-Gee commented Nov 13, 2024

[DRAFT] WIP QML Load Snapshot Signet #424

Are you sure you want to change the base?

[DRAFT] WIP QML Load Snapshot Signet #424

Conversation

D33r-Gee commented Oct 11, 2024 • edited Loading

Context & Background

Current State

What This PR Adds

Implementation Details

Testing Instructions

Future Work

Security Considerations

Notes

D33r-Gee commented Oct 15, 2024

yashrajd commented Oct 16, 2024

D33r-Gee commented Oct 16, 2024

D33r-Gee commented Oct 16, 2024 • edited Loading

D33r-Gee commented Oct 22, 2024 • edited Loading

D33r-Gee commented Oct 25, 2024

jarolrod left a comment

Choose a reason for hiding this comment

jarolrod Oct 26, 2024

Choose a reason for hiding this comment

D33r-Gee Oct 31, 2024

Choose a reason for hiding this comment

yashrajd commented Oct 29, 2024 • edited Loading

D33r-Gee commented Oct 31, 2024 • edited Loading

D33r-Gee commented Oct 31, 2024

D33r-Gee commented Nov 1, 2024

D33r-Gee commented Nov 4, 2024

D33r-Gee commented Nov 8, 2024

D33r-Gee commented Nov 12, 2024

D33r-Gee commented Nov 13, 2024

D33r-Gee commented Nov 13, 2024

D33r-Gee commented Oct 11, 2024 •

edited

Loading

D33r-Gee commented Oct 16, 2024 •

edited

Loading

D33r-Gee commented Oct 22, 2024 •

edited

Loading

yashrajd commented Oct 29, 2024 •

edited

Loading

D33r-Gee commented Oct 31, 2024 •

edited

Loading