fix: masternode/evonode identity load deadlock + identity unload capability (#889)

CHANGELOG.md

@@ -25,6 +25,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ### Added
 
+- **Unload an identity from this device**: Identity Hub → Settings now has a
+  working "Unload this identity from this device" action. It removes the
+  identity from the app on this device and clears the local keys, profile
+  information, DashPay details, and queued scheduled DPNS votes that the app can
+  identify for it, while leaving your other identities untouched. The
+  confirmation names how many scheduled votes will be cancelled. If cleanup
+  cannot finish, the app reports that clearly so you can retry. The identity
+  itself is unaffected on the network. Wallet-derived private keys can be
+  restored from the wallet. To load the identity again, you need recovery
+  information for any keys stored only on this device. Unloaded identities now
+  stay unloaded after automatic wallet discovery, and delayed profile results
+  are ignored after an unload.
+
 - **Automatic Platform node refresh during upgrades**: migrating a pre-1.0
   installation now triggers a best-effort Mainnet or Testnet node refresh.
   Failed attempts retry on later launches until fresh addresses are saved and
@@ -63,6 +76,84 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ### Fixed
 
+- **Identity unload and reload follow-ups for #889 / PR #925**: node load forms
+  now reject regular identities, wallet discovery keeps deliberately unloaded
+  identities unloaded until the user explicitly loads them again even when
+  discovery overlaps an unload, successful loads are not reported as failed
+  when marker cleanup leaves recoverable residue, associated voter removal
+  outcomes are reported accurately, partial cleanup failures still reconcile
+  the app's active identity state, and late DashPay profile responses can no
+  longer restore stale profile data. Reloading a masternode or evonode
+  identity that was left in a recoverable state by an earlier cleanup failure
+  now finishes that cleanup and completes the load, instead of being rejected
+  as already loaded; "delete all local data" now reaches those same
+  recoverable identities too. That same recovery now runs consistently across
+  every way an identity can be reloaded — overwriting, merging keys into an
+  existing record, loading from a wallet, loading by DPNS name, and automatic
+  wallet discovery — so a leftover key from an earlier interrupted unload is
+  always cleared before a fresh load replaces it, not just on one path.
+
+- **Identity unload/removal confirmations, feedback, and discovery guarantees, made consistent**:
+  the three places you can unload or remove an identity (Identity Hub →
+  Settings, the Identities screen, and a masternode's detail view) now show
+  the same specific-verb, input-blocking confirmation everywhere, instead of
+  one of them using a generic "Yes"/"No". Removing a masternode or evonode
+  from the Masternodes tab now reports the same success/warning outcome as
+  removing it anywhere else, instead of showing no feedback at all when
+  cleanup left residue behind. A wallet-wide identity search can no longer
+  bring back an identity you unloaded under any circumstance — previously
+  only the automatic background scan respected that — and a user-started
+  search now reports how many identities it left unloaded and how many it
+  failed to save, instead of looking identical to finding nothing new. The
+  unload confirmation's own wording was corrected to match: it no longer
+  implies a manual search can undo the unload, and it now names an actual,
+  reachable way back (the identity's own ID or username) instead of one
+  hidden behind an advanced-options toggle.
+
+- **Contest-decision ETA tooltip no longer under-reports the wait**: the
+  pending-username hover tooltip could round a decision estimate down by a
+  whole hour or day (e.g. showing "about 2 hours" for a decision actually
+  about 3 hours away), depending on exactly when it happened to be read. It
+  now rounds to the nearest hour/day instead of always rounding down.
+
+- **"Clear Database" no longer reports a clean wipe it did not finish**:
+  the wipe now keeps every identity reserved until the last step is done, so an
+  identity being loaded in the background cannot be written back to disk after
+  the wipe has already passed it and still be reported as erased. A failure
+  while removing retired shielded files no longer aborts the wipe partway or
+  makes identities that were erased successfully look as though they failed.
+  This closes the race against a concurrent load of the same identity in the
+  common case; one narrow exception remains (an identity that is both marked
+  unloaded and still on the device briefly reopens its slot between two
+  internal steps), and a wipe that loses that narrow race still reports itself
+  incomplete rather than succeeding silently. Other identity operations in
+  flight during a wipe (refreshing, adding a key, sending funds, and similar)
+  are not guarded yet and are tracked as follow-up work.
+
+- **A removal interrupted by a cleanup failure stays findable**: if the app
+  cannot finish clearing an identity's local data, it now records the identity
+  as unloaded, so loading it again finds the leftovers and finishes the job.
+  Previously such an identity could disappear from every list while its private
+  keys remained on the device with nothing able to reach them. The message
+  shown when this happens names that recovery — load the identity again, then
+  unload or remove it — instead of suggesting a retry that was impossible.
+
+- **Removing an identity now says what it really does**: the "Remove" action on
+  the Identities list and "Remove masternode" on the masternode page show the
+  same confirmation as Identity Hub → Settings, rather than wording that
+  suggested the identity was merely untracked. That confirmation is now
+  accurate too: it names the private keys and app entry that are deleted,
+  points to the "Clear Database" action in Settings for synced data such as
+  contacts and payment history that this action does not remove, and discloses
+  that the app remembers the unload so automatic discovery does not bring the
+  identity back. Removing a masternode still states that its voting identity
+  goes with it, and a masternode or evonode confirmation now describes how that
+  node is really restored — loaded again with its ProTxHash, with its private
+  keys entered again by hand if it held any — instead of promising wallet
+  recovery it never had. An identity with a name you chose is now also identified by its
+  full identifier, so two identities sharing a name cannot be confused on an
+  action that deletes keys.
+
 - **Wallet rename consistency**: renaming a wallet no longer overwrites other
   saved wallet details when metadata cannot be read. Overlapping renames and
   wallet removals also keep displayed aliases and deleted-wallet metadata
@@ -74,6 +165,11 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
   does not support shielded sending from when the current interface mode does
   not unlock it.
 
+- **Loading a masternode or evonode identity that was already loaded as a
+  regular identity no longer gets stuck**: the load screen now recognizes a
+  matching identity that hasn't finished loading and completes it correctly
+  instead of endlessly reporting it as already present.
+
 - **Identity Home actions simplified**: the action row previously had six
   buttons — several of which opened the same screen (`Send`/`Send to another
   identity`, `Receive`/`Add funds`). It's now one row of four: **Add funds**,

CLAUDE.md

@@ -107,7 +107,7 @@ Code lives by responsibility, not convenience:
 
 - **`model/`** — stateless data types and pure validation (format/length/charset). The single source of truth for validation. No `AppContext`, `Sdk`, DB, or `BackendTask`. All fee estimation goes in `model/fee_estimation.rs` — never inlined elsewhere.
 - **`backend_task/`** — async business logic, one submodule per domain; the authoritative enforcement layer. `TaskError` and its typed variants live in `backend_task/error.rs`.
-- **`database/`** — SQLite persistence, one module per domain.
+- **`database/`** — **Frozen legacy `data.db`, read-only in production.** Production opens an existing `data.db` with `SQLITE_OPEN_READ_ONLY` (`Database::open_legacy_read_only`, `src/app.rs`); the schema ladder in `database/initialization.rs` runs only on a fresh install that has no `data.db` yet, never on an existing one. Never add a table, column, or write path here — it becomes permanently unwritable after the install's first boot. All current durable state is a `DetKv` key (see `docs/kv-keys.md`) or a `SecretStore` entry (`wallet_backend/secret_seam.rs`); `database/` exists solely as a v0.9.3→v1.0 migration-read source and recovery artifact.
 - **`context/`** — `AppContext` submodules (`*_db.rs`, lifecycle, settings, status).
 - **`wallet_backend/`** — the wallet orchestration seam: adapters, views, backend-side live caches, signers, the secret chokepoint, the event bridge. All wallet secret bytes (HD seed, imported single key, identity private key) enter/leave the vault through ONE chokepoint, `wallet_backend/secret_seam.rs` (raw `SecretBytes`, no DET-side serialization). Per-secret at-rest encryption is implemented via `put_secret_protected`/`get_secret_protected` (Argon2id + XChaCha20-Poly1305, per-secret object-password envelope, AAD bound to `wallet_id ‖ label`); unprotected secrets use `put_secret`/`get_secret` (raw, keyless vault). Identity keys (imported/loaded, including masternode voting/owner/payout) enter unprotected (Tier-1 keyless) at load/creation time — the load flow has no password field — but can be sealed to Tier-2 per-identity afterward via `IdentityTask::ProtectIdentityKeys` (Key Info screen → "Add password protection…"; gated by vault-key scheme, not identity type). The keyless-vault residual is only no-password secrets and keys the user has not opted to protect. Design + migration: `docs/ai-design/2026-06-19-secret-storage-seam/`.
 - **`ui/<domain>/`** — screens (`ScreenLike`). UI may *call* `model/` validators for instant feedback but never implements its own validation.
@@ -151,7 +151,7 @@ User-facing error messages (shown in `MessageBanner` via `Display`) must follow
 - **Wallet Backend (`wallet_backend/`)** — Wallet orchestration seam: adapters, views, backend-side live caches, signers, the secret chokepoint (`secret_seam.rs`), and the event bridge. A thin adapter over the upstream `platform-wallet` crate.
 - **Context (`context/`)** — `AppContext`: shared state — network config, SDK client, database, wallets, settings cache, connection health (`ConnectionStatus` / `SpvManager`), split into submodules (`identity_db.rs`, `wallet_lifecycle.rs`, `settings_db.rs`, etc.). Glue between layers.
 - **Model (`model/`)** — Pure data types and stateless validation (amounts, fees, settings, wallet/identity models). No side effects, no IO. All fee estimation lives in `model/fee_estimation.rs` — never inline fee math elsewhere.
-- **Database (`database/`)** — SQLite persistence (rusqlite), one module per domain. Typed CRUD, no business decisions.
+- **Database (`database/`)** — Frozen legacy `data.db`. Read-only in production (migration-source reads and recovery only); no new tables, columns, or writes. Current persistence is `DetKv` over `det-app.sqlite` / `platform-wallet.sqlite` (`docs/kv-keys.md`) and `SecretStore`.
 - **Platform Integration** — Chain sync, address derivation, asset-lock/identity handling, and the shielded coordinator come from the upstream **`platform-wallet`** crate (git dep, dashpay/platform); DET is a thin adapter over it via `wallet_backend/`. SPV health is surfaced through `SpvManager` → `ConnectionStatus`. (DET's bespoke `src/spv/` stack and the `core_zmq_listener` module were removed in the platform-wallet migration.)
 
 ### Layer Rules
@@ -354,7 +354,9 @@ Consider whether a repeated or reused message belongs in a dedicated `TaskError`
 
 ## Database
 
-Single SQLite connection wrapped in `Mutex<Connection>`. Schema initialized in `database/initialization.rs`. Domain modules provide typed CRUD methods. Backend task errors use `TaskError` (`src/backend_task/error.rs`) — see App Task System section above.
+`AppContext.db` is the **legacy** `data.db` — a frozen migration-read source and recovery artifact, not a general persistence layer. Production opens it with `SQLITE_OPEN_READ_ONLY` whenever the file already exists, and only initializes (runs `database/initialization.rs`'s schema ladder) on a fresh install that has none yet; consequently that ladder never executes against an existing production install, and any write attempted on one fails at the SQLite layer. **Never add a table, column, or write path to `database/`.**
+
+All current durable state lives in `DetKv` (wraps the upstream `platform_wallet_storage::KvStore`; two backing SQLite files, `det-app.sqlite` and `spv/<network>/platform-wallet.sqlite` — see `docs/kv-keys.md` for the full key registry) or `SecretStore` (`wallet_backend/secret_seam.rs`). New persistent state is a new `DetKv` key registered in `docs/kv-keys.md`, never a new SQL table. Backend task errors use `TaskError` (`src/backend_task/error.rs`) — see App Task System section above.
 
 ## Platform Targets
 

docs/kv-keys.md

@@ -10,6 +10,11 @@ Three backing stores exist:
 | `platform-wallet.sqlite` | `<data_dir>/spv/<net>/platform-wallet.sqlite` | Per-network identities, tokens, contracts, DashPay overlays, platform addresses, selected wallet |
 | `SecretStore` | `<data_dir>/secrets/det-secrets.*` | Encrypted HD-wallet seed envelopes and imported single-key private bytes |
 
+Deliberately absent from this table: the legacy `data.db` behind `src/database/`. It is a
+frozen v0.9.3→v1.0 migration-read source, opened read-only in production whenever it already
+exists — never a target for new state. New persistent state is always a new `DetKv` key
+registered below, never a new SQL table.
+
 In the per-domain tables below, a `Scope` of `None` denotes `DetScope::Global`.
 
 ---
@@ -82,11 +87,16 @@ Source: `src/model/selected_wallet.rs`, `src/wallet_backend/mod.rs`
 
 The identity blob and top-up history are **identity-scoped** (`DetScope::Identity(&id)`) so the upstream soft-cascade reaps them when the identity row is deleted. `DetScope::Identity` has no cross-identity listing, so a Global `det:identity_index:v1` slot holds the complete id roster the load-all paths iterate. `det:identity_order:v1` is a separate user-ordering view (may lag the full set) and stays Global.
 
+`det:forgotten_identity:<id>` is Global for the opposite reason to the blob: the marker's whole purpose is to outlive the identity it names, so automatic discovery cannot resurrect a deliberate unload. An identity-scoped slot would be reaped by that same soft-cascade at exactly the moment the marker becomes load-bearing. Like the other Global identity keys it is per-network by virtue of the per-network store, not by anything in the key or the value.
+
+The markers are **one key per identity**, enumerated by prefix scan, rather than a single slot holding the whole set. `DetKv` takes the persister lock per call, so a shared collection would make every marker write an unguarded read-modify-write: two identities unloading concurrently would clobber each other, either dropping a marker (the unload silently forgotten) or resurrecting a cleared one. Independent keys collide no more than the per-row SQL table this replaced, and confine a damaged marker to the one identity it names.
+
 | Key | Scope | Store | Value type | Notes |
 |-----|-------|-------|------------|-------|
 | `det:identity:v1` | `DetScope::Identity(&id)` | `platform-wallet.sqlite` | `StoredQualifiedIdentity` | Fields: `qi_bytes` (inner bincode, redacted in `Debug`), `status: u8`, `identity_type: String`, `wallet_hash: Option<[u8;32]>`, `wallet_index: Option<u32>` |
 | `det:identity_index:v1` | `None` | `platform-wallet.sqlite` | `Vec<[u8;32]>` | Complete enumeration index of stored identity ids |
 | `det:identity_order:v1` | `None` | `platform-wallet.sqlite` | `Vec<[u8;32]>` | User-chosen display ordering of identity ID raw bytes |
+| `det:forgotten_identity:<base58_id>` | `None` | `platform-wallet.sqlite` | `()` | Presence-only flag: the user deliberately unloaded this identity and discovery must not restore it. One key per identity |
 | `det:top_ups:v1` | `DetScope::Identity(&id)` | `platform-wallet.sqlite` | `BTreeMap<u32, u64>` | Top-up history: account index → credits |
 
 Source: `src/context/identity_db.rs`
@@ -204,8 +214,8 @@ Source: `src/wallet_backend/single_key.rs` (`SINGLE_KEY_PRIV_LABEL_PREFIX`, `SIN
 | Store | Key count |
 |-------|-----------|
 | `det-app.sqlite` | 4 (settings, wallet-meta sidecar, single-key-meta sidecar, migration sentinel) |
-| `platform-wallet.sqlite` | 21 (across 8 domains) |
+| `platform-wallet.sqlite` | 22 (across 8 domains) |
 | `SecretStore` | 2 label patterns (seed envelopes, imported-key private bytes) |
-| **Total** | **27** |
+| **Total** | **28** |
 
 Prefixed/templated keys (e.g. `det:identity:<id>`) are counted once per prefix, not per instance. `SecretStore` entries are counted as label-pattern families, not per-wallet instances.