fix(drive): skip the ranked offset by counting instead of walking

book/src/drive/ranked-index-examples.md

@@ -321,11 +321,11 @@ The response carries the skip back in `RankedEntries.skipped` (see [The Response
 
 Three properties worth stating plainly:
 
-- **The skip is attested, not walked.** grovedb proves the skipped region from the counted subtree commitments (`HashWithCount` / `HashWithCountAndSum`) rather than by traversing it. Both the prover's work and the proof's size stay `O(log n + k)` **at any offset**.
-- **There is therefore no offset ceiling.** An offset of 4 and an offset of four billion cost the same, so there is no denial-of-service lever a cap would close — and a cap would only stop honest deep pagination.
-- **An offset past the end is a positive answer.** `entries` comes back empty and `skipped` is the ranking's *entire attested population*. "There are only 12 groups" is more information than a bare empty list.
+- **The skip is counted, not walked.** grovedb descends the secondary reading each subtree's aggregate count and collapses any subtree that fits entirely inside the remaining offset, instead of stepping through it. Both paths do this: the prover attests the skipped region from the counted subtree commitments (`HashWithCount` / `HashWithCountAndSum`), and the unproven read performs the same counted descent without building a proof. Work and proof size stay `O(log n + k)` **at any offset**.
+- **There is therefore no offset ceiling.** An offset of 4 and an offset of four billion cost the same order of work — on either path, the deeper one in fact cheaper, since a tree that fits entirely inside the offset collapses at the root. There is no denial-of-service lever a cap would close, and a cap would only stop honest deep pagination.
+- **An offset past the end is a positive answer.** `entries` comes back empty and `skipped` is the ranking's *entire reported population*. "There are only 12 groups" is more information than a bare empty list.
 
-On the **unproven** read there is nothing to attest and grovedb's read API does not report a short walk, so `skipped` simply echoes the requested offset. The proved and unproven paths therefore disagree in exactly one case — an offset past the end, where the unproven read reports the request and the proved one reports the truth. **Callers who need the population must prove.**
+Both paths report the same `skipped`: the offset you asked for when the skip succeeded, and the ranking's total population when the walk ran out of groups first. What differs is the warrant, not the value. On the proved path it is cryptographically attested, re-derived by the verifier from the counted commitments. On the unproven path it is an **unverified claim, exactly like the entries beside it** — equal to the attested value on an honest node, with nothing forcing a node to be honest. **Callers who need to trust the population, rather than merely receive it, must still prove.**
 
 ## The Response
 

packages/dapi-grpc/protos/platform/v0/platform.proto

@@ -1125,11 +1125,14 @@ message GetDocumentsRequest {
     // routes to the ranked executor (`group_by` + a single `order_by`
     // naming the selected aggregate), `offset` skips that many ranks
     // before the returned page, so `ORDER BY avg(grade) DESC LIMIT 1
-    // OFFSET 4` is the 5th-best group. The skip is **count-attested**,
-    // not walked: grovedb proves it from the counted subtree
-    // commitments, so the proof stays `O(log n + k)` at any offset and
-    // the response echoes the attested number in
-    // `RankedEntries.skipped`. There is deliberately no ceiling — an
+    // OFFSET 4` is the 5th-best group. The skip is **counted, not
+    // walked**: grovedb descends on each subtree's aggregate count and
+    // collapses whole subtrees that fit inside the remaining offset, so
+    // the work stays `O(log n + k)` at any offset and the response
+    // reports the skip it performed in `RankedEntries.skipped`. On a
+    // proved request that count is additionally *attested* — committed
+    // to by the proof and re-derived by the verifier; on an unproved
+    // one it is the node's own report. See `RankedEntries.skipped`. There is deliberately no ceiling — an
     // offset of 4 and an offset of four billion cost the same, so
     // there is no denial-of-service lever a cap would close. An offset
     // past the end of the ranking is a provable answer rather than an
@@ -1417,20 +1420,31 @@ message GetDocumentsResponse {
       // group rather than the best.
       //
       // **When a requested offset exceeds the population**, `entries`
-      // is empty and `skipped` is the ranking's attested *total*
+      // is empty and `skipped` is the ranking's *total* reported
       // population — a positive, useful answer ("there are only 12
       // groups") rather than a bare empty list.
       //
-      // On the proved path the number is grovedb's cryptographically
-      // attested count, re-derived by the verifier from the counted
-      // subtree commitments in the proof bytes rather than trusted
-      // from this field; a proving client should use the verified
-      // value. On the unproven read there is nothing to attest and
-      // grovedb's read API does not report a short walk, so the server
-      // echoes the requested offset. The two therefore disagree in
-      // exactly one case — an offset past the end, where the unproven
-      // read reports the request and the proved one reports the truth.
-      // Callers who need the population must prove.
+      // Both paths report the same quantity: the offset you asked for
+      // when the skip succeeded, and the ranking's total population
+      // when the walk ran out of groups first. They no longer disagree
+      // anywhere, including past the end.
+      //
+      // What differs is the *warrant*, not the value. On the proved
+      // path the number is cryptographically attested — re-derived by
+      // the verifier from the counted subtree commitments in the proof
+      // bytes rather than trusted from this field — so a proving client
+      // should use the verified value and ignore this one. On the
+      // unproven path it is an **unverified claim**, exactly like the
+      // entries beside it: it equals the attested value on an honest
+      // node, and nothing forces a node to be honest. Read "the true
+      // population" as "what this node says the population is".
+      // Callers who need to trust it, rather than merely receive it,
+      // must still prove.
+      //
+      // Do not assume this field equals the offset you requested. It
+      // equals the offset only when the skip succeeded; when the walk
+      // ran out of groups first it is smaller, and that is the answer
+      // rather than an inconsistency.
       optional uint64 skipped = 2 [jstype = JS_STRING];
     }
 

packages/rs-dpp/Cargo.toml

@@ -71,7 +71,7 @@ strum = { version = "0.26", features = ["derive"] }
 json-schema-compatibility-validator = { path = '../rs-json-schema-compatibility-validator', optional = true }
 once_cell = "1.19.0"
 tracing = { version = "0.1.41" }
-grovedb-commitment-tree = { git = "https://github.com/dashpay/grovedb", rev = "a2791bbdca756d6a6113024aec48f09f7a33faa9", optional = true }
+grovedb-commitment-tree = { git = "https://github.com/dashpay/grovedb", rev = "0100cb833075621659a68ddd3696baecc98e55b8", optional = true }
 
 [dev-dependencies]
 tokio = { version = "1.40", features = ["full"] }