feat(drive): support IN over pinned prefix properties in ranked and having-range queries

book/src/drive/document-ranked-trees.md

@@ -91,7 +91,7 @@ One asymmetry is worth knowing when authoring: **the meta-schema demands the lit
 
 Two structural rules, both enforced at contract-parse time in rs-dpp:
 
-- **No aggregating index on a compound ranked index's full prefix.** Ranked flags are allowed on compound indexes, with **per-prefix** semantics: a ranked `[identityId, class]` puts the indexed tree at each prefix value's terminal `class` property-name level — one ordered secondary per `identityId`, each ranking only that identity's `class` groups. There is deliberately no global cross-prefix ordering; the query surfaces require every leading property to be pinned by an equality `where` clause. The one shape that stays impossible — and is rejected per document type, where all indexes are visible (`validate_no_ranked_prefix_overlap`) — is a countable/summable index terminating at exactly the compound's leading prefix: its aggregating value trees would demand the `NonCounted` / `NotSummed` shell around the ranked terminal tree, and the storage layer structurally rejects any wrapper around an indexed tree, because the wrapper would neutralise the very aggregates the secondaries order by. (Drive's fail-closed guard behind the parse-time check is `INDEXED_INNER_UNWRAPPABLE`.) Only the exact `n-1` prefix conflicts: an aggregating index at a shorter prefix wraps a plain intermediate tree, and one extending past the ranked terminal lives inside its value trees — both supported.
+- **No aggregating index on a compound ranked index's full prefix.** Ranked flags are allowed on compound indexes, with **per-prefix** semantics: a ranked `[identityId, class]` puts the indexed tree at each prefix value's terminal `class` property-name level — one ordered secondary per `identityId`, each ranking only that identity's `class` groups. There is deliberately no global cross-prefix ordering; the query surfaces require every leading property to be pinned by a `where` clause — equalities, plus at most one `IN` that fans out across prefix branches and merges deterministically. The one shape that stays impossible — and is rejected per document type, where all indexes are visible (`validate_no_ranked_prefix_overlap`) — is a countable/summable index terminating at exactly the compound's leading prefix: its aggregating value trees would demand the `NonCounted` / `NotSummed` shell around the ranked terminal tree, and the storage layer structurally rejects any wrapper around an indexed tree, because the wrapper would neutralise the very aggregates the secondaries order by. (Drive's fail-closed guard behind the parse-time check is `INDEXED_INNER_UNWRAPPABLE`.) Only the exact `n-1` prefix conflicts: an aggregating index at a shorter prefix wraps a plain intermediate tree, and one extending past the ranked terminal lives inside its value trees — both supported.
 - **Non-unique indexes only.** `ranked aggregates are not supported on unique indexes: each group of a unique index contains at most one document, so there is nothing meaningful to rank`. Contested indexes are covered transitively — a contested index is unique by construction, so it hits the same check rather than needing its own.
 
 ### Version Gate
@@ -341,7 +341,7 @@ Note that the fixture puts each shape on its **own document type**. That's not a
 | Top / bottom K groups by sum of a property | `rankedSummable: true` on an index with `summable: "<prop>"` + `rangeSummable: true` |
 | Top / bottom K groups by average of a property | `rankedAverageable: true` on an index with `averageable: "<prop>"` + `rangeAverageable: true` (or the count+sum longhand) |
 | Two rankings on one index (e.g. by count *and* by average) | Both keywords. The tree is a PCPSIT carrying both axes in its TLV; you pay one secondary Merk per axis on every write. |
-| A ranking filtered by another property (`top 5 restaurants in London`) | A **compound ranked index** with the filter property leading: `[city, restaurantId]` with the ranked flags. Each city gets its own secondary; the query pins the prefix with an equality `where` (`WHERE city == "London" GROUP BY restaurantId ORDER BY <agg> DESC LIMIT 5`). Only equality pins — a range or `IN` on the prefix is rejected, and there is no cross-prefix (global) ordering on a compound ranked index. |
+| A ranking filtered by another property (`top 5 restaurants in London`) | A **compound ranked index** with the filter property leading: `[city, restaurantId]` with the ranked flags. Each city gets its own secondary; the query pins the prefix with an equality `where` (`WHERE city == "London" GROUP BY restaurantId ORDER BY <agg> DESC LIMIT 5`). Equality pins select one prefix; at most one pin may be an `IN` (2..=10 distinct elements, `null` legal for the absent-value prefix), which walks one secondary per element and merges by `(aggregate, encoded prefix, group key)` with per-branch proofs in one container — entries then carry `in_key`. A range operator on the prefix stays rejected, `OFFSET` is rejected together with `IN`, and there is still no global cross-prefix ordering beyond that merge. |
 | A ranking on a unique or contested index | Not available, and not meaningful: every group holds at most one document. |
 | Range aggregates without ranking (the 4.0 surface) | Just the `range*` flags. Ranking is strictly additive — adding it never changes what a range query returns. |
 | Nothing ranking-aware (default) | Don't set any `ranked*` flag. The terminal property-name tree keeps the type its range flags give it. |

packages/dapi-grpc/clients/drive/v0/nodejs/drive_pbjs.js

@@ -26727,6 +26727,7 @@ $root.org = (function() {
                                  * @property {number|Long|null} [count] RankedEntry count
                                  * @property {number|Long|null} [sum] RankedEntry sum
                                  * @property {number|null} [avg] RankedEntry avg
+                                 * @property {Uint8Array|null} [inKey] RankedEntry inKey
                                  */
 
                                 /**
@@ -26776,6 +26777,14 @@ $root.org = (function() {
                                  */
                                 RankedEntry.prototype.avg = 0;
 
+                                /**
+                                 * RankedEntry inKey.
+                                 * @member {Uint8Array} inKey
+                                 * @memberof org.dash.platform.dapi.v0.GetDocumentsResponse.GetDocumentsResponseV1.RankedEntry
+                                 * @instance
+                                 */
+                                RankedEntry.prototype.inKey = $util.newBuffer([]);
+
                                 // OneOf field names bound to virtual getters and setters
                                 var $oneOfFields;
 
@@ -26822,6 +26831,8 @@ $root.org = (function() {
                                         writer.uint32(/* id 3, wireType 0 =*/24).sint64(message.sum);
                                     if (message.avg != null && Object.hasOwnProperty.call(message, "avg"))
                                         writer.uint32(/* id 4, wireType 1 =*/33).double(message.avg);
+                                    if (message.inKey != null && Object.hasOwnProperty.call(message, "inKey"))
+                                        writer.uint32(/* id 5, wireType 2 =*/42).bytes(message.inKey);
                                     return writer;
                                 };
 
@@ -26868,6 +26879,9 @@ $root.org = (function() {
                                         case 4:
                                             message.avg = reader.double();
                                             break;
+                                        case 5:
+                                            message.inKey = reader.bytes();
+                                            break;
                                         default:
                                             reader.skipType(tag & 7);
                                             break;
@@ -26926,6 +26940,9 @@ $root.org = (function() {
                                         if (typeof message.avg !== "number")
                                             return "avg: number expected";
                                     }
+                                    if (message.inKey != null && message.hasOwnProperty("inKey"))
+                                        if (!(message.inKey && typeof message.inKey.length === "number" || $util.isString(message.inKey)))
+                                            return "inKey: buffer expected";
                                     return null;
                                 };
 
@@ -26966,6 +26983,11 @@ $root.org = (function() {
                                             message.sum = new $util.LongBits(object.sum.low >>> 0, object.sum.high >>> 0).toNumber();
                                     if (object.avg != null)
                                         message.avg = Number(object.avg);
+                                    if (object.inKey != null)
+                                        if (typeof object.inKey === "string")
+                                            $util.base64.decode(object.inKey, message.inKey = $util.newBuffer($util.base64.length(object.inKey)), 0);
+                                        else if (object.inKey.length >= 0)
+                                            message.inKey = object.inKey;
                                     return message;
                                 };
 
@@ -26982,14 +27004,22 @@ $root.org = (function() {
                                     if (!options)
                                         options = {};
                                     var object = {};
-                                    if (options.defaults)
+                                    if (options.defaults) {
                                         if (options.bytes === String)
                                             object.key = "";
                                         else {
                                             object.key = [];
                                             if (options.bytes !== Array)
                                                 object.key = $util.newBuffer(object.key);
                                         }
+                                        if (options.bytes === String)
+                                            object.inKey = "";
+                                        else {
+                                            object.inKey = [];
+                                            if (options.bytes !== Array)
+                                                object.inKey = $util.newBuffer(object.inKey);
+                                        }
+                                    }
                                     if (message.key != null && message.hasOwnProperty("key"))
                                         object.key = options.bytes === String ? $util.base64.encode(message.key, 0, message.key.length) : options.bytes === Array ? Array.prototype.slice.call(message.key) : message.key;
                                     if (message.count != null && message.hasOwnProperty("count")) {
@@ -27013,6 +27043,8 @@ $root.org = (function() {
                                         if (options.oneofs)
                                             object.value = "avg";
                                     }
+                                    if (message.inKey != null && message.hasOwnProperty("inKey"))
+                                        object.inKey = options.bytes === String ? $util.base64.encode(message.inKey, 0, message.inKey.length) : options.bytes === Array ? Array.prototype.slice.call(message.inKey) : message.inKey;
                                     return object;
                                 };