Skip to content
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions book/src/drive/ranked-index-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ SELECT avg(grade) FROM review

**This replaced a non-SQL spelling that never shipped.** An earlier draft put the ranking on the right of a `HAVING` clause — `HAVING avg(grade) IN TOP(3)`, with `TOP` / `BOTTOM` / `MAX` / `MIN` as cross-group primitives. It was removed before release rather than deprecated. The deliberate call: SQL conformance beats a bespoke primitive. Every client author already knows `ORDER BY … LIMIT`; nobody knows `IN TOP(n)`, and the two express exactly the same thing. The retired spelling also had a rough edge the SQL one simply does not have — `= MAX` means *every* group tied at the extreme, which a bounded read cannot prove, so `MAX` / `MIN` had to be permanently refused. `ORDER BY <agg> DESC LIMIT 1` is positional and has no such ambiguity.

`HAVING` survives as what it is in SQL: a boolean per-group predicate. It is not yet evaluated (every non-empty `having` is `Unsupported`), and it cannot currently be combined with a ranking `ORDER BY` — the ranked executor reads a pre-sorted secondary and has no way to drop groups from the middle of that walk.
`HAVING` survives as what it is in SQL: a boolean per-group predicate — and since protocol v14 it is evaluated. A grouped aggregate carrying exactly one `having` clause that bounds the selected aggregate (`GROUP BY hashtag HAVING count(*) > 100 LIMIT 100`) is served as a value-bounded range read of the same axis secondary the ranking walks, with the same completeness-proving envelope. An `ORDER BY` naming the selected aggregate may ride along to set the walk direction (`HAVING avg(grade) > 80 ORDER BY avg(grade) DESC LIMIT 5` — the best matches first); what a `having` request cannot carry is rank-window pagination (`OFFSET`, `starting_rank`), because a value-bounded page has no rank base — its continuation is "tighten the bound past the last value seen". The grammar's v1 boundaries: one clause only, on the aggregate the select projects, with a contiguous-range operator (`=`, `>`, `>=`, `<`, `<=`, `BETWEEN` variants; `!=` and `IN` are non-contiguous and refused).
Comment thread
QuantumExplorer marked this conversation as resolved.
Outdated

## The Restaurants Contract

Expand Down Expand Up @@ -666,7 +666,7 @@ Everything below is rejected *before* any grovedb work, and most of it is mirror
| **`start_at` / `start_after`** — `InvalidLimit` | The cursor names a document id, but a ranked walk iterates an aggregate-ordered keyspace in which document ids do not appear. |
| **`order_by` naming anything but the selected aggregate**, or more than one clause — `InvalidParameter` | The single ordering clause *is* the ranking, and the secondary is sorted by one aggregate only. An ordering on the `GROUP BY` property, on an unrelated field, or a second tie-break clause names an order the secondary cannot produce. Accepting and silently ignoring it is the one genuinely dangerous option. Use the aggregate's own name (`$count` for `COUNT(*)`), or flip `ASC` ↔ `DESC` to reverse the ranking. |
| **`group_by` with ≠ 1 property** — `InvalidParameter` | Ranked indexes are single-property, so there is no compound grouping to rank over. |
| **any non-empty `having`** — `Unsupported` | `HAVING` is a boolean per-group predicate and is not evaluated at any protocol version. It also cannot combine with a ranking `ORDER BY`: the ranked executor reads a pre-sorted secondary and has no way to drop groups from the middle of that walk. |
| **`having` that isn't one contiguous bound on the selected aggregate** | A grouped single-clause `having` bounding the selected aggregate is **served** since protocol v14 — it routes to the having-range executor, a value-bounded range read of the same axis secondary (see the `HAVING` paragraph above). What stays rejected: multiple clauses (a second predicate needs a per-candidate post-check no executor performs), a clause on a different aggregate than the select projects (same reason), non-contiguous operators (`!=`, `IN`), `having` without `group_by` (a single implicit group is a plain aggregate the client can bound itself), and `OFFSET` / `start_at` alongside `having` (a value-bounded page has no rank base; continuation is by tightening the bound). Protocol v13 and earlier reject every non-empty `having` unchanged. |
| **no `order_by` at all, on a grouped aggregate** — routed elsewhere | Without an ordering this is a plain grouped aggregate, not a ranking; the caller wanted the `DocumentSplitCounts` / `DocumentSplitSums` / `DocumentSplitAverages` surface. |
| **`COUNT(field)`** (non-`*`) — `Unsupported`; **`SUM` / `AVG` with an empty field** — `InvalidParameter` | The Count axis ranks group cardinality and takes no field; the Sum and Avg axes rank the property the index accumulates and require it. |
| **`limit` unset, `0`, or `> 100`** — `InvalidLimit` | A ranking with no `n` has no size, and `LIMIT 0` selects nothing. The ceiling is a **hard limit, not a clamp**, because `k` is echoed in the proof envelope and re-checked by the verifier — a silent clamp would produce a proof the client's own reconstruction rejects. |
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

56 changes: 37 additions & 19 deletions packages/dapi-grpc/protos/platform/v0/platform.proto
Original file line number Diff line number Diff line change
Expand Up @@ -703,12 +703,21 @@ message GetDocumentsRequest {
// before release rather than deprecated, because it invented
// non-SQL grammar for something SQL already expresses.
//
// **`HAVING` cannot yet combine with an aggregate `ORDER BY`.**
// The ranked executor reads a pre-sorted per-axis secondary and
// has no way to drop groups from the middle of that walk, so a
// request carrying both a non-empty `having` and a ranking
// `order_by` is rejected with `Unsupported` rather than served
// with one of the two silently ignored.
// **From protocol v14 a single `HAVING` clause is served as a
// bounded range read** (having-range mode): `SELECT <agg> GROUP BY
// p HAVING <agg> <op> <value> [ORDER BY <order-key> ASC|DESC]
// LIMIT n` answers from the same per-axis secondary as ranked
// mode, on an index declaring the matching ranked axis. The
// clause's aggregate must be the selected aggregate, the operator
// must describe one contiguous range (`NOT_EQUAL` / `IN` are
// rejected), and the optional `ORDER BY` picks the walk direction
// using the same order-key spelling as ranked mode: `f` for
// `SUM(f)` / `AVG(f)`, the `$count` sentinel for `COUNT(*)` —
// never an explicit `OrderClause.aggregate` target, which is
// rejected. See the supported-shape table on
// `GetDocumentsRequestV1`. On protocol v13 and earlier every
// non-empty `having` stays rejected with `Unsupported`, exactly as
// before.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
//
// The operator set mirrors `WhereOperator` minus `STARTS_WITH`
// (prefix matching has no natural meaning against a scalar
Expand Down Expand Up @@ -849,12 +858,16 @@ message GetDocumentsRequest {
// It returns `ResultData.ranked`. See `order_by` and the
// supported-shape table below.
//
// `having` is a boolean per-group predicate and is **still**
// `Unsupported` at every protocol version, ranked mode or not
// (`"HAVING clause is not yet implemented"`). It carries no ranking
// spelling: an earlier draft put cross-group ranking on the right of
// a `HAVING` (`HAVING AVG(grade) IN TOP(5)`) and that grammar was
// removed before release in favour of `ORDER BY` + `LIMIT`.
// **Having-range mode** is served from protocol v14: a single
// `having` clause whose aggregate is the selected aggregate turns
// the request into a bounded range read over the same per-axis
// secondary ranked mode walks, answered in `ResultData.ranked`.
// On protocol v13 and earlier every non-empty `having` is rejected
// (`"HAVING clause is not yet implemented"`). `having` carries no
// ranking spelling: an earlier draft put cross-group ranking on the
// right of a `HAVING` (`HAVING AVG(grade) IN TOP(5)`) and that
// grammar was removed before release in favour of `ORDER BY` +
// `LIMIT`. See the supported-shape table below.
//
// **Supported shapes** (everything else rejects with a typed
// `QuerySyntaxError::Unsupported` so callers can detect un-wired
Expand Down Expand Up @@ -883,8 +896,12 @@ message GetDocumentsRequest {
// - exactly one `group_by` property, exactly one `order_by` clause naming the select's aggregate (`f` for `SUM(f)` / `AVG(f)`, the `$count` sentinel for `COUNT(*)`), a `limit` in `1 ..= 100`, an optional `offset`, and no `where` / `having` / `start_at`, on an index declaring the matching `rankedCountable` / `rankedSummable` / `rankedAverageable` axis → ranked executor, answered in `ResultData.ranked`.
// - `DESC` is the "top n" reading (walk the axis from the largest aggregate down), `ASC` the "bottom n" reading. Worked example: `SELECT AVG(grade) GROUP BY restaurantId ORDER BY grade DESC LIMIT 1 OFFSET 4` is the 5th-best restaurant.
//
// `select=<COUNT(*)|SUM(f)|AVG(f)>, group_by=[p], having=[<the selected aggregate> <op> <value>]` (protocol v14+) — **having-range mode**:
// - exactly one `group_by` property, exactly one `having` clause whose aggregate is the select's aggregate, an operator describing one contiguous range (`EQUAL`, `GREATER_THAN[_OR_EQUALS]`, `LESS_THAN[_OR_EQUALS]`, `BETWEEN*`; `NOT_EQUAL` / `IN` rejected), a `limit` in `1 ..= 100`, an optional `order_by` naming the same aggregate (walk direction; ascending by default), and no `where` / `offset` / `start_at` / `start_after`, on an index declaring the matching ranked axis → having-range executor, answered in `ResultData.ranked`.
// - no offset or cursor pagination: a page cut at `limit` continues only by tightening the bound past the last *distinct* aggregate value seen; a cut inside a tie (several groups sharing the boundary aggregate) cannot be continued, so size `limit` above the widest expected tie.
//
// **Rejected shapes** (return `Unsupported`):
// - any non-empty `having`, at every protocol version.
// - any non-empty `having` on protocol v13 and earlier; at v14+, any `having` shape outside having-range mode above (multiple clauses, an aggregate other than the select's, `NOT_EQUAL` / `IN`, or a carried `where` / `offset` / cursor).
// - at v14+: a ranked-shaped request carrying a `where` clause, a `start_at` / `start_after` cursor, more than one `order_by`, or an `order_by` naming anything but the selected aggregate.
// - `select=DOCUMENTS` with non-empty `group_by`.
// - `select=COUNT` with `group_by` on a field that is not constrained by an `In` or range where clause.
Expand Down Expand Up @@ -1084,12 +1101,13 @@ message GetDocumentsRequest {
// `HavingClause` / `HavingAggregate` for the operator and
// aggregate-function catalogs.
//
// **Every non-empty `having` is rejected**, at every protocol
// version, with `Unsupported("HAVING clause is not yet
// implemented")`. The wire shape ships ahead of evaluation so
// callers can construct full `HAVING COUNT(*) > 5 AND
// SUM(amount) > 100` requests in their builders, and so the
// capability can land without another version bump.
// **From protocol v14 a single clause is served** as a bounded
// range read — having-range mode; see the message-level
// supported-shape table. On v13 and earlier every non-empty
// `having` is rejected with `Unsupported("HAVING clause is not
// yet implemented")`. Multi-clause `HAVING COUNT(*) > 5 AND
// SUM(amount) > 100` requests can still be constructed on the
// wire, but stay rejected until a multi-clause evaluator lands.
//
// **`having` does not express ranking.** "The n highest-scoring
// groups" is `ORDER BY <the selected aggregate> DESC LIMIT n`
Expand Down
Loading
Loading