Handle pushed down predicates in query collection (#681)

* Handle pushed down predicates in query collection

Co-authored-by: Kevin De Porre <[email protected]>
Co-authored-by: Sam Willis <[email protected]>

* Deduplicate loadSubset requests in Query collection

* Unit test for deduplicating limited ordered queries in query collection

* GC tanstack query when subscription is unsubscribed

---------

Co-authored-by: Sam Willis <[email protected]>
Co-authored-by: Kevin De Porre <[email protected]>

Author: 3 people

.changeset/silent-trains-tell.md

@@ -0,0 +1,5 @@
+---
+"@tanstack/query-db-collection": patch
+---
+
+Handle pushed-down predicates

docs/reference/classes/DeduplicatedLoadSubset.md

@@ -0,0 +1,120 @@
+---
+id: DeduplicatedLoadSubset
+title: DeduplicatedLoadSubset
+---
+
+# Class: DeduplicatedLoadSubset
+
+Defined in: [packages/db/src/query/subset-dedupe.ts:34](https://github.com/TanStack/db/blob/main/packages/db/src/query/subset-dedupe.ts#L34)
+
+Deduplicated wrapper for a loadSubset function.
+Tracks what data has been loaded and avoids redundant calls by applying
+subset logic to predicates.
+
+## Param
+
+The options for the DeduplicatedLoadSubset
+
+## Param
+
+The underlying loadSubset function to wrap
+
+## Param
+
+An optional callback function that is invoked when a loadSubset call is deduplicated.
+                             If the call is deduplicated because the requested data is being loaded by an inflight request,
+                             then this callback is invoked when the inflight request completes successfully and the data is fully loaded.
+                             This callback is useful if you need to track rows per query, in which case you can't ignore deduplicated calls
+                             because you need to know which rows were loaded for each query.
+
+## Example
+
+```ts
+const dedupe = new DeduplicatedLoadSubset({ loadSubset: myLoadSubset, onDeduplicate: (opts) => console.log(`Call was deduplicated:`, opts) })
+
+// First call - fetches data
+await dedupe.loadSubset({ where: gt(ref('age'), val(10)) })
+
+// Second call - subset of first, returns true immediately
+await dedupe.loadSubset({ where: gt(ref('age'), val(20)) })
+
+// Clear state to start fresh
+dedupe.reset()
+```
+
+## Constructors
+
+### Constructor
+
+```ts
+new DeduplicatedLoadSubset(opts): DeduplicatedLoadSubset;
+```
+
+Defined in: [packages/db/src/query/subset-dedupe.ts:67](https://github.com/TanStack/db/blob/main/packages/db/src/query/subset-dedupe.ts#L67)
+
+#### Parameters
+
+##### opts
+
+###### loadSubset
+
+(`options`) => `true` \| `Promise`\<`void`\>
+
+###### onDeduplicate?
+
+(`options`) => `void`
+
+#### Returns
+
+`DeduplicatedLoadSubset`
+
+## Methods
+
+### loadSubset()
+
+```ts
+loadSubset(options): true | Promise<void>;
+```
+
+Defined in: [packages/db/src/query/subset-dedupe.ts:85](https://github.com/TanStack/db/blob/main/packages/db/src/query/subset-dedupe.ts#L85)
+
+Load a subset of data, with automatic deduplication based on previously
+loaded predicates and in-flight requests.
+
+This method is auto-bound, so it can be safely passed as a callback without
+losing its `this` context (e.g., `loadSubset: dedupe.loadSubset` in a sync config).
+
+#### Parameters
+
+##### options
+
+[`LoadSubsetOptions`](../../type-aliases/LoadSubsetOptions.md)
+
+The predicate options (where, orderBy, limit)
+
+#### Returns
+
+`true` \| `Promise`\<`void`\>
+
+true if data is already loaded, or a Promise that resolves when data is loaded
+
+***
+
+### reset()
+
+```ts
+reset(): void;
+```
+
+Defined in: [packages/db/src/query/subset-dedupe.ts:198](https://github.com/TanStack/db/blob/main/packages/db/src/query/subset-dedupe.ts#L198)
+
+Reset all tracking state.
+Clears the history of loaded predicates and in-flight calls.
+Use this when you want to start fresh, for example after clearing the underlying data store.
+
+Note: Any in-flight requests will still complete, but they will not update the tracking
+state after the reset. This prevents old requests from repopulating cleared state.
+
+#### Returns
+
+`void`

docs/reference/functions/isLimitSubset.md

@@ -0,0 +1,43 @@
+---
+id: isLimitSubset
+title: isLimitSubset
+---
+
+# Function: isLimitSubset()
+
+```ts
+function isLimitSubset(subset, superset): boolean;
+```
+
+Defined in: [packages/db/src/query/predicate-utils.ts:768](https://github.com/TanStack/db/blob/main/packages/db/src/query/predicate-utils.ts#L768)
+
+Check if one limit is a subset of another.
+Returns true if the subset limit requirements are satisfied by the superset limit.
+
+## Parameters
+
+### subset
+
+The limit requirement to check
+
+`number` | `undefined`
+
+### superset
+
+The limit that might satisfy the requirement
+
+`number` | `undefined`
+
+## Returns
+
+`boolean`
+
+true if subset is satisfied by superset
+
+## Example
+
+```ts
+isLimitSubset(10, 20) // true (requesting 10 items when 20 are available)
+isLimitSubset(20, 10) // false (requesting 20 items when only 10 are available)
+isLimitSubset(10, undefined) // true (requesting 10 items when unlimited are available)
+```

docs/reference/functions/isOrderBySubset.md

@@ -0,0 +1,42 @@
+---
+id: isOrderBySubset
+title: isOrderBySubset
+---
+
+# Function: isOrderBySubset()
+
+```ts
+function isOrderBySubset(subset, superset): boolean;
+```
+
+Defined in: [packages/db/src/query/predicate-utils.ts:713](https://github.com/TanStack/db/blob/main/packages/db/src/query/predicate-utils.ts#L713)
+
+Check if one orderBy clause is a subset of another.
+Returns true if the subset ordering requirements are satisfied by the superset ordering.
+
+## Parameters
+
+### subset
+
+The ordering requirements to check
+
+[`OrderBy`](../../@tanstack/namespaces/IR/type-aliases/OrderBy.md) | `undefined`
+
+### superset
+
+The ordering that might satisfy the requirements
+
+[`OrderBy`](../../@tanstack/namespaces/IR/type-aliases/OrderBy.md) | `undefined`
+
+## Returns
+
+`boolean`
+
+true if subset is satisfied by superset
+
+## Example
+
+```ts
+// Subset is prefix of superset
+isOrderBySubset([{expr: age, asc}], [{expr: age, asc}, {expr: name, desc}]) // true
+```

docs/reference/functions/isPredicateSubset.md

@@ -0,0 +1,44 @@
+---
+id: isPredicateSubset
+title: isPredicateSubset
+---
+
+# Function: isPredicateSubset()
+
+```ts
+function isPredicateSubset(subset, superset): boolean;
+```
+
+Defined in: [packages/db/src/query/predicate-utils.ts:801](https://github.com/TanStack/db/blob/main/packages/db/src/query/predicate-utils.ts#L801)
+
+Check if one predicate (where + orderBy + limit) is a subset of another.
+Returns true if all aspects of the subset predicate are satisfied by the superset.
+
+## Parameters
+
+### subset
+
+[`LoadSubsetOptions`](../../type-aliases/LoadSubsetOptions.md)
+
+The predicate requirements to check
+
+### superset
+
+[`LoadSubsetOptions`](../../type-aliases/LoadSubsetOptions.md)
+
+The predicate that might satisfy the requirements
+
+## Returns
+
+`boolean`
+
+true if subset is satisfied by superset
+
+## Example
+
+```ts
+isPredicateSubset(
+  { where: gt(ref('age'), val(20)), limit: 10 },
+  { where: gt(ref('age'), val(10)), limit: 20 }
+) // true
+```

docs/reference/functions/isWhereSubset.md

@@ -0,0 +1,47 @@
+---
+id: isWhereSubset
+title: isWhereSubset
+---
+
+# Function: isWhereSubset()
+
+```ts
+function isWhereSubset(subset, superset): boolean;
+```
+
+Defined in: [packages/db/src/query/predicate-utils.ts:21](https://github.com/TanStack/db/blob/main/packages/db/src/query/predicate-utils.ts#L21)
+
+Check if one where clause is a logical subset of another.
+Returns true if the subset predicate is more restrictive than (or equal to) the superset predicate.
+
+## Parameters
+
+### subset
+
+The potentially more restrictive predicate
+
+[`BasicExpression`](../../@tanstack/namespaces/IR/type-aliases/BasicExpression.md)\<`boolean`\> | `undefined`
+
+### superset
+
+The potentially less restrictive predicate
+
+[`BasicExpression`](../../@tanstack/namespaces/IR/type-aliases/BasicExpression.md)\<`boolean`\> | `undefined`
+
+## Returns
+
+`boolean`
+
+true if subset logically implies superset
+
+## Examples
+
+```ts
+// age > 20 is subset of age > 10 (more restrictive)
+isWhereSubset(gt(ref('age'), val(20)), gt(ref('age'), val(10))) // true
+```
+
+```ts
+// age > 10 AND name = 'X' is subset of age > 10 (more conditions)
+isWhereSubset(and(gt(ref('age'), val(10)), eq(ref('name'), val('X'))), gt(ref('age'), val(10))) // true
+```

docs/reference/functions/minusWherePredicates.md

@@ -0,0 +1,73 @@
+---
+id: minusWherePredicates
+title: minusWherePredicates
+---
+
+# Function: minusWherePredicates()
+
+```ts
+function minusWherePredicates(fromPredicate, subtractPredicate): 
+  | BasicExpression<boolean>
+  | null;
+```
+
+Defined in: [packages/db/src/query/predicate-utils.ts:338](https://github.com/TanStack/db/blob/main/packages/db/src/query/predicate-utils.ts#L338)
+
+Compute the difference between two where predicates: `fromPredicate AND NOT(subtractPredicate)`.
+Returns the simplified predicate, or null if the difference cannot be simplified
+(in which case the caller should fetch the full fromPredicate).
+
+## Parameters
+
+### fromPredicate
+
+The predicate to subtract from
+
+[`BasicExpression`](../../@tanstack/namespaces/IR/type-aliases/BasicExpression.md)\<`boolean`\> | `undefined`
+
+### subtractPredicate
+
+The predicate to subtract
+
+[`BasicExpression`](../../@tanstack/namespaces/IR/type-aliases/BasicExpression.md)\<`boolean`\> | `undefined`
+
+## Returns
+
+  \| [`BasicExpression`](../../@tanstack/namespaces/IR/type-aliases/BasicExpression.md)\<`boolean`\>
+  \| `null`
+
+The simplified difference, or null if cannot be simplified
+
+## Examples
+
+```ts
+// Range difference
+minusWherePredicates(
+  gt(ref('age'), val(10)),      // age > 10
+  gt(ref('age'), val(20))       // age > 20
+) // → age > 10 AND age <= 20
+```
+
+```ts
+// Set difference
+minusWherePredicates(
+  inOp(ref('status'), ['A', 'B', 'C', 'D']),  // status IN ['A','B','C','D']
+  inOp(ref('status'), ['B', 'C'])             // status IN ['B','C']
+) // → status IN ['A', 'D']
+```
+
+```ts
+// Common conditions
+minusWherePredicates(
+  and(gt(ref('age'), val(10)), eq(ref('status'), val('active'))),  // age > 10 AND status = 'active'
+  and(gt(ref('age'), val(20)), eq(ref('status'), val('active')))   // age > 20 AND status = 'active'
+) // → age > 10 AND age <= 20 AND status = 'active'
+```
+
+```ts
+// Complete overlap - empty result
+minusWherePredicates(
+  gt(ref('age'), val(20)),     // age > 20
+  gt(ref('age'), val(10))      // age > 10
+) // → {type: 'val', value: false} (empty set)
+```