Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .agents/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,15 +29,15 @@ URLEncoder (@rapiq/codec-url-simple)
The `Query` AST is an **intermediate representation (IR)**. Every package plays exactly one role around it:

1. **Define & interact** — client-side construction (plan 012): `defineQuery<RECORD>(QueryBuildInput)` + per-parameter `define*` fragment factories desugar typed input (scalars → `eq`, bare arrays → `in` with `null` legal, `$`-operator objects, condition-helper trees) straight to the AST — schema-free, no parsing. Condition helpers (`parameter/filters/helpers/`, one per `FilterFieldOperator`; `in` → `inArray` since `in` is reserved) build `Filter`/`Filters` nodes directly. Queries compose immutably via `mergeQueries` (left-priority; fields/relations/sorts keyed by name, pagination per-property) and the `Filters` combinators: `merge()` = per-field replace, flat root-AND only (typed `MergeError`, `ErrorCode.FILTERS_NOT_FLAT`); `and()`/`or()` = wrap & inject (server scoping — injected conditions can't be displaced by later merges). `$and`/`$or` object keys stay reserved for the mongo parser dialect (`@rapiq/parser-mongo`). `QueryBuilder` was removed — `defineQuery` replaces it.
2. **Parse to IR** — parsers transform *dialect* input (a spec for how parameters are written: "simple" object shapes, "expression" strings) into the IR, validated against a `Schema`. Parsers are **transport-agnostic**: they read only the canonical `Parameter` keys (`fields`, `filters`, `pagination`, `relations`, `sort`) and know nothing about how the input crossed a process boundary.
2. **Parse to IR** — parsers transform *dialect* input (a spec for how parameters are written: "simple" object shapes, "expression" strings) into the IR, validated against a `Schema`. The `filters.validate` hook runs on every resolved/coerced leaf and may synchronously or asynchronously accept, replace or reject it without flattening compound structure. `parse()` keeps a strictly synchronous return type and throws `SCHEMA_VALIDATOR_ASYNC_REQUIRES_ASYNC_PARSER` on a Promise/thenable; `parseAsync()` awaits validators sequentially in tree order. Defaults apply if validation removes every leaf. Parsers are **transport-agnostic**: they read only the canonical `Parameter` keys (`fields`, `filters`, `pagination`, `relations`, `sort`) and know nothing about how the input crossed a process boundary.
3. **Consume the IR** — either interpret/walk it directly (`@rapiq/sql`, `@rapiq/typeorm` via visitors; `@rapiq/memory` compiles it into plain functions to evaluate in-memory objects/arrays), or…
4. **Transport the IR between application boundaries via a codec** — `@rapiq/codec-url-simple` is *one* such codec (HTTP URI scheme). The codec owns the complete wire format: the parameter wire names (`URLParameter`: `filter`, `page`, `include`, …) live **only** there, and `URLDecoder` is the boundary adapter — it accepts a raw query string *or* a pre-parsed query object (express `req.query`), maps wire names to canonical parameters and delegates to a schema-aware `SimpleParser`. App2 then works with the same IR. `@rapiq/codec-url-expression` is the sibling codec for the expression dialect (nested filter compounds in a single `filter=and(...)` param; the other four parameters share the simple wire machinery).

Codec rules settled during plan 007 (2026-07):

- **Subset law**: each dialect expresses only a subset of the IR — within it `decode(encode(q)) ≍ q` *modulo scalar type normalization* (the wire is untyped: `'5'` → `5`, `'true'` → `true`); outside it `encode` throws typed `FEATURE_UNSUPPORTED`/`OPERATOR_UNSUPPORTED` instead of silently changing semantics. The simple encoder enforces this pointwise: every emitted wire token is re-parsed and must decode back to the operator it came from.
- **Codec identity is in-band** (reverses the earlier out-of-band-only stance): `@rapiq/codec-url` ships `URLCodecRegistry` — encoding through it stamps a reserved `codec` parameter; decoding dispatches on it (absent → default simple, so plain clients keep working; unregistered name → typed `CodecError`, never a silent mis-decode). Each codec package also exports its identifier constant for out-of-band negotiation.
- **Schema-aware encode** validates by piping the plain-encoded output through the schema-bound decoder and re-encoding — parser-exact semantics by construction (drop by default, schema `throwOnFailure` opts into throwing); parameters absent from the input query are masked so schema defaults don't materialize onto the wire.
- **Schema-aware encode** validates by piping the plain-encoded output through the schema-bound decoder and re-encoding — parser-exact semantics by construction (drop by default, schema `throwOnFailure` opts into throwing); parameters absent from the input query are masked so schema defaults don't materialize onto the wire. The URL codecs and registry mirror the parser split with `encodeAsync()` / `decodeAsync()`; registry codec async hooks are optional so sync-only third-party codecs remain compatible.
- The shared filter-value wire grammar (`parseFilterScalar`/`parseFilterValue`/`parseFilterWireValue`/`serializeFilterValue`) lives in `@rapiq/parser-simple` (`parameter/filters/value.ts`) — the single source for scalar coercion and operator-marker parsing used by both parsers and the simple codec.

Placement rules that follow (settled during plan 006, don't re-litigate):
Expand All @@ -58,7 +58,7 @@ A `Schema<RECORD>` declares what a client *may* request per parameter (`allowed`

### 3. Dialects as small option objects, not subclasses

`@rapiq/sql` is database-agnostic; per-database behavior is injected via `DialectOptions` callbacks (`escapeField`, `paramPlaceholder`, `regexp`). Presets live in `packages/sql/src/dialect/`.
`@rapiq/sql` is database-agnostic; per-database behavior is injected via `DialectOptions` callbacks (`escapeField`, `paramPlaceholder`, `regexp`). Presets live in `packages/sql/src/dialect/`. Regex strings pass through unchanged for the database engine to interpret and validate; JavaScript `RegExp` values contribute their `source` and `ignoreCase` flag.

## Key Abstractions

Expand Down Expand Up @@ -163,7 +163,7 @@ type DialectOptions = {
};
```

`@rapiq/typeorm`: `TypeormAdapter` mirrors the SQL adapter but mutates a TypeORM query builder. The builder is bound at construction (`new TypeormAdapter({ queryBuilder: qb })`); `adapter.execute(query)` then walks the query and applies the accumulated state to that builder in a single call.
`@rapiq/typeorm`: `TypeormAdapter` mirrors the SQL adapter but mutates a TypeORM query builder. The builder is bound at construction (`new TypeormAdapter({ queryBuilder: qb })`); `adapter.execute(query)` then walks the query and applies the accumulated state to that builder in a single call. Filters use `andWhere`, preserving application-owned tenant/auth predicates already present on the builder. Relation aliases come from @rapiq/sql's shared, injective length-prefixed `buildRelationAlias` derivation; fields, filters, sorts and joins must all use that same function.

`@rapiq/memory`: compile-once functional visitors — the core visitor interfaces implemented with `R = compiled function` (`IFiltersVisitor<Predicate>`, `ISortsVisitor<Comparator>`, `IFieldsVisitor<Projector>`, `IPaginationVisitor<Slicer>`, `IQueryVisitor<CompiledQuery>`): `compileFilters(condition)` → `(input) => boolean`, `applyQuery(query, data)` → `{ data, total, pagination }`. The semantics contract (SQL parity for positive operators, complement law for negations — `ne`/`nin`/`not*` match null/missing —, same-element join-row binding for dotted paths over arrays, keep-tree projection where relations widen a sparse field selection) is settled in `.agents/plans/014-memory.md`; do not re-litigate decisions recorded there.

Expand Down
7 changes: 5 additions & 2 deletions .agents/conventions.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,11 @@ No AI-attribution trailers in commits, issues, or PRs (see AGENTS.md).

## Release Process

- **release-please** (`release-please-config.json`, manifest-driven) manages versions and changelogs across workspaces; currently in `prerelease: true` mode with `alpha` versioning, and it bumps internal peer dependency ranges automatically.
- Release workflow: `.github/workflows/release.yml`; CI runs on `develop`, `master`, `next`, `beta`, `alpha`.
- **release-please** (`release-please-config.json`, manifest-driven) manages all ten public workspaces as one linked version group. It is currently in `prerelease: true` / `beta` mode, emits component-qualified tags, and updates internal peer dependency ranges through the node-workspace plugin.
- `release-as: 2.0.0-beta.0` bootstraps the first v2 beta. Remove that one-time override immediately after the beta release PR is merged so subsequent betas increment normally.
- Every public package publishes with npm access `public` and dist-tag `beta`; a prerelease must never update npm's `latest` tag.
- The private `@rapiq/docs` workspace keeps its internal `@rapiq/*` build inputs in `devDependencies` with `*` ranges. This ensures clean installs always link the current workspaces across major/prerelease bumps without adding docs tooling to the production audit.
- Release workflow: `.github/workflows/release.yml` runs on `master`. After release-please creates releases it installs, builds, lints, runs coverage, uploads coverage, and only then publishes. General CI runs on `develop`, `master`, `next`, `beta`, `alpha`.
- Do not bump versions or edit `CHANGELOG.md` manually.

## References
Expand Down
11 changes: 9 additions & 2 deletions .agents/migration-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,14 @@ Resolution semantics:
10. Alias mappings whose target contains dots (`mapping: { realmName: 'realm.name' }`) resolve through relation traversal: every segment is walked (per-level relations gate + registry/schemaMapping lookup) and the leaf validates against the *related* schema instead of the root allow-list — matching what direct dotted input always did. The emitted field keeps the full mapped path. In the fields parser such aliases are requeued through the relation machinery (child `execute()` semantics apply) and duplicate field nodes are deduplicated by name. Relation traversal is bounded (32 levels): cyclic mapping/schemaMapping configurations yield a `schemaUnresolvable` verdict (or `keyPathInvalid` under `throwOnFailure`) instead of unbounded recursion.
11. `SimpleParser.parse` / `ExpressionParser.parse` without a schema no longer bind the parameter parsers to a manufactured empty schema — schemaless parsing is uniformly unconstrained (dotted keys survive), consistent with `URLDecoder`.

Known pre-existing issue (out of scope here, plan 006): `BaseParser.expandObject` self-references its accumulator for genuinely nested object input (e.g. filters `{ user: { name: 'x' } }`), causing infinite recursion; flat dotted keys are unaffected.
`BaseParser.expandObject` now creates a fresh child object for nested input. Earlier v2 builds self-referenced the parent accumulator for input such as `{ user: { name: 'x' } }`, causing infinite recursion.

## 2.0.0-beta hardening

- Filter schema `validate` hooks are synchronous and run for every parsed leaf in the simple, expression and Mongo dialects. They may return the original/replacement filter or `undefined` to reject it; compound structure is retained and defaults apply when every leaf is rejected.
- Expression filters reject every unmatched source character, preserve leading underscores and cap recursive compounds/negations at 32 levels. The Mongo parser applies the same traversal cap.
- `regex(field, pattern)` accepts `RegExp` or string consistently in memory and SQL backends; invalid string patterns throw typed `AdapterError`. Oracle now renders `REGEXP_LIKE` with `:n` placeholders instead of inherited PostgreSQL syntax.
- TypeORM filters append with `andWhere`, preserving caller-owned predicates. The default relation alias is now the injective length-prefixed `buildRelationAlias` (`realm` → `r5_realm`, `role.realm` → `r4_role_5_realm`) so underscores in relation names cannot collide with path separators.

## Public-API triage (plan 008, items 1+2)

Expand Down Expand Up @@ -62,5 +69,5 @@ Known pre-existing issue (out of scope here, plan 006): `BaseParser.expandObject
- **Empty `in`/`nin` lists**: `in(field, [])` renders `1 = 0` (matches nothing) and `nin(field, [])` renders `1 = 1` — previously the invalid SQL `field in()`.
- **sqlite preset** no longer inherits mysql's `regexp` callback (stock SQLite has no `REGEXP` function): anchored operators fall back to `LIKE`, the `regex` operator throws a typed `AdapterError`.
- `FiltersVisitor`: `visitFilterNotEndsWith`/`visitFilterNotContains` signatures used wrong operator type parameters (copy-paste); in/nin and the six anchored-operator methods now share `whereIn`/`whereAnchored` helpers.
- **Literal matching for anchored operators** (from PR #742 review): `createFilterRegexPattern` escapes regex metacharacters — the input is a filter value, not a regex. Previously `contains(name, 'a.b')` matched `axb` on regexp dialects (while the LIKE fallback matched literally) and values like `'('` threw a raw `SyntaxError`. The `regex` operator is unaffected (takes a real `RegExp`).
- **Literal matching for anchored operators** (from PR #742 review): `createFilterRegexPattern` escapes regex metacharacters — the input is a filter value, not a regex. Previously `contains(name, 'a.b')` matched `axb` on regexp dialects (while the LIKE fallback matched literally) and values like `'('` threw a raw `SyntaxError`. The `regex` operator is unaffected and interprets a `RegExp` or string as a real pattern.
- `notStartsWith` on regexp dialects now matches the empty string (`^(?!foo).*`, was `.+`), consistent with `NOT LIKE 'foo%'`.
10 changes: 5 additions & 5 deletions .agents/references/typeorm-extension.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ The de-facto v1 server-side integration: both authup and PrivateAIM/hub call rap
| `src/query/module.ts` `applyQuery(qb, input, options)` | `parseQuery` (rapiq v1) + apply each parse output to the query builder; returns parse output incl. pagination | SimpleParser.parse → `Query.accept(QueryVisitor)` → `TypeormAdapter.execute()` |
| `src/query/parameter/filters/module.ts` `transformParsedFilters` | v1 `FiltersParseOutput` (FLAT array of `{key, path, operator, value}`) → `{statement, binding}` list | @rapiq/sql `FiltersVisitor` + `FiltersBaseAdapter` |
| `applyFiltersTransformed` | one `Brackets` with `andWhere` per condition — **AND-only, no OR trees in v1** | v2 `Filters` compound (and/or) — strictly more expressive |
| `src/query/parameter/relations/module.ts` | `leftJoinAndSelect` per relation + `onJoin(key, alias, qb)` hook; nested keys via `parts.slice(-2)` | @rapiq/typeorm `RelationsAdapter` (currently `innerJoinAndSelect` — **join-type divergence**, see below) |
| `src/query/parameter/relations/module.ts` | `leftJoinAndSelect` per relation + `onJoin(key, alias, qb)` hook; nested keys via `parts.slice(-2)` | @rapiq/typeorm `RelationsAdapter` (left join by default, configurable `joinType`, `onJoin` supported) |
| `src/query/parameter/fields/module.ts` + `src/query/utils/alias.ts` | `query.select(fields.map(...))` with alias resolved per field via `getAliasForPath(relationsOutput, field.path)` | @rapiq/typeorm `FieldsAdapter` + shared `RelationsAdapter` (same cross-parameter alias idea) |
| `bindingKey?: (key) => string` option | customizable parameter binding names (`filter_user_id`) | no v2 equivalent (placeholder indexer only) |

Expand All @@ -18,7 +18,7 @@ The de-facto v1 server-side integration: both authup and PrivateAIM/hub call rap
1. **Secure-by-default opt-out**: `applyQuery` sets `options.fields/filters/relations/sort = false` (parameter fully disabled) unless `allowed`/`default` is explicitly defined (`isQueryOptionDefined`). rapiq v2 schemas treat *undefined* `allowed` as "any syntactically valid property" — **less restrictive**. Migrating consumers who omit a parameter's schema would silently open it up. v2 needs either matching opt-out semantics or a loud migration callout.
2. **Null semantics live in the adapter**: `value === null` → `IS NULL` / `IS NOT NULL`; a `null` inside an IN-array is spliced out and rewritten as `(key IN (...) OR key IS NULL)` (AND/NOT variant for NOT_IN). This is what makes authup's `filter: { realm_id: [id, null] }` realm-scoping pattern work. v2's sql/typeorm filter visitors must implement equivalent null handling or the single most common consumer filter breaks.
3. **v1 `~` LIKE is starts-with**: `transformParsedFilters` appends only a trailing `%` (`filter.value += '%'`). Client `~text` therefore means `text%`. v2's distinct STARTS_WITH/CONTAINS operators are richer, but codec/migration mapping of `~` must preserve starts-with semantics.
4. **Join type**: relations are applied with `leftJoinAndSelect`; current @rapiq/typeorm `RelationsAdapter` uses `innerJoinAndSelect`/`innerJoin` — inner joins drop rows with absent relations, a silent result-set change for every migrated endpoint.
5. **`onJoin` hook**: consumers (8+ hub repos, all authup repos) rely on it to `addGroupBy(`${alias}.id`)` because their root queries use `groupBy`. No v2 equivalent yet.
6. **Pagination is echoed back**: `applyQuery` returns the parse output; consumers destructure `{ pagination }` for response `meta`. v2 `execute()` currently returns nothing.
7. **Join aliasing diverges (v2 breaking fix, PR #760)**: typeorm-extension aliases nested joins by the path's *last segment* (`parts.slice(-2)`: `role.realm` joins as `realm`) — same-named relations on different branches collide. @rapiq/typeorm aliases by the *full path* with `.` → `_` (`role.realm` → `role_realm`, `buildRelationAlias` in @rapiq/sql), dedupes by that alias, and passes it to `onJoin`; custom derivations via `relations.relationAlias`. Migrated code that hard-codes leaf aliases (e.g. `andWhere('realm.name = ...')` on a nested join) must switch to the path-qualified alias.
4. **Join type**: relations are applied with `leftJoinAndSelect`; @rapiq/typeorm now preserves this as the default and exposes `relations.joinType: 'inner'` as an opt-in.
5. **`onJoin` hook**: consumers (8+ hub repos, all authup repos) rely on it to `addGroupBy(`${alias}.id`)` because their root queries use `groupBy`. @rapiq/typeorm exposes the equivalent `relations.onJoin(path, alias, queryBuilder)` hook.
6. **Pagination is echoed back**: `applyQuery` returns the parse output; consumers destructure `{ pagination }` for response `meta`. `TypeormAdapter.execute()` returns the applied `{ pagination: { limit, offset } }` shape.
7. **Join aliasing diverges (v2 breaking fix, PR #760; hardened before beta)**: typeorm-extension aliases nested joins by the path's *last segment* (`parts.slice(-2)`: `role.realm` joins as `realm`) — same-named relations on different branches collide. @rapiq/typeorm uses @rapiq/sql's injective, length-prefixed `buildRelationAlias` (`realm` → `r5_realm`, `role.realm` → `r4_role_5_realm`), dedupes by that alias, and passes it to `onJoin`; custom derivations via `relations.relationAlias`. Migrated code that hard-codes aliases must use the helper or the `onJoin` alias.
Loading
Loading