Skip to content

feat(memory): opt-in Qdrant scalar int8 quantization (F4.4 Q1) - #4187

Merged
diegosouzapw merged 1 commit into
release/v3.8.29from
feat/memory-qdrant-quantization
Jun 18, 2026
Merged

diegosouzapw merged 1 commit into
release/v3.8.29from
feat/memory-qdrant-quantization

Conversation

@diegosouzapw

Copy link
Copy Markdown
Collaborator

F4.4 / Q1 — Opt-in Qdrant scalar int8 quantization

Closes the Q1 half of the rodada4 F4.4 (vector quantization) gap — the last fully-local
implementation gap in the compression track (most of rodada4 was already done; audit
2026-06-18). Q2 (sqlite-vec int8) ships as a separate follow-up PR (recall-sensitive).

What

Adds an opt-in vector quantization mode to the Qdrant memory collection. Default is
none, so existing deployments are byte-identical — no silent behavior change.

When set to int8 (or binary):

  • new collections are created with a quantization_config
    (int8: scalar { type: int8, always_ram: true, quantile: 0.99 }; binary: { always_ram: true })
    → ~4× (int8) / up to ~32× (binary) less vector RAM;
  • searches send params.quantization.rescore: true so the original vectors refine the
    quantized shortlist (preserves recall).

Scope (surgical, opt-in)

  • src/lib/memory/qdrant.ts — QdrantQuantization type + QdrantConfig.quantization;
    pure helpers buildQuantizationConfig / searchQuantizationParams wired into
    ensureCollection (create body) and searchSemanticMemory (search params).
  • src/shared/schemas/qdrant.ts — QdrantQuantizationSchema enum in settings + update schemas.
  • src/app/api/settings/qdrant/route.ts — persist + expose qdrantQuantization.

Deliberately NOT in this PR (follow-ups)

  • Existing collections are left untouched — switching modes on a populated collection
    is a destructive recreate+reindex that must be an explicit UI action (never automatic,
    per the spec's "what NOT to do").
  • UI select (none/int8/binary) in MemorySkillsTab + the migration endpoint.
  • Q2 — sqlite-vec int8 local storage (separate PR; recall@K fixture validation).

Tests (TDD)

src/lib/memory/__tests__/qdrant-wiring.test.ts (vitest) — 9/9 green: default none;
mode parse (int8/binary/invalid→none); buildQuantizationConfig (none→undefined =
byte-identical body, int8→scalar, binary→binary); searchQuantizationParams (rescore only
when quantized). Adjacent suites green: qdrant UI card 5/5, qdrant integration 13/13.

Gates: typecheck:core / lint / check:cycles clean.

Add an opt-in vector quantization mode to the Qdrant memory collection.
Default is 'none' so existing deployments are byte-identical (no silent
change). When set to int8/binary, new collections are created with a
quantization_config and searches request rescore:true to preserve recall.

- qdrant.ts: QdrantQuantization type + QdrantConfig.quantization;
  pure helpers buildQuantizationConfig / searchQuantizationParams wired
  into ensureCollection (create body) and searchSemanticMemory (params).
- schemas/qdrant.ts: QdrantQuantizationSchema enum in settings + update.
- settings/qdrant route: persist + expose qdrantQuantization.

Existing collections are left untouched (recreate+reindex migration is a
documented UI follow-up). sqlite-vec int8 (Q2) is a separate follow-up PR.

TDD: 9/9 green (default none, mode parse, create-body helper, search
rescore helper). typecheck:core / lint / check:cycles clean.
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for vector quantization (none, int8, and binary) in Qdrant memory collections. It updates the API routes, configuration normalization, collection creation, and search logic to handle quantization parameters and rescoring, and adds corresponding Zod schema validation and unit tests. No review comments were provided, so there is no feedback to address.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@diegosouzapw
diegosouzapw merged commit 0df3131 into release/v3.8.29 Jun 18, 2026
4 checks passed
@diegosouzapw
diegosouzapw deleted the feat/memory-qdrant-quantization branch June 19, 2026 12:06
tkgo11 pushed a commit to tkgo11/OmniRoute that referenced this pull request Sep 23, 2026
…er-inference#4187)

Integrated into release/v3.8.29. Opt-in Qdrant scalar int8 quantization (F4.4 Q1); default 'none' keeps the create body byte-identical. Validated: vitest 9/9, file-size OK, lint clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant