perf(search): CCIP pgvector HNSW index (#626)

[hmjn023]

概要

Issue #626 の CCIP 類似検索を pgvector の HNSW index と構造化計測ログで改善します。

変更内容

• ccip_embeddings.embedding に cosine distance 用 HNSW index を追加
• DB候補検索、media再取得、Rust再ランキングの所要時間と件数を構造化ログへ記録
• EXPLAIN ANALYZE、hnsw.ef_search、p50/p95、LanceDB比較のベンチマーク手順を文書化
• Drizzle migration と snapshot を更新

検証

• server / root typecheck 成功
• CCIP service unit tests: 3 passed
• PostgreSQL CCIP vector store integration test: 1 passed
• git diff --check 成功

本番相当の約88k vectorsでの実測値は、利用可能なDB接続がないため未取得です。

Fixes #626

Summary by CodeRabbit

• 改善

  • 類似コンテンツ検索にベクトル検索用インデックスを追加し、大量データ環境での検索性能向上を図りました。
  • 候補取得、メディア検索、再ランキングの処理時間や件数を記録できるようになり、検索性能の確認と改善がしやすくなりました。
  • 候補が見つからない場合は、不要な処理を行わず空の検索結果を返します。

• ドキュメント

  • 類似検索のベンチマーク手順と、検索性能を比較・分析する方法を追加しました。

[coderabbitai]

Warning

Review limit reached

@hmjn023, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 48 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d778df09-4d12-4b7e-ba0d-2145f944e4e6

📥 Commits

Reviewing files that changed from the base of the PR and between 19c331c and e7d0c23.

📒 Files selected for processing (5)

• apps/server/drizzle/0019_purple_devos.sql
• apps/server/src/application/services/ccip-vector-service.ts
• apps/server/src/infrastructure/ai/postgres-ccip-vector-store.ts
• apps/server/src/infrastructure/bootstrap.ts
• docs/design/ccip-pgvector-benchmark.md

📝 Walkthrough

Walkthrough

CCIPのpgvector検索にコサイン距離用HNSWインデックスを追加し、候補取得・media取得・Rust再ランキングの実行時間と件数をログ出力する計測を導入した。あわせてDrizzleメタデータとベンチマーク手順を更新した。

Changes

CCIP pgvector検索

Layer / File(s)	Summary
コサイン検索用HNSWインデックス packages/db/src/schema.ts, apps/server/drizzle/0019_purple_devos.sql, apps/server/drizzle/meta/*	ccip_embeddings.embedding に vector_cosine_ops を使用するHNSWインデックスを追加し、m=16、ef_construction=64 を設定した。Drizzleのスナップショットとジャーナルも更新した。
検索処理の計測ログ packages/application/src/services/ccip-vector-service.ts, apps/server/src/application/services/ccip-vector-service.ts, apps/server/src/infrastructure/ai/postgres-ccip-vector-store.ts	候補件数、limit、mediaSourceフィルタ、media取得時間、Rust再ランキング時間、候補検索時間を任意のILoggerへ出力するよう変更した。
ベンチマーク手順 docs/design/ccip-pgvector-benchmark.md	EXPLAIN (ANALYZE, BUFFERS)、hnsw.ef_search比較、p50/p95集計、LanceDBとの比較条件を記録した。

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• hmjn023/solid-imager#559: 類似検索ハンドラの実行時間と結果をログ出力する変更。
• hmjn023/solid-imager#622: PostgresCcipVectorStore とccip_embeddingsによるpgvector検索を導入した変更。

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	タイトルはCCIP向けpgvector HNSW索引追加という主要変更を簡潔に表しています。
Linked Issues check	✅ Passed	HNSW索引、構造化計測ログ、ベンチマーク手順の追加で、#626の主要要件に合致しています。
Out of Scope Changes check	✅ Passed	変更は索引追加・計測・ベンチ文書化に限られ、#626の目的から逸脱した内容は見当たりません。
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/issue-626-ccip-pgvector-hnsw

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/server/drizzle/0019_purple_devos.sql`:
- Line 1: Update the idx_ccip_embeddings_embedding_cosine HNSW index creation to
use CREATE INDEX CONCURRENTLY, and configure the migration/deployment procedure
to execute it outside a transaction.

In `@apps/server/src/application/services/ccip-vector-service.ts`:
- Line 6: Remove the direct infrastructure logger import from the CCIP vector
service factory and preserve the dependency direction by accepting an ILogger
through composition-root injection. Update the factory and its callers to pass
the injected logger, or move the factory to infrastructure/bootstrap only if
that better matches its responsibilities.

In `@packages/db/src/schema.ts`:
- Around line 275-277: Update packages/db/src/schema.ts lines 275-277 around
embeddingCosineIndex to use a filtered-vector-search design that reliably
returns the requested count for model/version filters, such as an appropriate
partial index, partitioning, or iterative scan configuration. Update
docs/design/ccip-pgvector-benchmark.md lines 12-32 to compare cases including
source, exact-search differences, returned counts, and whether iterative scan is
enabled.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 871b8950-9aea-44ef-a319-e80e273398ee

📥 Commits

Reviewing files that changed from the base of the PR and between 43e121d and 19c331c.

📒 Files selected for processing (8)

• apps/server/drizzle/0019_purple_devos.sql
• apps/server/drizzle/meta/0019_snapshot.json
• apps/server/drizzle/meta/_journal.json
• apps/server/src/application/services/ccip-vector-service.ts
• apps/server/src/infrastructure/ai/postgres-ccip-vector-store.ts
• docs/design/ccip-pgvector-benchmark.md
• packages/application/src/services/ccip-vector-service.ts
• packages/db/src/schema.ts