Prioritize matching provider operations

[kentcdodds]

Summary

• prevent unrelated high-vector builtins from outranking strongly named synthesized providers
• add bounded provider/entity affinity and semantic noun/state matching
• preserve exact package/value ordering with a single transitive scalar score

Production evidence

After #830, the Sonos query fell to 0.8s but still ranked repo_get_check_status first. This change targets that final relevance gap and asserts sonos_list_players plus sonos_get_player_status ahead of the observed competitor.

Validation

• npm run validate
• unit/worker tests: 1011 passed
• Playwright E2E: 17 passed
• focused ranking tests: 15 passed

System recap — extends existing primitives (medium risk)

Mode: recap · Base: main @ 4837c435 · Head: c4478a72

Classification: extends — MCP search scoring changes without adding a primitive.

Primitives touched

Primitive	Group	Impact
mcp-server	surfaces	extends — provider/entity affinity and semantic live-status scoring

System map

Search combines existing lexical/vector scores with bounded provider/entity affinity while preserving exact package and value matches.

Legend: green = composes (wiring only) · amber = extended by this PR · red = new primitive · gray = context (unchanged, included only when an edge crosses it).

flowchart LR
  mcpServer["mcp-server<br/>MCP endpoint (/mcp)"]:::extended
  mcpServer -->|"bounded provider/entity affinity"| mcpServer
  mcpServer -->|"noun/state semantic scoring"| mcpServer
  classDef touched fill:#1a7f37,color:#fff
  classDef extended fill:#9a6700,color:#fff
  classDef added fill:#cf222e,color:#fff
  classDef untouched fill:#57606a,color:#fff

Loading

Invariants

• per-user-isolation: scoring only uses the caller-scoped candidate set.
• compact-mcp-surface: no public schema or tool changes.

  

Summary by CodeRabbit

• Improvements
  • Improved capability search relevance across remote providers, MCP servers, OpenAPI sources, and repository tools.
  • Enhanced semantic understanding of terms, actions, and constraints to improve ranking quality.
  • Strengthened identity-aware reranking so remote provider identity is reflected and remote matches rank ahead of fallback results.
  • Exact package/value matches now rank above broader provider-derived matches.
• Tests
  • Updated and expanded search/ranking scenarios to cover identity handling, semantic matching behavior, and live-status ordering.

[coderabbitai]

📝 Walkthrough

Walkthrough

Search ranking now derives provider identities from capability sources, applies semantic identity affinity during reranking, preserves exact non-provider matches, and expands tests for package, value, builtin, and remote-provider ordering.

Changes

Provider-aware MCP search ranking

Layer / File(s)	Summary
Capability identity metadata packages/worker/src/mcp/tools/search.ts	Capability candidates now include identity fields and synthesized provider metadata derived from their source.
Semantic identity and term matching packages/worker/src/mcp/tools/search.ts	Identity terms are derived from intent, while constraint and action scoring use semantic concept matching.
Provider-affinity reranking packages/worker/src/mcp/tools/search.ts	rerankCandidates computes providerEntityAffinity, suppresses it for exact non-provider targets, and includes it in final score components.
Online ranking coverage packages/worker/src/mcp/tools/search.node.test.ts	Tests cover mixed package/value candidates, repository competitors, exact matches, remote provider identities, captured filters, and AI run counts.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CapabilitySpec
  participant capabilityMatchToCandidate
  participant rerankCandidates
  participant OnlineSearchTest
  CapabilitySpec->>capabilityMatchToCandidate: provide source and capability identity
  capabilityMatchToCandidate->>rerankCandidates: pass identity fields and provider key
  rerankCandidates->>OnlineSearchTest: return provider-aware ranked capabilities
  OnlineSearchTest->>OnlineSearchTest: assert exact and provider ordering

Loading

Possibly related PRs

• kentcdodds/kody#246: Both update MCP search intent and entity reranking logic.
• kentcdodds/kody#272: Both modify capability candidate conversion and scoring order.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title is concise and accurately captures the main change to prioritize provider operation matches in search ranking.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch cursor/optimize-kody-search-5f24

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit ac8f33f. Configure here.}

[github-actions]

🔎 Preview deployed: https://kody-pr-831.kody-a99.workers.dev

Worker: kody-pr-831
D1: kody-pr-831-db
KV: kody-pr-831-oauth-kv

Mocks:

• cloudflare: https://kody-pr-831-mock-cloudflare.kody-a99.workers.dev/__mocks (kody-pr-831-mock-cloudflare)

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/worker/src/mcp/tools/search.node.test.ts`:
- Around line 1860-1865: Anchor both ordering assertions around the rankedNames
checks for remote:living-room:get_player_status and
remote:living-room:list_players by first verifying each expected name is
present, then compare their positions with repo_get_check_status. Preserve the
existing ordering expectations while preventing missing entries from passing
through indexOf(-1).
- Around line 1703-1718: Update the ordering assertions around capabilityNames
to first assert that every referenced Sonos capability and repo_get_check_status
exists with a non-negative index (or equivalent known-present assertion), then
compare their ordering. Apply the same presence-anchoring approach to the
related assertions in the new test, while preserving the intended
Sonos-before-repo ordering checks.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 4559c924-57db-43e7-84a0-2f3c6194301b

📥 Commits

Reviewing files that changed from the base of the PR and between 4837c43 and ac8f33f.

📒 Files selected for processing (2)

• packages/worker/src/mcp/tools/search.node.test.ts
• packages/worker/src/mcp/tools/search.ts