feat(integrations): add retrieval-only external context search - #7586
Conversation
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
E2E test reportTested commit: Extension loading
Behavioral verification
Repository verification
Limitations
中文测试报告测试提交: 扩展加载
行为验证
仓库验证
限制
|
Code Coverage Summary
CLI Package - Full Text ReportCore Package - Full Text ReportFor detailed HTML reports, please see the 'coverage-reports-22.x-ubuntu-latest' artifact from the main CI run. |
Preserve provider timeout classification, reject ambiguous Mem0 statuses, release rejected response bodies, and clarify credential and workspace deployment boundaries. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Final broad auditAudited commit: The final implementation and proposal were re-audited across architecture boundaries, provider-side authorization, repository isolation, configuration and credential handling, HTTP failure paths, Mem0 semantics, Hook/MCP behavior, deployment scope, packaging, tests, compatibility, maintainability, and simpler alternatives. The audit fixed four concrete classes of problems:
Verification passed: 47 integration tests, 3 focused Core scheduler tests, compiled stdio MCP plus command Hook Fake Provider E2E, isolated Qwen extension link/user-disable/workspace-enable discovery, full repository build, typecheck, lint, lockfile check, package dry-run, and diff/format checks. The runtime critical-advisory threshold passed; npm reported existing non-critical repository dependency advisories, but no newly reachable issue was found in this stdio-only integration. After the last fix, two consecutive open-ended reviews of the final diff found no new clear, actionable issue. 中文审计说明审计提交: 最终实现与 proposal 已从架构边界、Provider 端鉴权、仓库隔离、配置与凭证处理、HTTP 故障路径、Mem0 语义、Hook/MCP 行为、部署 scope、打包、测试、兼容性、可维护性和更简单替代方案等维度重新完成全面审计。 本轮修复了四类明确问题:
验证全部通过:47 个 integration 测试、3 个 Core scheduler 定向测试、编译后 stdio MCP 与 command Hook 的真实进程 Fake Provider E2E、隔离 Qwen 环境中的 link/user-disable/workspace-enable 发现验证、全仓 build、typecheck、lint、lockfile check、package dry-run,以及 diff/format 检查。运行时 critical 漏洞阈值通过;npm 报告了仓库已有的非 critical 依赖公告,但未发现该 stdio-only integration 新增的可达问题。 最后一次修复后,又对最终 diff 完成两轮连续的无方向复审,没有发现新的明确可执行问题。 |
gwinthis
left a comment
There was a problem hiding this comment.
Architecture Review: COMMENT (C=0)
Summary
Large feature (+2980/-0) adding a Direct External Context Provider integration — a read-only context_search MCP tool with optional context_remember write tool, adapter-based provider support (Mem0 V3, Generic HTTP V1), and admin-owned security selectors.
Architecture Assessment
Strengths:
- Extension, not Core API — built from monorepo as a Qwen extension, not a Core API. Keeps Core unchanged, enables per-workspace opt-in.
- Admin-owned security selectors — model tool arguments cannot set tenant, user, repository, namespace,
app_id, or arbitrary filters. These are fixed by configuration, not model discretion. This is the key security boundary. - Trust boundary explicitly documented — one process, one configured provider, one repository corpus, no model-controlled security selectors. Explicitly does NOT claim trusted personal identity, per-user audit, per-document ACL, DLP, credential isolation from same-UID tools, retention/deletion governance, or multi-workspace serving.
- Untrusted JSON rendering — provider responses are rendered as bounded JSON under
UserPromptSubmit.additionalContext, not as trusted system instructions. - Fail-open automatic recall — provider failures skip recall and the Qwen request continues. Sanitized bounded query: fenced code and credential-shaped values are stripped before sending to provider.
- Write safety — Mem0 writes never retried or polled, report only
acceptedorunknown. Generic HTTP has no write protocol.PreToolUseconfirmation hook for shared-memory writes. YOLO mode required for non-interactive.
Scope observations (non-blocking):
- "Direct Profile v1" requires built monorepo checkout — not self-contained as npm tarball
- Credential visible to same-UID processes is acknowledged as a known limitation
- Decision boundary with governed Gateway/Orchestrator Profile (#7449) is clearly drawn
Pattern
Admin-owned security selectors for model-facing tools: When a tool exposes provider-specific parameters (tenant, user, namespace) that are authorization boundaries, fix them in admin-owned configuration — never let the model choose them. The model can choose query and limit; it cannot choose tenant_id or app_id. This turns the tool's parameter surface into a non-authorization boundary, keeping the trust model simple.
中文说明
架构评审:COMMENT (C=0)
概要
大型功能(+2980/-0):Direct External Context Provider 集成。只读 context_search MCP 工具 + 可选 context_remember 写入工具,适配器支持 Mem0 V3 和 Generic HTTP V1。
架构评估
- 扩展而非 Core API — 从 monorepo 构建为 Qwen 扩展
- 管理员拥有的安全选择器 — 模型不能设置 tenant/user/repository/namespace — 由配置固定
- 信任边界明确文档化 — 一个进程、一个 provider、一个仓库语料库
- 不可信 JSON 渲染 — provider 响应作为有界 JSON 渲染
模式
模型工具的管理员拥有安全选择器: 工具暴露的授权边界参数(tenant、user、namespace)由管理员配置固定,不让模型选择。模型只能选 query 和 limit。
— qwen3.7-max via Qwen Code /review
Limit Phase 1 to one provider-bound search tool, remove hooks and writes, and document the direct profile's actual permission and isolation boundaries. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
Phase 1 E2E validationThis report supersedes the earlier E2E comment for commit
Not validated here: a live production Mem0 project, Windows runtime behavior, Linux runtime behavior, or a standalone tarball deployment. The integration remains a private monorepo workspace. 第一阶段 E2E 验证本报告替代基于提交
本轮未验证:真实生产 Mem0 Project、Windows 运行时、Linux 运行时或独立 tarball 部署。该集成仍是 monorepo 内的私有 workspace。 |
Phase 1 scope and final auditThis comment supersedes the earlier final-audit comment for commit The PR now provides only on-demand external retrieval through The audit corrected several boundary and compatibility issues:
After the last tracked fix, two consecutive independent, broad diff audits found no remaining clear, actionable local code or documentation issue. The final tracked diff and its verification were reviewed together; 36/36 tests and all build, typecheck, lint, lockfile, packaging, and fake-provider checks pass. This is deliberately not the full governed design from #7449 or the complete roadmap in #7585. Identity, per-user or per-document authorization, DLP, writes, automatic recall, immutable approvals, compliance audit, and multi-workspace serving remain outside this PR. 第一阶段范围与最终审计本评论替代基于提交 本 PR 现在只通过 审计修正了以下边界与兼容性问题:
最后一次受跟踪修复完成后,两轮连续、独立、无方向的完整 diff 审计均未发现剩余的明确可执行本地代码或文档问题。最终 diff 与验证证据已一并复核;36/36 测试以及 build、typecheck、lint、lockfile、打包和 Fake Provider 检查全部通过。 这不是 #7449 的完整受管设计,也不是 #7585 全部路线图的实现。身份、逐用户或逐文档授权、DLP、写入、自动召回、不可绕过审批、合规审计和多 workspace serving 仍不在本 PR 范围内。 |
Pin the managed MCP source through an administrator-owned command-line configuration, document the Direct Profile trust boundary, and remove unused logging/runtime abstractions. Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
|
Final multi-route audit completed on Changes made from the audit:
Verification:
已在 本轮修正了受管 MCP 同名覆盖、 最终结果:External Context 34/34 测试、workspace 与仓库级 build/typecheck/lint、lockfile、格式、diff check、dry-run package、编译后 stdio MCP E2E 全部通过;架构/正确性、安全/协议、测试/故障/打包三路审计均为 CLEAN,最后一次修复后又取得两轮连续清洁架构复审。 |
doudouOUC
left a comment
There was a problem hiding this comment.
Reviewed by checking out the branch and running it, not just reading the diff. Verification I did: vitest run → 34/34 pass, tsc --noEmit clean, tsc --build clean, eslint integrations/external-context/src clean. So the stated test/build/lint evidence holds up.
The security envelope is the strong part of this PR and I did not find a hole in it: model input really is confined to query, redirect: 'manual' prevents credential forwarding, the response is bounded both by declared content-length and by streaming, invalid UTF-8 is rejected fatally, cancellation propagates, and no provider detail escapes into the tool result. The threat model in the README is unusually honest about what the Direct Profile does not give you.
What I did find are correctness bugs in the data path — the part that decides which retrieved results the model actually sees. I reproduced each one against the compiled dist/ with a loopback provider; the reproductions are in the inline comments. The theme is that both the parse layer and the render layer fail silently and destructively: valid content gets dropped or blanked, and the model receives items: [] or content: "" with no way to tell that from "the corpus genuinely has nothing." For a retrieval tool that is the worst failure mode, because the model will confidently answer as though no context exists.
The existing tests don't catch these because every fixture is small and well-formed — no null optional fields, no long documents, no escape-heavy content. Suggest adding cases for exactly those.
Grouping the findings by priority:
Should fix before merge (silent data loss):
parseItemFieldsrejects a whole item when an optional field is malformed —score: null, or acontent/title/uriover its cap. Reproduced: item vanishes, tool returns[].trimNewestItemToBudgetmixes units and emitscontent: ""items. Reproduced with escape-heavy content:[999, 811, 0, 0, 0].MAX_ITEM_CONTENT_CHARS * MAX_ITEMS = 5000 > MAX_RENDERED_CHARS = 4000is structurally unsatisfiable, so the tail item is always degraded.- Provenance
uriis the first field dropped under budget pressure — the field most needed for citation, whilescoreis never dropped.
Worth fixing (operability / Simplicity First):
5. Local query-validation failures are reported as provider failures.
6. Startup errors discard already-sanitized ConfigurationError messages, leaving administrators nothing to debug with.
7. The four-class provider error taxonomy has no non-test consumer.
Repo-level, needs maintainer sign-off: this adds a new top-level integrations/ tier to workspaces and to scripts/build.js, so every contributor's npm run build / test / typecheck now pays for a private integration that no shipped artifact consumes. That's a cross-cutting change to package.json + eslint.config.js + scripts/build.js + package-lock.json and per AGENTS.md deserves an explicit maintainer decision rather than riding along with the feature.
Details inline.
Co-authored-by: Qwen-Coder <qwen-coder@alibabacloud.com>
|
Addressed the latest review on commit
Verification on the exact commit:
|
Code Review — #7586
|
|
Thanks for the thorough review. I independently checked all four suggestions against
The Node-only |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reviewed — no blockers. 1 Suggestion-level finding(s) could not be anchored to a changed line and were dropped; nothing further to act on here.
— qwen3.7-max via Qwen Code /review
What this PR does
This PR adds Phase 1 of the private Direct External Context Provider integration for the narrow deployment profile where one interactive Qwen CLI process serves trusted collaborators and the external provider can issue a credential restricted to one intended corpus.
The integration exposes exactly one on-demand, retrieval-only MCP tool,
context_search({ query }). It has no hooks, automatic recall, write tools, personal memory, or runtime provider plugin system. Provider selection and all corpus selectors are fixed by administrator-controlled configuration before the MCP server starts; the model supplies only the search query.Two explicit adapters are included. Mem0 Platform V3 Search fixes
app_id, result count, threshold, and reranking in the adapter. Generic HTTP Search V1 sends only a normalized query and fixed result limit to/v1/context/search, allowing an existing knowledge base, RAG service, or enterprise search endpoint to provide the same normalized result contract.Requests use a bounded timeout without retries or caching, and client cancellation aborts the in-flight provider request. Redirects are rejected, HTTPS is required except for loopback development, provider responses are capped and strictly validated, and the final untrusted JSON envelope is bounded. Provider failures return a stable local error. Phase 1 emits no local per-request audit record and does not write queries, results, credentials, provider errors, or operation metadata to stderr.
The managed deployment pins the reviewed MCP command through an administrator-owned
--mcp-config, because a settings or project server with the same name would otherwise replace the extension manifest contribution while retaining the same permission-rule name. The managed launcher constructs the complete Qwen argument vector without passing through arbitrary caller arguments, starts from an administrator-approved environment, and supplies a launcher-specific system settings path that disables/cdand allows only the pinned search tool. The Node executable, checkout, dependency tree, provider configuration, and credential binding remain administrator-controlled and immutable for the session. The extension manifest remains available for local trusted trials but is not the managed source binding.The design documents the actual trust boundary. The provider-side credential, project, index, or corpus must enforce single-corpus access; Mem0
app_idand other client-side filters are classification rather than authorization. The Direct Profile requires the repository, its.envand.qwen/.envfiles, and same-UID code to be trusted. It does not provide DLP, credential or process isolation from the CLI user, trusted user identity, per-document ACL enforcement, non-bypassable confirmation, compliance audit, or protection from prompt injection in retrieved content. Deployments needing those controls use the governed Gateway/Orchestrator Profile in #7449.The repository lint configuration now applies the normal TypeScript and Vitest house rules to private integration workspaces, and the implementation removes the earlier single-use runtime wrapper and dead provider-binding metadata.
Why it's needed
Teams that already have repository-scoped memory or knowledge services need a low-deployment-cost way to retrieve that context from Qwen Code without changing Qwen Core or first operating the full governed gateway.
Exposing an unrestricted provider MCP server is too broad because model-controlled tenant, project, namespace, endpoint, or filter parameters can defeat the intended corpus boundary. This integration provides the minimum reviewed wrapper for trusted-team retrieval while deferring automatic recall, writes, personal identity, and policy enforcement whose authorization, privacy, latency, and lifecycle semantics require the governed profile or a separate phase.
Reviewer Test Plan
How to verify
external-contextservers in user settings and the repository MCP configuration, then start the managed profile. Confirm the command-line MCP entry wins and only the reviewedcontext_searchtool is exposed.QWEN_CODE_SYSTEM_SETTINGS_PATHonly at the managed profile settings. Confirm/cdis hidden and refused and the exact pinned search tool is allowed. Unmanaged Qwen sessions must not inherit this automatic allow rule.{"query":"normalized query","limit":5}at/v1/context/search, and Qwen receives bounded JSON underuntrusted_external_context.app_id,top_k: 5,threshold: 0.1, andrerank: false, and no add, update, or delete surface exists. Production configuration must use the fixed Mem0 API origin.External context search failed., cancellation aborts the provider request, there is no retry, and stderr remains empty of per-request data or metadata.Evidence (Before & After)
N/A. This is a non-UI private integration, tests, deployment examples, lint coverage, and design documentation.
Tested on
Environment (optional)
Node.js 22.22.3, npm 10.9.8, Qwen Code monorepo workspace. Verified 42/42 workspace tests with real loopback providers, workspace and repository build/typecheck/lint, lockfile validation, formatting, dry-run packaging, and a compiled stdio MCP fake-provider E2E with empty stderr.
Risk & Scope
permissions.askis bypassed by YOLO, and a mutable configuration path could change the corpus binding after an MCP restart; the profile is appropriate only when the provider credential is already restricted to one intended corpus and every managed input remains controlled for the session.Linked Issues
Related to #7585
Related to #7449
中文说明
本 PR 做了什么
本 PR 增加私有 Direct External Context Provider 集成的第一阶段,面向一个交互式 Qwen CLI 进程服务可信协作者,且外部 Provider 能够签发仅限一个目标语料库凭证的窄场景。
该集成只暴露一个按需、仅检索的 MCP 工具
context_search({ query })。它不包含 Hook、自动召回、写入工具、个人记忆或运行时 Provider 插件系统。Provider 选择和所有语料库 selector 都在 MCP Server 启动前由管理员控制的配置固定;模型只能提供搜索 query。本 PR 包含两个显式 Adapter。Mem0 Platform V3 Search 在 Adapter 中固定
app_id、结果数量、threshold 和 reranking。Generic HTTP Search V1 只向/v1/context/search发送归一化 query 和固定结果上限,使现有知识库、RAG 服务或企业搜索端点能够提供相同的归一化结果契约。请求使用有界超时,不重试也不缓存;客户端取消会中止正在进行的 Provider 请求。重定向会被拒绝;除 loopback 开发外必须使用 HTTPS;Provider 响应具有大小上限并经过严格校验;最终的非可信 JSON envelope 也受到边界限制。Provider 故障只返回稳定的本地错误。Phase 1 不产生本地逐请求审计记录,也不会向 stderr 写入 query、结果、凭证、Provider 错误或操作元数据。
受管部署通过管理员拥有的
--mcp-config固定经过审阅的 MCP 命令,因为同名 settings 或 project server 否则可以替换扩展 manifest 的贡献,同时继续使用相同的权限规则名。受管启动器完整构造 Qwen 参数,不透传任意调用方参数;从管理员批准的环境启动;并提供仅供该启动器使用的 system settings 路径,用于禁用/cd且只允许固定的搜索工具。Node 可执行文件、checkout、依赖树、Provider 配置和凭证绑定在会话期间都由管理员控制并保持不可变。扩展 manifest 仍可用于本地可信试用,但不是受管模式的来源绑定。设计文档明确记录真实信任边界。Provider 侧凭证、Project、Index 或 Corpus 必须强制单语料库访问;Mem0
app_id和其他客户端 filter 只是分类信息,不是鉴权。Direct Profile 要求仓库、其.env与.qwen/.env文件以及同 UID 代码可信。它不提供 DLP、对 CLI 用户的凭证或进程隔离、可信用户身份、逐文档 ACL 强制、不可绕过的确认、合规审计,也不能防止召回内容中的提示注入。需要这些控制的部署应使用 #7449 中的受管 Gateway/Orchestrator Profile。仓库 lint 配置现在会把常规 TypeScript 和 Vitest 规则应用到私有 integration workspace;实现同时删除了早期仅使用一次的 runtime wrapper 和无读取方的 Provider binding 元数据。
为什么需要
已经拥有仓库级记忆或知识服务的团队,需要一种低部署成本的方式从 Qwen Code 中检索这些上下文,同时不修改 Qwen Core,也不要求先运维完整的受管 Gateway。
直接暴露无限制的 Provider MCP Server 范围过大,因为模型可控的 tenant、project、namespace、endpoint 或 filter 参数可能破坏预期语料库边界。该集成为可信团队检索提供最小的经过审阅的封装,同时将自动召回、写入、个人身份和策略执行推迟到后续阶段或受管 Profile,因为这些能力的鉴权、隐私、延迟和生命周期语义需要单独设计。
Reviewer 测试计划
如何验证
external-contextserver,然后启动受管 Profile。确认命令行 MCP 入口胜出,并且只暴露经过审阅的context_search工具。QWEN_CODE_SYSTEM_SETTINGS_PATH指向对应 settings。确认/cd被隐藏并拒绝执行,且精确匹配的固定搜索工具被允许。非受管 Qwen 会话不得继承该自动 allow 规则。/v1/context/search收到的内容只有{"query":"normalized query","limit":5},同时 Qwen 在untrusted_external_context下收到有界 JSON。app_id、top_k: 5、threshold: 0.1和rerank: false,并且不存在 add、update 或 delete 接口。生产配置必须使用固定的 Mem0 API origin。External context search failed.,取消会中止 Provider 请求,不发生重试,且 stderr 中不包含逐请求数据或元数据。证据(Before & After)
N/A。这是非 UI 的私有集成、测试、部署示例、lint 覆盖和设计文档。
已测试平台
环境(可选)
Node.js 22.22.3、npm 10.9.8、Qwen Code monorepo workspace。已验证使用真实 loopback Provider 的 42/42 workspace 测试、workspace 与仓库级 build/typecheck/lint、lockfile 校验、格式检查、dry-run 打包,以及编译后 stdio MCP Fake Provider E2E,且 stderr 为空。
风险与范围
permissions.ask,可变配置路径还可能在 MCP 重启后改变语料库绑定;该 Profile 仅适用于 Provider 凭证已经限制到一个目标语料库且所有受管输入在会话期间都保持受控的场景。关联 Issue
关联 #7585
关联 #7449