feat(mcp): UIState recipe bridge, evidence matrix, KB PR, feedback - #130
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Sorry @tonythethompson, your pull request is larger than the review limit of 150000 diff characters
|
Warning Review limit reached
Next review available in: 9 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (37)
📝 WalkthroughWalkthroughThe change adds loopback Studio recipe tools, aggregate troubleshooting feedback, evidence-backed Olive compatibility validation, deterministic knowledge-base refresh automation, related server and UI integrations, tests, workflows, and documentation. ChangesMCP platform expansion
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 8✅ Passed checks (8 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
✨ Simplify code
Warning Review ran into problems🔥 ProblemsLinked repositories: Public OSS repositories can only analyze public repositories installed in this organization. Analyzed Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThis PR adds the Studio UIState-to-recipe bridge, MCP validation and feedback tools, evidence-backed compatibility data, and an automated reviewable knowledge-base refresh process.
Confidence Score: 4/5The PR does not yet appear safe to merge because the KB refresh workflow can still miss older human commits before replacing its refresh branch. The workflow starts from a shallow checkout, fetches the refresh branch without explicitly obtaining its complete ancestry, and only deepens the default branch. Its author scan can therefore omit older refresh-branch commits and proceed to rebuild and force-update a branch containing human work. Files Needing Attention: .github/workflows/kb-update.yml
|
| Filename | Overview |
|---|---|
| .github/workflows/kb-update.yml | Adds generation, validation, artifact upload, and refresh-PR branch management; the previously reported human-edit preservation issue remains outstanding. |
| src/server/routes/mcp.ts | Adds the loopback Studio recipe endpoint and integrates request rate limiting. |
| src/server/services/mcp/studioRecipeBridge.ts | Bridges partial UI state into the shared recipe evaluation pipeline. |
| olive-mcp-server/olive_mcp_server/tools/studio_recipe.py | Adds MCP clients for validating UI state and retrieving Studio-built recipes through a fixed loopback endpoint. |
| olive-mcp-server/olive_mcp_server/tools/feedback.py | Implements bounded local aggregate feedback persistence for troubleshooting entries. |
| olive-mcp-server/olive_mcp_server/tools/troubleshooting.py | Incorporates aggregate feedback as a capped adjustment to hybrid troubleshooting ranking. |
| olive-mcp-server/olive_mcp_server/knowledge_base/compatibility_matrix.json | Expands model and hardware compatibility records with Olive pass identifiers and provenance. |
| src/components/features/MCPDiagnosticCard.tsx | Adds optional thumbs feedback controls for diagnostics with stable matched entries. |
Reviews (14): Last reviewed commit: "fix(codefactor): split BatchProcessingPa..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
PR Summary by Qodofeat(mcp): UIState↔recipe bridge, evidence-backed KB, feedback loop
AI Description
Diagram
High-Level Assessment
Files changed (42)
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 139ec59e8a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "record_troubleshoot_feedback": ( | ||
| "olive_mcp_server.tools.feedback", | ||
| "record_troubleshoot_feedback", | ||
| ), |
There was a problem hiding this comment.
Registering record_troubleshoot_feedback here exposes a filesystem-writing, ranking-changing tool through the existing unauthenticated POST /api/mcp/tool proxy. Since server.ts binds Express to 0.0.0.0 and that proxy has no local-only middleware, any reachable LAN client can repeatedly increment the capped counters and persistently influence later troubleshooting matches, contrary to the tool's local-only contract. Gate this tool at the proxy or otherwise authenticate/restrict its callers.
Useful? React with 👍 / 👎.
| path = get_feedback_path() | ||
| with _lock: | ||
| store = _load_store_unlocked(path) |
There was a problem hiding this comment.
When two feedback submissions overlap, this lock does not protect the read-modify-write cycle because the HTTP MCP proxy launches a fresh Python process for every request. Both processes can load the same counters, increment independently, and atomically replace the file, causing one acknowledged vote to be lost; use a cross-process file lock or another atomic persistence mechanism around the load and replace.
Useful? React with 👍 / 👎.
| git fetch origin "${BRANCH}" --depth=50 | ||
| git fetch origin "${DEFAULT_BRANCH}" --depth=50 |
There was a problem hiding this comment.
In the inspected kb-update.yml refresh workflow, both branch tips are fetched with depth 50 before evaluating origin/${DEFAULT_BRANCH}..origin/${BRANCH}. If the default branch advances beyond that shallow window between refreshes, the histories have no common ancestor and git log includes older human-authored commits from the refresh branch, so the workflow falsely reports human edits and permanently skips bot updates. Fetch full history or deepen until a merge base is available before applying the author filter.
Useful? React with 👍 / 👎.
Expose loopback-only Studio recipe validation for MCP tools, evidence-backed compatibility matrix with CI pass enumeration, reviewable KB refresh PRs, and local aggregate troubleshooting feedback with UI thumbs controls.
139ec59 to
c391d6c
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Code Review by Qodo
1.
|
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
Qodo Fixer✅ Committed (3) · ☑ Fixed (3) Commits pushed directly to this PR — no separate fix PR opened. Process — 3 fixed
|
Add UiStatePatch so bridge partial passes type-check, enumerate Olive passes from installed olive_config.json without importing olive, require keyword evidence for explicit studio/olive troubleshoot domains, and fix MCPDiagnosticCard ref sync plus fetch mock typing. Co-authored-by: Cursor <cursoragent@cursor.com>
Compare matrix olive_pass claims to Olive 0.12.x registry keys case-insensitively, and allow duplicate diagnostic titles in ExecutionWorkspace history test. Co-authored-by: Cursor <cursoragent@cursor.com>
Treat QNNQuantization, OnnxModelOptimizer, and AzureMLQuantization as aliases or cloud-only claims so olive-pass-availability matches the pinned olive-ai 0.12.1 config. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Resolve MCP conflicts by keeping the studio-recipe bridge and loopback gates from #130 while adopting main's bodyGuard, tool allowlist, circuit breaker, and 503 unavailable handling. Preserve validation-error payloads that use status: "error" so they are not unwrapped as proxy failures. Co-authored-by: Anthony Thompson <github@trackdub.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Dismissed because a newer commit was pushed; Greptile will re-review the current head.
Merge duplicate @/types imports, replace bare except/pass with logged fallbacks, remount MCP feedback controls via key instead of setState in effects, and split high-complexity helpers flagged by CodeFactor. Co-authored-by: Anthony Thompson <github@trackdub.com>
Dismissed because a newer commit was pushed; Greptile will re-review the current head.
|
CodeFactor found multiple issues: Complex Method
|
Extract batch queue/SSE helpers, feedback UI/button class helpers, MCP diagnostic/feedback parsers, bridge enum/pass assigners, and compatibility-matrix claim validators so CodeFactor complex-method findings stay under threshold. Co-authored-by: Anthony Thompson <github@trackdub.com>
Dismissed because a newer commit was pushed; Greptile will re-review the current head.
|
CodeFactor found an issue: Complex Method It's currently on: |
|
@coderabbitai full review |
❌ Action failedReview failed. |
Extract queue-job helpers so CodeFactor no longer flags handleStartQueue as a complex method on PR 130. Co-authored-by: Anthony Thompson <github@trackdub.com>
Dismissed because a newer commit was pushed; Greptile will re-review the current head.
Summary
Implements docs/mcp-validation-kb-feedback.plan.md:
createDefaultPipelineState,projectUiStateToRecipeEvaluation,studioRecipeBridge, loopback-onlyPOST /api/mcp/studio-recipe+ rate limitvalidate_ui_state_recipe,get_recipe_for_ui_state(fixedOLIVE_STUDIO_API_URLloopback only)olive_pass+ provenance; expanded matrix;test_compatibility_matrix.py; CIolive-pass-availabilityjob (enumerate only, no optimization)kb-update.ymlruns generators+tests and opens labeledkb-refreshPR (no auto-merge)record_troubleshoot_feedback, bounded ranking in hybrid scorer, thumbs UI onMCPDiagnosticCardwired through ExecutionWorkspace + BatchProcessingPanelTest plan
pnpm exec vitest run --config vitest.config.ts src/lib/__tests__/recipePipeline.test.tspnpm exec vitest run --config vitest.integration.config.ts src/server/__tests__/routes.integration.test.tspnpm exec vitest run --config vitest.component.config.ts(MCPDiagnosticCard / EW / Batch)cd olive-mcp-server && python -m pytest tests/test_studio_recipe.py tests/test_compatibility_matrix.py tests/test_feedback.py tests/test_troubleshooting_hybrid.py tests/test_integration.py -qNotes
feature/mcp-validation-kb-feedback(original dirty branch preserved)