feat(slack): make Slack agent proactive about spawning workspaces#4660
Conversation
|
Ready to review this PR? Stage has broken it down into 1 individual chapter for you:
Chapters generated by Stage for commit c815024 on May 17, 2026 1:27am UTC. |
|
Capy auto-review is paused for this organization because the monthly auto-review limit has been reached. Increase the limit or turn it off in billing settings to resume automatic reviews. |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
✅ Files skipped from review due to trivial changes (1)
📝 WalkthroughWalkthroughThis PR updates the Slack agent's SYSTEM_PROMPT to a Superset-focused assistant: it emphasizes using ChangesSlack Agent Instruction Set
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint skipped: no ESLint configuration detected in root package.json. To enable, add Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThis PR rewrites the static
Confidence Score: 3/5The change is prompt-only, but the new spawning guidance names v2-specific tools that don't exist on the v1 code path, and the bias-toward-action instruction amplifies the impact of that mismatch on every code-change request routed through v1. The new sections hard-reference four tool names absent on the v1 MCP path; with the bias-toward-action instruction now in place, the model will actively try to spawn for any code-change request and on v1 will either error out or appear to comply without actually doing anything. apps/api/src/app/api/integrations/slack/events/utils/run-agent/run-agent.ts — the v1/v2 tool-name mismatch in the new prompt sections warrants a second look before merging.
|
| Filename | Overview |
|---|---|
| apps/api/src/app/api/integrations/slack/events/utils/run-agent/run-agent.ts | Static SYSTEM_PROMPT extended with "Bias toward action" and "Spawning workspaces and agents" sections; the new sections reference v2-only tool names that are absent on the v1 MCP path, creating a behavioral mismatch, and the host-to-user name matching heuristic may reliably fall back to sandbox. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Slack mention received] --> B{v1 or v2 MCP path?}
B -->|v2| C[Load tools incl. superset_projects_list]
B -->|v1| D[Load legacy tools - NO spawn tools]
C --> E[Apply SYSTEM_PROMPT with spawning guidance]
D --> E
E --> F{Code-change request?}
F -->|Intent clear| G[Call superset_projects_list]
F -->|Ambiguous| H[Ask clarifying question]
F -->|Track for later| I[Create task only]
G --> J[Call superset_workspaces_list]
J -->|Exists| K[Mention existing workspace]
J -->|None| L[Derive kebab-case branch slug]
L --> M{User host online?}
M -->|Matches| N[Use user host]
M -->|No match| O[Fall back to sandbox]
N --> P[superset_workspaces_create]
O --> P
P --> Q[Agent runs with claude preset]
style D fill:#f99,stroke:#c00
Prompt To Fix All With AI
Fix the following 3 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 3
apps/api/src/app/api/integrations/slack/events/utils/run-agent/run-agent.ts:426-432
**V2-only tool names referenced on both code paths**
The prompt explicitly names `superset_projects_list`, `superset_workspaces_list`, `superset_workspaces_create`, and `superset_agents_run` — tools that only exist on the v2 MCP path. When the v1 path is active (`FEATURE_FLAGS.SLACK_MCP_V2` off), the Anthropic model sees these names in its system prompt but those tools are absent from its schema. Claude is likely to either attempt the named call (resulting in a tool-not-found error) or describe spawning a workspace without actually invoking any tool, confusing users who expected action. The "Bias toward action" section amplifies this risk because the model is now actively directed to spawn for any code-change request. Consider gating the new sections behind the feature flag, or replacing specific tool names with generic descriptions so the model can map them to whichever names its current tool list exposes.
### Issue 2 of 3
apps/api/src/app/api/integrations/slack/events/utils/run-agent/run-agent.ts:431
**Host-to-user name matching may always fall through to `sandbox`**
The instruction "match the host name against the current user" assumes a host's `name` field (e.g. `"john-mbp"`) correlates with the current user's display name or email. In the v2 context block built by `fetchAgentContext`, hosts are listed as `${h.name} (id: ${h.id}, online: yes/no)` and the current user as `${currentUser.name ?? currentUser.email}` — two unrelated naming conventions. When a user's machine is named something like `"MacBook-Pro-2"` or a corporate asset tag, the fuzzy match will reliably fail and the agent will always fall back to `sandbox`. Consider surfacing ownership metadata to make the heuristic more reliable.
### Issue 3 of 3
apps/api/src/app/api/integrations/slack/events/utils/run-agent/run-agent.ts:427
**Hardcoded `superset` slug is a silent failure risk**
The prompt hard-wires the string `superset` as the project slug for all Superset-related requests. If that project is ever renamed or its slug changes in the database, the bot will silently pass the wrong slug to `superset_workspaces_create` and the spawn will fail (or land in the wrong repo). A softer phrasing — e.g. "use the project whose slug or name best matches 'superset'" — would let the model recover by calling `superset_projects_list` and selecting the closest match at runtime.
Reviews (1): Last reviewed commit: "feat(slack): make Slack agent proactive ..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| - Before spawning, call superset_projects_list and pick the project that best matches the request. Match on repository name, project slug, and topic keywords drawn from the user's message and the thread/channel context. | ||
| - If the request concerns the Slack app, Superset features, the tasks or automations UI, or anything about Superset itself, use the Superset project (slug: \`superset\`). | ||
| - If several projects could plausibly match, pick the strongest candidate, state in one short sentence which you picked and why, and proceed — don't stall on the choice. | ||
| - Also call superset_workspaces_list and check for an existing active workspace on a relevant branch. superset_workspaces_create always creates a new workspace, so when a relevant one already exists, mention it to the user instead of silently duplicating it. | ||
| - Derive the branch name from the request as a short kebab-case slug (e.g. \`fix-slack-retry\`, \`add-csv-export\`). | ||
| - For the host, use the requesting user's own machine when it appears online in the Hosts list in your context (match the host name against the current user); otherwise fall back to the \`sandbox\` host. | ||
| - Spawn the agent as part of workspace creation when possible — superset_workspaces_create accepts an \`agents\` array — or call superset_agents_run against the workspace afterward. Use the \`claude\` agent preset unless the user asks for another. Give the agent a clear, self-contained prompt describing the change to make. |
There was a problem hiding this comment.
The prompt explicitly names superset_projects_list, superset_workspaces_list, superset_workspaces_create, and superset_agents_run — tools that only exist on the v2 MCP path. When the v1 path is active (FEATURE_FLAGS.SLACK_MCP_V2 off), the Anthropic model sees these names in its system prompt but those tools are absent from its schema. Claude is likely to either attempt the named call (resulting in a tool-not-found error) or describe spawning a workspace without actually invoking any tool, confusing users who expected action. The "Bias toward action" section amplifies this risk because the model is now actively directed to spawn for any code-change request. Consider gating the new sections behind the feature flag, or replacing specific tool names with generic descriptions so the model can map them to whichever names its current tool list exposes.
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/api/src/app/api/integrations/slack/events/utils/run-agent/run-agent.ts
Line: 426-432
Comment:
**V2-only tool names referenced on both code paths**
The prompt explicitly names `superset_projects_list`, `superset_workspaces_list`, `superset_workspaces_create`, and `superset_agents_run` — tools that only exist on the v2 MCP path. When the v1 path is active (`FEATURE_FLAGS.SLACK_MCP_V2` off), the Anthropic model sees these names in its system prompt but those tools are absent from its schema. Claude is likely to either attempt the named call (resulting in a tool-not-found error) or describe spawning a workspace without actually invoking any tool, confusing users who expected action. The "Bias toward action" section amplifies this risk because the model is now actively directed to spawn for any code-change request. Consider gating the new sections behind the feature flag, or replacing specific tool names with generic descriptions so the model can map them to whichever names its current tool list exposes.
How can I resolve this? If you propose a fix, please make it concise.| - If several projects could plausibly match, pick the strongest candidate, state in one short sentence which you picked and why, and proceed — don't stall on the choice. | ||
| - Also call superset_workspaces_list and check for an existing active workspace on a relevant branch. superset_workspaces_create always creates a new workspace, so when a relevant one already exists, mention it to the user instead of silently duplicating it. | ||
| - Derive the branch name from the request as a short kebab-case slug (e.g. \`fix-slack-retry\`, \`add-csv-export\`). | ||
| - For the host, use the requesting user's own machine when it appears online in the Hosts list in your context (match the host name against the current user); otherwise fall back to the \`sandbox\` host. |
There was a problem hiding this comment.
sandbox
The instruction "match the host name against the current user" assumes a host's name field (e.g. "john-mbp") correlates with the current user's display name or email. In the v2 context block built by fetchAgentContext, hosts are listed as ${h.name} (id: ${h.id}, online: yes/no) and the current user as ${currentUser.name ?? currentUser.email} — two unrelated naming conventions. When a user's machine is named something like "MacBook-Pro-2" or a corporate asset tag, the fuzzy match will reliably fail and the agent will always fall back to sandbox. Consider surfacing ownership metadata to make the heuristic more reliable.
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/api/src/app/api/integrations/slack/events/utils/run-agent/run-agent.ts
Line: 431
Comment:
**Host-to-user name matching may always fall through to `sandbox`**
The instruction "match the host name against the current user" assumes a host's `name` field (e.g. `"john-mbp"`) correlates with the current user's display name or email. In the v2 context block built by `fetchAgentContext`, hosts are listed as `${h.name} (id: ${h.id}, online: yes/no)` and the current user as `${currentUser.name ?? currentUser.email}` — two unrelated naming conventions. When a user's machine is named something like `"MacBook-Pro-2"` or a corporate asset tag, the fuzzy match will reliably fail and the agent will always fall back to `sandbox`. Consider surfacing ownership metadata to make the heuristic more reliable.
How can I resolve this? If you propose a fix, please make it concise.|
|
||
| Spawning workspaces and agents: | ||
| - Before spawning, call superset_projects_list and pick the project that best matches the request. Match on repository name, project slug, and topic keywords drawn from the user's message and the thread/channel context. | ||
| - If the request concerns the Slack app, Superset features, the tasks or automations UI, or anything about Superset itself, use the Superset project (slug: \`superset\`). |
There was a problem hiding this comment.
superset slug is a silent failure risk
The prompt hard-wires the string superset as the project slug for all Superset-related requests. If that project is ever renamed or its slug changes in the database, the bot will silently pass the wrong slug to superset_workspaces_create and the spawn will fail (or land in the wrong repo). A softer phrasing — e.g. "use the project whose slug or name best matches 'superset'" — would let the model recover by calling superset_projects_list and selecting the closest match at runtime.
Prompt To Fix With AI
This is a comment left during a code review.
Path: apps/api/src/app/api/integrations/slack/events/utils/run-agent/run-agent.ts
Line: 427
Comment:
**Hardcoded `superset` slug is a silent failure risk**
The prompt hard-wires the string `superset` as the project slug for all Superset-related requests. If that project is ever renamed or its slug changes in the database, the bot will silently pass the wrong slug to `superset_workspaces_create` and the spawn will fail (or land in the wrong repo). A softer phrasing — e.g. "use the project whose slug or name best matches 'superset'" — would let the model recover by calling `superset_projects_list` and selecting the closest match at runtime.
How can I resolve this? If you propose a fix, please make it concise.
The system prompt framed the bot as a task-tracking assistant and didn't mention it can spawn workspaces and run coding agents. Reframe the opener, add that capability to the list, and add one guideline to default to action — preferring a workspace + agent over a task for code-change requests, while still asking when the request is genuinely ambiguous.
saddlepaddle
force-pushed
the
slack-prompt-proactive
branch
from
4b20af7 to
c815024
Compare
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
🧹 Preview Cleanup CompleteThe following preview resources have been cleaned up:
Thank you for your contribution! 🎉 |
1 participant
Summary
SYSTEM_PROMPTopener — it described the bot as a task-management assistant, but it can also spawn workspaces and run coding agents.Why / Context
For "can someone fix X" style messages the bot tended to file a task or ask questions rather than kick off the work, partly because the prompt only framed it around task tracking. This nudges it toward action while keeping it a general Slack assistant.
Testing
bunx biome checkon the changed file — cleanNotes
SYSTEM_PROMPTstring changed; the dynamic context interpolation (org, channel, members, statuses, hosts) is untouched.FEATURE_FLAGS.SLACK_MCP_V2). Workspace/agent spawning is a v2 capability; on v1 the bot adapts to the tools it's given.Summary by CodeRabbit