-
Notifications
You must be signed in to change notification settings - Fork 1.2k
feat(cli): send pasted images to the /btw side agent #3619
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
9cc4333
ef918b3
275a264
8e51319
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@moonshot-ai/kimi-code": minor | ||
| --- | ||
|
|
||
| Add image support to /btw: pasted images now reach the side agent, which can also view image and video files. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@moonshot-ai/kimi-code": minor | ||
| --- | ||
|
|
||
| Add read-only tools to the /btw side agent. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,6 +2,7 @@ import { Spacer } from '@moonshot-ai/pi-tui'; | |
| import type { | ||
| Event, | ||
| KimiHarness, | ||
| PromptInput, | ||
| Session, | ||
| TurnEndedEvent, | ||
| } from '@moonshot-ai/kimi-code-sdk'; | ||
|
|
@@ -14,14 +15,40 @@ import { createMarkdownTheme } from '../theme/pi-tui-theme'; | |
| import type { InlineSkillActivation } from '../types'; | ||
| import type { TUIState } from '../tui-state'; | ||
|
|
||
| import type { StagingLease } from './staging-leases'; | ||
|
|
||
| const BTW_BUSY_NOTICE = 'Wait for /btw to finish before sending another question.'; | ||
|
|
||
| export interface BtwPreparedPrompt { | ||
| /** Media-expanded RPC input; undefined means send the plain text prompt. */ | ||
| readonly input?: PromptInput; | ||
| /** Staged-media lease and its exact-binding submission id (plain prompts only). */ | ||
| readonly lease?: StagingLease; | ||
| readonly submissionId?: string; | ||
| } | ||
|
|
||
| export interface BtwPanelHost { | ||
| state: TUIState; | ||
| session: Session | undefined; | ||
| readonly harness: KimiHarness; | ||
|
|
||
| showError(msg: string): void; | ||
| /** | ||
| * Expand pasted image/video placeholders into daemon file-ref parts for the | ||
| * side agent (the /btw counterpart of the main send path's media | ||
| * preparation). Returns undefined when preparation failed — the error was | ||
| * already shown. | ||
| */ | ||
| prepareBtwPrompt( | ||
| text: string, | ||
| opts: { readonly stage: boolean }, | ||
| ): Promise<BtwPreparedPrompt | undefined>; | ||
| /** Track a prompt dispatch carrying staged media; see StagingLeaseTracker.trackDispatch. */ | ||
| trackBtwDispatch( | ||
| lease: StagingLease | undefined, | ||
| request: Promise<unknown>, | ||
| onError: (error: unknown) => void, | ||
| ): void; | ||
| } | ||
|
|
||
| export class BtwPanelController { | ||
|
|
@@ -180,27 +207,51 @@ export class BtwPanelController { | |
| panel: BtwPanelComponent, | ||
| inlineSkillActivations?: readonly InlineSkillActivation[], | ||
| ): void { | ||
| void this.prepareAndPrompt(agentId, prompt, panel, inlineSkillActivations); | ||
| } | ||
|
|
||
| private async prepareAndPrompt( | ||
| agentId: string, | ||
| prompt: string, | ||
| panel: BtwPanelComponent, | ||
| inlineSkillActivations?: readonly InlineSkillActivation[], | ||
| ): Promise<void> { | ||
| const session = this.host.session; | ||
| if (session === undefined) { | ||
| panel.markFailed(NO_ACTIVE_SESSION_MESSAGE); | ||
| this.host.state.ui.requestRender(); | ||
| return; | ||
| } | ||
| const send = | ||
| inlineSkillActivations !== undefined && inlineSkillActivations.length > 0 | ||
| ? () => | ||
| session.promptWithSkills( | ||
| prompt, | ||
| inlineSkillActivations.map((activation) => ({ | ||
| name: activation.skillName, | ||
| args: activation.args, | ||
| })), | ||
| ) | ||
| : () => session.prompt(prompt); | ||
| void this.withInteractiveAgent(agentId, send).catch((error: unknown) => { | ||
| panel.markFailed(`Failed to send /btw prompt: ${formatErrorMessage(error)}`); | ||
| const useSkills = inlineSkillActivations !== undefined && inlineSkillActivations.length > 0; | ||
| // Skill bundles have no prompt-id channel, so they match the main turn's | ||
| // inline-skill path: media rides along without a staged lease. | ||
| const prepared = await this.host.prepareBtwPrompt(prompt, { stage: !useSkills }); | ||
| if (prepared === undefined) { | ||
| panel.markFailed('Failed to prepare the media attachment.'); | ||
| this.host.state.ui.requestRender(); | ||
| }); | ||
| return; | ||
| } | ||
| const input = prepared.input ?? prompt; | ||
| const send = useSkills | ||
| ? () => | ||
| session.promptWithSkills( | ||
| input, | ||
| inlineSkillActivations.map((activation) => ({ | ||
| name: activation.skillName, | ||
| args: activation.args, | ||
| })), | ||
| ) | ||
| : prepared.submissionId !== undefined | ||
| ? () => session.prompt(input, { promptId: prepared.submissionId }) | ||
| : () => session.prompt(input); | ||
| this.host.trackBtwDispatch( | ||
| prepared.lease, | ||
| this.withInteractiveAgent(agentId, send), | ||
|
Comment on lines
+247
to
+249
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When an actual Useful? React with 👍 / 👎. |
||
| (error: unknown) => { | ||
| panel.markFailed(`Failed to send /btw prompt: ${formatErrorMessage(error)}`); | ||
| this.host.state.ui.requestRender(); | ||
| }, | ||
| ); | ||
| } | ||
|
|
||
| private async cancelAgent(agentId: string): Promise<void> { | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If a just-pasted image or video is still being ingested, this await can last up to two seconds. During that interval, Esc or Ctrl-C closes/unregisters the panel and calls
session.cancel()while the child is still idle, but the continuation never checks whether the panel remains active and subsequently dispatches the prompt anyway. This produces an invisible side-agent request after the user explicitly canceled it (and can overlap a newly opened/btwpanel); cancellation or panel identity should be rechecked before dispatch, with any prepared lease released on abandonment.Useful? React with 👍 / 👎.