-
Notifications
You must be signed in to change notification settings - Fork 2.3k
fix: make image processing work in github copilot provider #5687
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: make image processing work in github copilot provider #5687
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR fixes image processing in the GitHub Copilot provider by adding the Copilot-Vision-Request header when images are detected in the request payload.
- Introduces
payload_contains_image()helper function to detect images in message content - Modifies
get_github_headers()to accept optional payload and conditionally add the vision header - Updates all call sites to pass
Nonewhen payload inspection is not needed
Reviewed Changes
Copilot reviewed 1 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| crates/goose/src/providers/githubcopilot.rs | Added image detection logic and updated header generation to include Copilot-Vision-Request header when images are present in the payload |
| ui/desktop/openapi.json | Added newline at end of file for proper formatting |
| fn payload_contains_image(payload: &Value) -> bool { | ||
| if let Some(messages) = payload.get("messages").and_then(|m| m.as_array()) { | ||
| for msg in messages { | ||
| if let Some(content) = msg.get("content") { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you could do this more functional, like messages.iter.any and then cast the is_object to a single list but up to you
| let url = url::Url::parse(&format!("{}/chat/completions", endpoint)) | ||
| .map_err(|e| ProviderError::RequestFailed(format!("Invalid base URL: {e}")))?; | ||
|
|
||
| // Build headers and include Copilot-Vision-Request when payload contains processed images |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd drop this comment, but do what it says :)
adding payload to get_github_headers seems a tad awkward. if you just break up that builder below you just go:
let mut request = self.client.post(..).headers()
if has_images:
request = request.header(that_header)
response = request.header(..).send.await?
or some such
then you can leave the other bits alone
863febf to
290e845
Compare
* origin/main: (29 commits) chore: Update governance to include Discord (#5690) Ollama improvements (#5609) feat: add Supabase MCP server to registry (#5629) Unlist VS Code extension tutorials from MCP and experimental sections (#5677) fix: make image processing work in github copilot provider (#5687) fix: do not take into account gitignore in developer mcp (#5688) docs: session storage migration (#5682) New maintainers (#5685) chore: Update governance (#5660) chore(release): release version 1.14.0 (minor) (#5676) fix : action icons overlap session title in chat history (#5684) Document recent goose PRs (#5683) docs: add GOOSE_PATH_ROOT environment variable documentation (#5678) feat: SessionManager integration for acp sessions (#5657) teach copilot our CI (#5672) bump openapi version directly (#5674) governance: update MAINTAINERS.md to reflect new maintainers (#5675) chore: upgrade rmcp to 0.8.5 (#5673) Update release instructions (#5662) Swapped out to_string_lossy with display for user facing text (#5666) ...
* 'main' of github.com:block/goose: Fix: Always show autocompact threshold ui (#5701) chore: Update governance to include Discord (#5690) Ollama improvements (#5609) feat: add Supabase MCP server to registry (#5629) Unlist VS Code extension tutorials from MCP and experimental sections (#5677) fix: make image processing work in github copilot provider (#5687) fix: do not take into account gitignore in developer mcp (#5688) docs: session storage migration (#5682) New maintainers (#5685) chore: Update governance (#5660) chore(release): release version 1.14.0 (minor) (#5676)
* main: (27 commits) hackathon banner (#5710) Fix documentation-only change detection for push events (#5712) Added transaction commits to multi sql functions in session_manager (#5693) fix: improve and simplify tool call chain rendering (#5704) Fix: Always show autocompact threshold ui (#5701) chore: Update governance to include Discord (#5690) Ollama improvements (#5609) feat: add Supabase MCP server to registry (#5629) Unlist VS Code extension tutorials from MCP and experimental sections (#5677) fix: make image processing work in github copilot provider (#5687) fix: do not take into account gitignore in developer mcp (#5688) docs: session storage migration (#5682) New maintainers (#5685) chore: Update governance (#5660) chore(release): release version 1.14.0 (minor) (#5676) fix : action icons overlap session title in chat history (#5684) Document recent goose PRs (#5683) docs: add GOOSE_PATH_ROOT environment variable documentation (#5678) feat: SessionManager integration for acp sessions (#5657) teach copilot our CI (#5672) ...
…eanup * 'main' of github.com:block/goose: (46 commits) Fix context progress bar not resetting after /clear command (#5652) docs: removing double announcements (#5714) docs: mcp sampling support (#5708) hackathon banner (#5710) Fix documentation-only change detection for push events (#5712) Added transaction commits to multi sql functions in session_manager (#5693) fix: improve and simplify tool call chain rendering (#5704) Fix: Always show autocompact threshold ui (#5701) chore: Update governance to include Discord (#5690) Ollama improvements (#5609) feat: add Supabase MCP server to registry (#5629) Unlist VS Code extension tutorials from MCP and experimental sections (#5677) fix: make image processing work in github copilot provider (#5687) fix: do not take into account gitignore in developer mcp (#5688) docs: session storage migration (#5682) New maintainers (#5685) chore: Update governance (#5660) chore(release): release version 1.14.0 (minor) (#5676) fix : action icons overlap session title in chat history (#5684) Document recent goose PRs (#5683) ...
* origin/main: (51 commits) Compaction resiliency improvements (#5618) docs: ask goose button (#5730) Update prompt injection detection metrics (due to errors exporting to datadog) (#5692) Spence/icon2 tokyo drift (#5728) docs: logs rotation and misc updates (#5727) docs: automatic ollama model detection (#5725) Fix context progress bar not resetting after /clear command (#5652) docs: removing double announcements (#5714) docs: mcp sampling support (#5708) hackathon banner (#5710) Fix documentation-only change detection for push events (#5712) Added transaction commits to multi sql functions in session_manager (#5693) fix: improve and simplify tool call chain rendering (#5704) Fix: Always show autocompact threshold ui (#5701) chore: Update governance to include Discord (#5690) Ollama improvements (#5609) feat: add Supabase MCP server to registry (#5629) Unlist VS Code extension tutorials from MCP and experimental sections (#5677) fix: make image processing work in github copilot provider (#5687) fix: do not take into account gitignore in developer mcp (#5688) ...
Signed-off-by: Blair Allan <[email protected]>
Fixes #5497 by sending the
Copilot-Vision-Requestheader with a value of true when there is a processed image in the messageWorking example after the change: