Skip to content

fix(#5626): handle multi-text content better for openai format models #5838

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
alexhancock wants to merge 1 commit into from
Closed

fix(#5626): handle multi-text content better for openai format models #5838

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 25 additions & 5 deletions crates/goose/src/providers/formats/openai.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,25 +63,23 @@ pub fn format_messages(messages: &[Message], image_format: &ImageFormat) -> Vec<
});

let mut output = Vec::new();
let mut text_parts = Vec::new();

for content in &message.content {
match content {
MessageContent::Text(text) => {
if !text.text.is_empty() {
// Check for image paths in the text
if let Some(image_path) = detect_image_path(&text.text) {
// Try to load and convert the image
if let Ok(image) = load_image_file(image_path) {
converted["content"] = json!([
{"type": "text", "text": text.text},
convert_image(&image, image_format)
]);
Comment on lines 72 to 77
Copy link

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When an image path is detected and successfully loaded (lines 72-77), this code directly assigns to converted["content"], which will overwrite any previously accumulated text_parts. This breaks the fix for multiple text content chunks.

If a message has multiple text blocks and one contains an image path, only the text with the image will be included, losing the other text blocks. Consider accumulating the text in text_parts and handling the image separately, or restructuring to build the content array incrementally.

Copilot uses AI. Check for mistakes.
} else {
// If image loading fails, just use the text
converted["content"] = json!(text.text);
text_parts.push(text.text.clone());
}
} else {
converted["content"] = json!(text.text);
text_parts.push(text.text.clone());
}
}
}
Expand Down Expand Up @@ -244,9 +242,14 @@ pub fn format_messages(messages: &[Message], image_format: &ImageFormat) -> Vec<
}
}

if !text_parts.is_empty() {
converted["content"] = json!(text_parts.join("\n"));
}

if converted.get("content").is_some() || converted.get("tool_calls").is_some() {
output.insert(0, converted);
}

messages_spec.extend(output);
}

Expand Down Expand Up @@ -1328,6 +1331,23 @@ mod tests {
Ok(())
}

#[test]
fn test_format_messages_multiple_text_blocks() -> anyhow::Result<()> {
let message = Message::user()
.with_text("--- Resource: file:///test.md ---\n# Test\n\n---\n")
.with_text(" What is in the file?");

let spec = format_messages(&[message], &ImageFormat::OpenAi);

assert_eq!(spec.len(), 1);
assert_eq!(spec[0]["role"], "user");
assert_eq!(
spec[0]["content"],
"--- Resource: file:///test.md ---\n# Test\n\n---\n\n What is in the file?"
);
Ok(())
}

#[tokio::test]
async fn test_streamed_multi_tool_response_to_messages() -> anyhow::Result<()> {
let response_lines = r#"
Expand Down
Loading