Skip to content

修复豆包图像编辑(图生图)功能 #2090

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
creamlike1024 merged 3 commits into from
Oct 23, 2025
Merged

修复豆包图像编辑(图生图)功能 #2090

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fcf0f95
feat: doubao-seedream-4-0-250828 image to image
feitianbubu Oct 23, 2025
3ac9ff6
feat: doubao-seedream support image edit
feitianbubu Oct 23, 2025
95a2d02
fix: fail get moel by
feitianbubu Oct 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions common/gin.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@ package common

import (
"bytes"
"encoding/json"
"io"
"mime/multipart"
"net/http"
"net/url"
"strings"
"time"

Expand Down Expand Up @@ -40,6 +42,10 @@ func UnmarshalBodyReusable(c *gin.Context, v any) error {
contentType := c.Request.Header.Get("Content-Type")
if strings.HasPrefix(contentType, "application/json") {
err = Unmarshal(requestBody, &v)
} else if strings.Contains(contentType, gin.MIMEPOSTForm) {
err = parseFormData(requestBody, &v)
} else if strings.Contains(contentType, gin.MIMEMultipartPOSTForm) {
err = parseMultipartFormData(c, requestBody, &v)
} else {
// skip for now
// TODO: someday non json request have variant model, we will need to implementation this
Expand Down Expand Up @@ -138,3 +144,57 @@ func ParseMultipartFormReusable(c *gin.Context) (*multipart.Form, error) {
c.Request.Body = io.NopCloser(bytes.NewBuffer(requestBody))
return form, nil
}

func parseFormData(data []byte, v any) error {
values, err := url.ParseQuery(string(data))
if err != nil {
return err
}
formMap := make(map[string]any)
for key, vals := range values {
if len(vals) == 1 {
formMap[key] = vals[0]
} else {
formMap[key] = vals
}
}
jsonData, err := json.Marshal(formMap)
if err != nil {
return err
}

return json.Unmarshal(jsonData, v)
}

func parseMultipartFormData(c *gin.Context, data []byte, v any) error {
contentType := c.Request.Header.Get("Content-Type")
boundary := ""
if idx := strings.Index(contentType, "boundary="); idx != -1 {
boundary = contentType[idx+9:]
}

if boundary == "" {
return json.Unmarshal(data, v) // Fallback to JSON
}
Comment on lines +176 to +178

@coderabbitai coderabbitai Bot Oct 23, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Questionable fallback: Missing boundary should be an error.

If Content-Type is multipart/form-data but the boundary parameter is missing, that's a malformed request. Falling back to JSON parsing (line 177) masks this error and could lead to confusing behavior.

Consider returning an error instead:

 	if boundary == "" {
-		return json.Unmarshal(data, v) // Fallback to JSON
+		return errors.New("multipart/form-data Content-Type is missing required boundary parameter")
 	}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if boundary == "" {
return json.Unmarshal(data, v) // Fallback to JSON
}
if boundary == "" {
return errors.New("multipart/form-data Content-Type is missing required boundary parameter")
}
🤖 Prompt for AI Agents 
In common/gin.go around lines 176-178, do not silently fall back to JSON when
Content-Type is multipart/form-data but the boundary is missing; instead return
a clear error indicating a malformed multipart request (e.g., "missing multipart
boundary" or a wrapped error) so callers can handle it as a bad request; remove
the json.Unmarshal fallback and return that error (ensure the function's
signature supports returning the error type expected by callers).


reader := multipart.NewReader(bytes.NewReader(data), boundary)
form, err := reader.ReadForm(32 << 20) // 32 MB max memory
if err != nil {
return err
}
defer form.RemoveAll()
formMap := make(map[string]any)
for key, vals := range form.Value {
if len(vals) == 1 {
formMap[key] = vals[0]
} else {
formMap[key] = vals
}
}
jsonData, err := json.Marshal(formMap)
if err != nil {
return err
}

return json.Unmarshal(jsonData, v)
}
3 changes: 2 additions & 1 deletion dto/openai_image.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@ type ImageRequest struct {
OutputCompression json.RawMessage `json:"output_compression,omitempty"`
PartialImages json.RawMessage `json:"partial_images,omitempty"`
// Stream bool `json:"stream,omitempty"`
Watermark *bool `json:"watermark,omitempty"`
Watermark *bool `json:"watermark,omitempty"`
Image json.RawMessage `json:"image,omitempty"`
// 用匿名参数接收额外参数
Extra map[string]json.RawMessage `json:"-"`
}
Expand Down
4 changes: 3 additions & 1 deletion middleware/distributor.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"errors"
"fmt"
"net/http"
"slices"

@coderabbitai coderabbitai Bot Oct 23, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Unused import after fixing the Content-Type check.

After applying the fix above, the slices import will no longer be needed and should be removed.

🤖 Prompt for AI Agents 
In middleware/distributor.go at line 7 the import "slices" is now unused after
the Content-Type check fix; remove the "slices" entry from the import block (and
tidy imports if needed) so the file compiles without unused-import errors.

"strconv"
"strings"
"time"
Expand Down Expand Up @@ -245,7 +246,8 @@ func getModelRequest(c *gin.Context) (*ModelRequest, bool, error) {
modelRequest.Model = common.GetStringIfEmpty(modelRequest.Model, "dall-e")
} else if strings.HasPrefix(c.Request.URL.Path, "/v1/images/edits") {
//modelRequest.Model = common.GetStringIfEmpty(c.PostForm("model"), "gpt-image-1")
if strings.Contains(c.Request.Header.Get("Content-Type"), "multipart/form-data") {
contentType := c.Request.Header.Get("Content-Type")
if slices.Contains([]string{gin.MIMEPOSTForm, gin.MIMEMultipartPOSTForm}, contentType) {
modelRequest.Model = c.PostForm("model")
}

@coderabbitai coderabbitai Bot Oct 23, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Critical: Content-Type matching will fail for real requests.

The exact match check using slices.Contains will fail because Content-Type headers typically include parameters. For example, multipart/form-data; boundary=----WebKitFormBoundary... will not match gin.MIMEMultipartPOSTForm ("multipart/form-data").

Note that common/gin.go (lines 45-48) correctly uses strings.Contains() for the same purpose, which handles parameterized headers properly.

Apply this diff to fix the Content-Type matching:

-	contentType := c.Request.Header.Get("Content-Type")
-	if slices.Contains([]string{gin.MIMEPOSTForm, gin.MIMEMultipartPOSTForm}, contentType) {
+	contentType := c.Request.Header.Get("Content-Type")
+	if strings.Contains(contentType, gin.MIMEPOSTForm) || strings.Contains(contentType, gin.MIMEMultipartPOSTForm) {
 		modelRequest.Model = c.PostForm("model")
 	}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
contentType := c.Request.Header.Get("Content-Type")
if slices.Contains([]string{gin.MIMEPOSTForm, gin.MIMEMultipartPOSTForm}, contentType) {
modelRequest.Model = c.PostForm("model")
}
contentType := c.Request.Header.Get("Content-Type")
if strings.Contains(contentType, gin.MIMEPOSTForm) || strings.Contains(contentType, gin.MIMEMultipartPOSTForm) {
modelRequest.Model = c.PostForm("model")
}
🤖 Prompt for AI Agents 
In middleware/distributor.go around lines 249 to 252, the Content-Type check
uses slices.Contains which does exact matches and fails for parameterized
headers (e.g. "multipart/form-data; boundary=..."); change the logic to use
strings.Contains(c.Request.Header.Get("Content-Type"), gin.MIMEPOSTForm) ||
strings.Contains(..., gin.MIMEMultipartPOSTForm) so parameterized types match,
and add an import for the strings package if not already present; keep the
existing c.PostForm("model") assignment.

}
Expand Down
212 changes: 107 additions & 105 deletions relay/channel/volcengine/adaptor.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,7 @@ import (
"errors"
"fmt"
"io"
"mime/multipart"
"net/http"
"net/textproto"
"path/filepath"
"strings"

Expand Down Expand Up @@ -104,106 +102,107 @@ func (a *Adaptor) ConvertImageRequest(c *gin.Context, info *relaycommon.RelayInf
switch info.RelayMode {
case constant.RelayModeImagesGenerations:
return request, nil
case constant.RelayModeImagesEdits:

var requestBody bytes.Buffer
writer := multipart.NewWriter(&requestBody)

writer.WriteField("model", request.Model)

formData := c.Request.PostForm
for key, values := range formData {
if key == "model" {
continue
}
for _, value := range values {
writer.WriteField(key, value)
}
}

if err := c.Request.ParseMultipartForm(32 << 20); err != nil {
return nil, errors.New("failed to parse multipart form")
}

if c.Request.MultipartForm != nil && c.Request.MultipartForm.File != nil {
var imageFiles []*multipart.FileHeader
var exists bool

if imageFiles, exists = c.Request.MultipartForm.File["image"]; !exists || len(imageFiles) == 0 {
if imageFiles, exists = c.Request.MultipartForm.File["image[]"]; !exists || len(imageFiles) == 0 {
foundArrayImages := false
for fieldName, files := range c.Request.MultipartForm.File {
if strings.HasPrefix(fieldName, "image[") && len(files) > 0 {
foundArrayImages = true
for _, file := range files {
imageFiles = append(imageFiles, file)
}
}
}

if !foundArrayImages && (len(imageFiles) == 0) {
return nil, errors.New("image is required")
}
}
}

for i, fileHeader := range imageFiles {
file, err := fileHeader.Open()
if err != nil {
return nil, fmt.Errorf("failed to open image file %d: %w", i, err)
}
defer file.Close()

fieldName := "image"
if len(imageFiles) > 1 {
fieldName = "image[]"
}

mimeType := detectImageMimeType(fileHeader.Filename)

h := make(textproto.MIMEHeader)
h.Set("Content-Disposition", fmt.Sprintf(`form-data; name="%s"; filename="%s"`, fieldName, fileHeader.Filename))
h.Set("Content-Type", mimeType)

part, err := writer.CreatePart(h)
if err != nil {
return nil, fmt.Errorf("create form part failed for image %d: %w", i, err)
}

if _, err := io.Copy(part, file); err != nil {
return nil, fmt.Errorf("copy file failed for image %d: %w", i, err)
}
}

if maskFiles, exists := c.Request.MultipartForm.File["mask"]; exists && len(maskFiles) > 0 {
maskFile, err := maskFiles[0].Open()
if err != nil {
return nil, errors.New("failed to open mask file")
}
defer maskFile.Close()

mimeType := detectImageMimeType(maskFiles[0].Filename)

h := make(textproto.MIMEHeader)
h.Set("Content-Disposition", fmt.Sprintf(`form-data; name="mask"; filename="%s"`, maskFiles[0].Filename))
h.Set("Content-Type", mimeType)

maskPart, err := writer.CreatePart(h)
if err != nil {
return nil, errors.New("create form file failed for mask")
}

if _, err := io.Copy(maskPart, maskFile); err != nil {
return nil, errors.New("copy mask file failed")
}
}
} else {
return nil, errors.New("no multipart form data found")
}

writer.Close()
c.Request.Header.Set("Content-Type", writer.FormDataContentType())
return bytes.NewReader(requestBody.Bytes()), nil
// 根据官方文档,并没有发现豆包生图支持表单请求:https://www.volcengine.com/docs/82379/1824121
//case constant.RelayModeImagesEdits:
//
// var requestBody bytes.Buffer
// writer := multipart.NewWriter(&requestBody)
//
// writer.WriteField("model", request.Model)
//
// formData := c.Request.PostForm
// for key, values := range formData {
// if key == "model" {
// continue
// }
// for _, value := range values {
// writer.WriteField(key, value)
// }
// }
//
// if err := c.Request.ParseMultipartForm(32 << 20); err != nil {
// return nil, errors.New("failed to parse multipart form")
// }
//
// if c.Request.MultipartForm != nil && c.Request.MultipartForm.File != nil {
// var imageFiles []*multipart.FileHeader
// var exists bool
//
// if imageFiles, exists = c.Request.MultipartForm.File["image"]; !exists || len(imageFiles) == 0 {
// if imageFiles, exists = c.Request.MultipartForm.File["image[]"]; !exists || len(imageFiles) == 0 {
// foundArrayImages := false
// for fieldName, files := range c.Request.MultipartForm.File {
// if strings.HasPrefix(fieldName, "image[") && len(files) > 0 {
// foundArrayImages = true
// for _, file := range files {
// imageFiles = append(imageFiles, file)
// }
// }
// }
//
// if !foundArrayImages && (len(imageFiles) == 0) {
// return nil, errors.New("image is required")
// }
// }
// }
//
// for i, fileHeader := range imageFiles {
// file, err := fileHeader.Open()
// if err != nil {
// return nil, fmt.Errorf("failed to open image file %d: %w", i, err)
// }
// defer file.Close()
//
// fieldName := "image"
// if len(imageFiles) > 1 {
// fieldName = "image[]"
// }
//
// mimeType := detectImageMimeType(fileHeader.Filename)
//
// h := make(textproto.MIMEHeader)
// h.Set("Content-Disposition", fmt.Sprintf(`form-data; name="%s"; filename="%s"`, fieldName, fileHeader.Filename))
// h.Set("Content-Type", mimeType)
//
// part, err := writer.CreatePart(h)
// if err != nil {
// return nil, fmt.Errorf("create form part failed for image %d: %w", i, err)
// }
//
// if _, err := io.Copy(part, file); err != nil {
// return nil, fmt.Errorf("copy file failed for image %d: %w", i, err)
// }
// }
//
// if maskFiles, exists := c.Request.MultipartForm.File["mask"]; exists && len(maskFiles) > 0 {
// maskFile, err := maskFiles[0].Open()
// if err != nil {
// return nil, errors.New("failed to open mask file")
// }
// defer maskFile.Close()
//
// mimeType := detectImageMimeType(maskFiles[0].Filename)
//
// h := make(textproto.MIMEHeader)
// h.Set("Content-Disposition", fmt.Sprintf(`form-data; name="mask"; filename="%s"`, maskFiles[0].Filename))
// h.Set("Content-Type", mimeType)
//
// maskPart, err := writer.CreatePart(h)
// if err != nil {
// return nil, errors.New("create form file failed for mask")
// }
//
// if _, err := io.Copy(maskPart, maskFile); err != nil {
// return nil, errors.New("copy mask file failed")
// }
// }
// } else {
// return nil, errors.New("no multipart form data found")
// }
//
// writer.Close()
// c.Request.Header.Set("Content-Type", writer.FormDataContentType())
// return bytes.NewReader(requestBody.Bytes()), nil

default:
return request, nil
Expand Down Expand Up @@ -251,10 +250,11 @@ func (a *Adaptor) GetRequestURL(info *relaycommon.RelayInfo) (string, error) {
return fmt.Sprintf("%s/api/v3/chat/completions", baseUrl), nil
case constant.RelayModeEmbeddings:
return fmt.Sprintf("%s/api/v3/embeddings", baseUrl), nil
case constant.RelayModeImagesGenerations:
//豆包的图生图也走generations接口: https://www.volcengine.com/docs/82379/1824121
case constant.RelayModeImagesGenerations, constant.RelayModeImagesEdits:
return fmt.Sprintf("%s/api/v3/images/generations", baseUrl), nil
case constant.RelayModeImagesEdits:
return fmt.Sprintf("%s/api/v3/images/edits", baseUrl), nil
//case constant.RelayModeImagesEdits:
// return fmt.Sprintf("%s/api/v3/images/edits", baseUrl), nil
case constant.RelayModeRerank:
return fmt.Sprintf("%s/api/v3/rerank", baseUrl), nil
case constant.RelayModeAudioSpeech:
Expand All @@ -278,6 +278,8 @@ func (a *Adaptor) SetupRequestHeader(c *gin.Context, req *http.Header, info *rel
}
req.Set("Content-Type", "application/json")
return nil
} else if info.RelayMode == constant.RelayModeImagesEdits {
req.Set("Content-Type", gin.MIMEJSON)
}

req.Set("Authorization", "Bearer "+info.ApiKey)
Expand Down