fix: open source models use v1 endpoints instead of v1beta1

[seefs001]

fix #2404

Summary by CodeRabbit

• New Features
  • Added support for MaAs (Model as a Service) models. Users can now leverage MaAs model providers with automatic request routing and response handling.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

A new request mode RequestModeMaas is introduced to the Vertex AI relay channel. The adaptor recognizes model names containing "-maas" and routes them through the same logic path as Llama models for URL construction, request conversion, and response handling.

Changes

Cohort / File(s)	Summary
New RequestMode constant relay/channel/vertex/adaptor.go	Adds RequestModeMaas constant (value 4) to identify MaAs request types
Initialization logic relay/channel/vertex/adaptor.go	Extends Init to recognize upstream model names containing "-maas" and set mode to RequestModeMaas
Request routing & URL construction relay/channel/vertex/adaptor.go	Updates getRequestUrl and GetRequestURL to handle MaAs URL generation; treats MaAs as non-Gemini path equivalent to Llama
Request conversion & response handling relay/channel/vertex/adaptor.go	Extends ConvertOpenAIRequest and DoResponse (streaming and non-streaming) to route MaAs requests through OpenAI handlers identical to Llama behavior

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

• Single file modified with localized, pattern-following changes
• New logic mirrors existing Llama implementation rather than introducing novel logic
• Straightforward routing extension across multiple functions

Possibly related PRs

• QuantumNous/new-api#2236: Maps "-maas" models to RequestModeLlama as an alternative approach to MaAs handling
• QuantumNous/new-api#1817: Modifies model-specific request mode branching logic for Gemini/Claude/Llama

Poem

🐰 A fluffy hop through models fine,
MaAs now joins the routing line,
Like Llama's path, through Vertex it flows,
No "-maas" confusion—success it shows! ✨

Pre-merge checks and finishing touches

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: fixing open source models to use v1 endpoints instead of v1beta1, which directly addresses the PR's core objective.
Linked Issues check	✅ Passed	The changes implement the required fix by adding MaAs request mode support and routing it through v1 endpoints, directly addressing issue #2404's requirement to fix Vertex AI open-source model endpoint URLs.
Out of Scope Changes check	✅ Passed	All changes are focused on adding MaAs support and routing logic to fix the v1/v1beta1 endpoint issue, with no unrelated modifications detected.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (1)

relay/channel/vertex/adaptor.go (1)

352-356: Consider combining switch cases to reduce duplication.

RequestModeLlama and RequestModeMaas have identical handlers in both streaming and non-streaming paths.

Apply this diff to consolidate:

-		case RequestModeLlama:
-			return openai.OaiStreamHandler(c, info, resp)
-		case RequestModeMaas:
+		case RequestModeLlama, RequestModeMaas:
 			return openai.OaiStreamHandler(c, info, resp)

And similarly for the non-streaming case (lines 370-373):

-		case RequestModeLlama:
-			return openai.OpenaiHandler(c, info, resp)
-		case RequestModeMaas:
+		case RequestModeLlama, RequestModeMaas:
 			return openai.OpenaiHandler(c, info, resp)

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 11593bd and 2b0d48d.

📒 Files selected for processing (1)

• relay/channel/vertex/adaptor.go (7 hunks)

🧰 Additional context used

🧠 Learnings (1)

📚 Learning: 2025-08-05T17:14:17.246Z

Learnt from: neotf
Repo: QuantumNous/new-api PR: 1511
File: setting/ratio_setting/model_ratio.go:118-123
Timestamp: 2025-08-05T17:14:17.246Z
Learning: Claude models handle "-thinking" variants differently from Gemini models. For Claude models, only the base model (without "-thinking") gets an entry in defaultModelRatio map. The "-thinking" variants rely on the Claude relay handler stripping the suffix using strings.TrimSuffix(textRequest.Model, "-thinking") before looking up the ratio, so they automatically use the base model's ratio.

Applied to files:

• relay/channel/vertex/adaptor.go

🧬 Code graph analysis (1)

relay/channel/vertex/adaptor.go (1)

relay/channel/openai/relay-openai.go (2)

• OaiStreamHandler (106-194)
• OpenaiHandler (196-294)

🔇 Additional comments (4)

relay/channel/vertex/adaptor.go (4)

26-31: LGTM!

The new RequestModeMaas constant follows the existing naming and numbering convention.

---

79-89: LGTM!

The detection logic correctly identifies MaaS models using the -maas suffix pattern, and the ordering ensures no conflicts with existing model types.

---

219-221: LGTM!

MaaS correctly shares the same URL construction path as Llama since both use the OpenAI-compatible endpoints API.

---

317-319: LGTM!

Correctly passes through the OpenAI request without conversion for MaaS endpoints.

[coderabbitai]

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

find . -name "adaptor.go" -o -name "*adaptor*" 2>/dev/null | head -20

Repository: QuantumNous/new-api

Length of output: 739

---

🏁 Script executed:

find . -path "*vertex*" -name "*.go" 2>/dev/null | head -20

Repository: QuantumNous/new-api

Length of output: 282

---

🏁 Script executed:

cat -n ./relay/channel/vertex/adaptor.go | head -160

Repository: QuantumNous/new-api

Length of output: 6346

---

🏁 Script executed:

cat -n ./relay/channel/vertex/adaptor.go | sed -n '100,150p'

Repository: QuantumNous/new-api

Length of output: 1880

---

🏁 Script executed:

cat -n ./relay/channel/vertex/adaptor.go | sed -n '136,150p'

Repository: QuantumNous/new-api

Length of output: 637

---

Missing global region handling for MaaS models.

The code at lines 143-150 does not handle the global region case. When region == "global", it incorrectly produces https://global-aiplatform.googleapis.com/v1/... instead of the correct https://aiplatform.googleapis.com/v1/... (without the region prefix).

The global endpoint for MaaS uses the format https://aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/global/endpoints/..., which requires omitting the region prefix. Compare with the correct implementations for Gemini (lines 100-117) and Claude (lines 118-135), which both check if region == "global" and construct URLs without the region prefix.

Apply this diff to fix:

 } else if a.RequestMode == RequestModeMaas {
+    if region == "global" {
+        return fmt.Sprintf(
+            "https://aiplatform.googleapis.com/v1/projects/%s/locations/global/endpoints/openapi/chat/completions",
+            adc.ProjectID,
+        ), nil
+    }
     return fmt.Sprintf(
         "https://%s-aiplatform.googleapis.com/v1/projects/%s/locations/%s/endpoints/openapi/chat/completions",
         region,
         adc.ProjectID,
         region,
     ), nil
 }

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	} else if a.RequestMode == RequestModeMaas {
	return fmt.Sprintf(
	"https://%s-aiplatform.googleapis.com/v1/projects/%s/locations/%s/endpoints/openapi/chat/completions",
	region,
	adc.ProjectID,
	region,
	), nil
	} else if a.RequestMode == RequestModeMaas {
	if region == "global" {
	return fmt.Sprintf(
	"https://aiplatform.googleapis.com/v1/projects/%s/locations/global/endpoints/openapi/chat/completions",
	adc.ProjectID,
	), nil
	}
	return fmt.Sprintf(
	"https://%s-aiplatform.googleapis.com/v1/projects/%s/locations/%s/endpoints/openapi/chat/completions",
	region,
	adc.ProjectID,
	region,
	), nil

🤖 Prompt for AI Agents

In relay/channel/vertex/adaptor.go around lines 143 to 149, the MaaS URL
construction always prefixes the host with the region (producing
https://global-aiplatform.googleapis.com/...), so add a conditional like the
Gemini/Claude paths: if region == "global" build the URL using host
"aiplatform.googleapis.com" (no region prefix) and locations/global in the path;
otherwise keep the existing "%s-aiplatform.googleapis.com" host with the region
inserted. Ensure the returned path uses
projects/{project}/locations/{region}/endpoints/... (or locations/global when
region == "global") and return nil error as before.