feat: configure default max_tokens for anthropic translator

[herewasmike]

Description

max_tokens and max_completion_tokens are optional in openAI spec
https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create

However when I try to send the request from the source I get 422 if my AIServiceBackend has AWSAnthropic schema.
It's impossible to add this field with bodyMutation due to ordering (I've considered mutating body before the translation but it seems to be a whole redesign rather than small fix with providing default)

Special notes for reviewers (if applicable)

Per gen AI policy I disclose that claude did help me with setting up this PR

[dosubot]

Related Documentation

3 document(s) may need updating based on files changed in this PR:

Envoy's Space

gcp-vertexai /ai-gateway/blob/main/site/docs/getting-started/connect-providers/gcp-vertexai.md

View Suggested Changes

@@ -104,6 +104,10 @@
   $GATEWAY_URL/v1/chat/completions
 ```
 
+:::note
+The `max_completion_tokens` parameter (or `max_tokens`) is optional and defaults to 4096 if not specified. The example above includes it to demonstrate setting an explicit limit.
+:::
+
 Expected output:
 
 ```json

[Accept] [Decline]

gcp-vertexai /ai-gateway/blob/main/site/versioned_docs/version-0.4/getting-started/connect-providers/gcp-vertexai.md

View Suggested Changes

@@ -101,6 +101,10 @@
   $GATEWAY_URL/v1/chat/completions
 ```
 
+:::note
+The `max_completion_tokens` parameter is optional and defaults to 4096 if not specified. It's recommended to set it explicitly to control response length and costs.
+:::
+
 Expected output:
 
 ```json
@@ -136,6 +140,10 @@
   }' \
   $GATEWAY_URL/anthropic/v1/messages
 ```
+
+:::note
+The `max_tokens` parameter is optional and defaults to 4096 if not specified. It's recommended to set it explicitly to control response length and costs.
+:::
 
 ## Troubleshooting
 

[Accept] [Decline]

gcp-vertexai /ai-gateway/blob/main/site/versioned_docs/version-0.5/getting-started/connect-providers/gcp-vertexai.md

View Suggested Changes

@@ -104,6 +104,10 @@
   $GATEWAY_URL/v1/chat/completions
 ```
 
+:::note
+The `max_completion_tokens` parameter is optional. If not specified, it defaults to 4096 tokens.
+:::
+
 Expected output:
 
 ```json
@@ -139,6 +143,10 @@
   }' \
   $GATEWAY_URL/anthropic/v1/messages
 ```
+
+:::note
+The `max_tokens` parameter is optional. If not specified, it defaults to 4096 tokens.
+:::
 
 ## Troubleshooting
 

[Accept] [Decline]

Note: You must be authenticated to accept/decline updates.

^{How did I do? Any feedback?}  

[herewasmike]

I'm not sure if arbitrary default value + bodyMutator is the best approach.
Though, no idea how anthropic api reacts if supplied with max_tokens value greater than the model supports.
Happy to take suggestions here

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.38%. Comparing base (d50462a) to head (14f1f38).
⚠️ Report is 28 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1933      +/-   ##
==========================================
- Coverage   84.38%   84.38%   -0.01%     
==========================================
  Files         130      130              
  Lines       18113    18112       -1     
==========================================
- Hits        15284    15283       -1     
  Misses       1883     1883              
  Partials      946      946              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[nutanix-Hrushikesh]

Though, no idea how anthropic api reacts if supplied with max_tokens value greater than the model supports.

it returns 400 Bad Request

[johnugeorge]

This might be a bit confusing if low default is observed by default. Isn't it better for the client to be aware of and use the right values? Are you using a client where it is not configurable?

[herewasmike]

Yes, I'm connecting 3rd party code in this particular case so it's not easy to change the client's behavior.
I'll rework this PR following up the conversation on slack, though, to avoid the confusion

Simply allow requests to pass through and fail on the provider side (one would be able to detect it and mutate request body)

[herewasmike]

/retest

[herewasmike]

/retest