chore: adds harness test cases for file apis in anthropic

[sammaji]

Summary

Adds end-to-end test coverage for the Anthropic Files API, including file upload with an explicit content_type override, file upload with content_type inferred from the file part, using an uploaded file as a document source in /v1/messages, and using an uploaded file via input_file in /v1/responses. Cleanup requests delete both test files after the scenarios run.

Changes

• Added response shape detection for Anthropic file objects (type: "file") and deleted file objects (type: "file_deleted") in the provider harness test script so the content-validation assertion correctly recognises Anthropic Files API responses.
• Added two upload scenarios: one that passes an explicit content_type field (text/markdown) alongside the file, and one that relies on the content type being inferred from the uploaded file part.
• Added a preview test for referencing an uploaded file as a document source with file_id in an Anthropic /v1/messages request (the proxy is expected to inject the files-api-2025-04-14 beta header automatically).
• Added a preview test for referencing the same file via input_file / file_id through the native /v1/responses endpoint with an Anthropic model.
• Added DELETE cleanup requests for both uploaded files (anthropicUploadFileIdA and anthropicUploadFileIdB) to avoid leaving orphaned files after the test run.

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

Run the Postman/Newman collection against a running instance with a valid anthropicKey and baseUrl set:

newman run tests/e2e/api/collections/provider-harness.json \
  --env-var "baseUrl=http://localhost:8080" \
  --env-var "anthropicKey=<your-key>"

Expected outcomes:

• Both upload requests return HTTP 200 with a non-empty id field.
• The document and input_file preview requests return a valid model response.
• Both DELETE cleanup requests succeed, removing the uploaded files.

Breaking changes

• No

Related issues

Security considerations

The anthropicKey is supplied via an environment variable and is not hardcoded in the collection. No PII or secrets are stored in the test fixtures.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e04b7d12-8f96-4429-8d1c-13813178a75f

📥 Commits

Reviewing files that changed from the base of the PR and between ff036c1 and 356140e.

📒 Files selected for processing (1)

• tests/e2e/api/collections/provider-harness.json

🚧 Files skipped from review as they are similar to previous changes (1)

• tests/e2e/api/collections/provider-harness.json

---

📝 Walkthrough

Summary by CodeRabbit

• Tests
  • Expanded end-to-end coverage for file uploads and deletions.
  • Added checks for uploads with and without a content-type override.
  • Updated file response detection to better recognize file and deleted-file responses.
  • Added downstream preview coverage using uploaded file IDs.

Walkthrough

Updates the e2e provider harness JSON test collection: file response classification switches from j.object to j.type for Anthropic file shapes, and new file upload, preview usage, and cleanup test cases are added.

Changes

Anthropic file harness

Layer / File(s)	Summary
File response classification logic tests/e2e/api/collections/provider-harness.json	Switches Anthropic file response detection from j.object === 'file' checks to j.type === 'file'/j.type === 'file_deleted', updating shape labels to anthropic-file-object/anthropic-file-deleted while keeping hasContent = !!j.id.
Anthropic file upload and cleanup test cases tests/e2e/api/collections/provider-harness.json	Adds file upload tests with and without content_type override, stores returned ids, adds [PREVIEW] message and responses tests using one uploaded id, and adds cleanup DELETE requests for both uploaded files.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: akshaydeo, danpiths, roroghost17

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title is concise and matches the main change: Anthropic file API harness test cases.
Description check	✅ Passed	The description includes the required summary, changes, testing, checklist, and risk sections.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch 07-06-chore_adds_harness_test_cases_for_file_apis_in_anthropic

---

_{Comment @coderabbitai help to get the list of available commands.}

[sammaji]

• chore: adds harness test cases for file apis in anthropic #4967 👈 (View in Graphite)
• fix: adds file id for anthropic files integration and adds forwarding of content type  #4956
• chore: renames NewEnvVar to NewSecretVar in docs #4955
• dev

This stack of pull requests is managed by Graphite. Learn more about stacking.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[coderabbitai]

🧹 Nitpick comments (1)

tests/e2e/api/collections/provider-harness.json (1)

14899-15024: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Add assertions that content_type override/inference actually took effect.

Both new upload tests ("with content_type override" and "content_type inferred from file part") only assert status is 200 and that j.id is a non-empty string. Per Anthropic's Files API, the upload response includes a mime_type field. Since the stated purpose of these two tests is to exercise explicit vs. inferred content_type behavior, asserting j.mime_type (e.g., text/markdown for the override case) would make the test actually verify the feature under test rather than just verify the upload succeeded.

♻️ Proposed assertion addition (override test)

                       "pm.test('status is 200', function () { pm.response.to.have.status(200); });",
                       "var j = pm.response.json();",
                       "pm.test('has file id', function () { pm.expect(j.id).to.be.a('string').and.not.empty; });",
+                      "pm.test('content_type override applied', function () { pm.expect(j.mime_type).to.equal('text/markdown'); });",
                       "pm.collectionVariables.set('anthropicUploadFileIdA', j.id);"

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/e2e/api/collections/provider-harness.json` around lines 14899 - 15024,
The two Anthropic upload tests only verify success and file ID, so they do not
confirm the content_type behavior they are meant to cover. Update the test
scripts in the "Anthropic: file upload with content_type override" and
"Anthropic: file upload content_type inferred from file part" cases to assert
the response's mime_type field, using the existing pm.response.json() result and
the file upload response shape. Make the override case check the expected
explicit type and the inferred case check the type derived from the uploaded
file, so these tests actually validate the feature under test.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/e2e/api/collections/provider-harness.json`:
- Around line 14899-15024: The two Anthropic upload tests only verify success
and file ID, so they do not confirm the content_type behavior they are meant to
cover. Update the test scripts in the "Anthropic: file upload with content_type
override" and "Anthropic: file upload content_type inferred from file part"
cases to assert the response's mime_type field, using the existing
pm.response.json() result and the file upload response shape. Make the override
case check the expected explicit type and the inferred case check the type
derived from the uploaded file, so these tests actually validate the feature
under test.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 08ca6fd5-e486-4a31-95e1-4d972982e05d

📥 Commits

Reviewing files that changed from the base of the PR and between e717c35 and d795008.

📒 Files selected for processing (1)

• tests/e2e/api/collections/provider-harness.json

The base branch was changed.

[greptile-apps]

Confidence Score: 5/5

The change is additive test scaffolding only — no production code is touched, and the additions are consistent with the collection's established patterns.

All new test steps follow the existing harness conventions: upload tests assert HTTP 200 and a non-empty id, cleanup DELETE steps assert type === 'file_deleted' and id round-trip (matching the Vertex cleanup pattern), fixture paths mirror existing entries, and the two [PREVIEW] exploratory requests correctly omit event arrays in line with every other [PREVIEW]-tagged request in the collection. The previously-raised concern about missing delete assertions has been resolved in this PR.

No files require special attention.

Important Files Changed

Filename	Overview
tests/e2e/api/collections/provider-harness.json	Adds Anthropic Files API test coverage: shape detection for file/file_deleted objects, two upload tests, two PREVIEW functional tests, and two DELETE cleanup tests with proper assertions matching the Vertex cleanup pattern.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant N as Newman
    participant B as Bifrost proxy
    participant A as Anthropic Files API

    N->>B: POST /anthropic/v1/files (content_type: text/markdown)
    B->>A: upload multipart/form-data
    A-->>B: "{type:"file", id: fileIdA}"
    B-->>N: "200 {type:"file", id: fileIdA}"
    Note over N: set anthropicUploadFileIdA

    N->>B: POST /anthropic/v1/files (content_type inferred)
    B->>A: upload multipart/form-data
    A-->>B: "{type:"file", id: fileIdB}"
    B-->>N: "200 {type:"file", id: fileIdB}"
    Note over N: set anthropicUploadFileIdB

    N->>B: "[PREVIEW] POST /anthropic/v1/messages (document file_id=fileIdB)"
    Note over B: auto-injects files-api-2025-04-14 beta header
    B->>A: messages with file reference
    A-->>B: model response
    B-->>N: 200 (no assertions — PREVIEW)

    N->>B: "[PREVIEW] POST /v1/responses (input_file file_id=fileIdB)"
    B->>A: responses with input_file
    A-->>B: model response
    B-->>N: 200 (no assertions — PREVIEW)

    N->>B: DELETE /anthropic/v1/files/fileIdA
    B->>A: delete file A
    A-->>B: "{type:"file_deleted", id: fileIdA}"
    B-->>N: 200
    Note over N: asserts type=file_deleted, id matches

    N->>B: DELETE /anthropic/v1/files/fileIdB
    B->>A: delete file B
    A-->>B: "{type:"file_deleted", id: fileIdB}"
    B-->>N: 200
    Note over N: asserts type=file_deleted, id matches

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant N as Newman
    participant B as Bifrost proxy
    participant A as Anthropic Files API

    N->>B: POST /anthropic/v1/files (content_type: text/markdown)
    B->>A: upload multipart/form-data
    A-->>B: "{type:"file", id: fileIdA}"
    B-->>N: "200 {type:"file", id: fileIdA}"
    Note over N: set anthropicUploadFileIdA

    N->>B: POST /anthropic/v1/files (content_type inferred)
    B->>A: upload multipart/form-data
    A-->>B: "{type:"file", id: fileIdB}"
    B-->>N: "200 {type:"file", id: fileIdB}"
    Note over N: set anthropicUploadFileIdB

    N->>B: "[PREVIEW] POST /anthropic/v1/messages (document file_id=fileIdB)"
    Note over B: auto-injects files-api-2025-04-14 beta header
    B->>A: messages with file reference
    A-->>B: model response
    B-->>N: 200 (no assertions — PREVIEW)

    N->>B: "[PREVIEW] POST /v1/responses (input_file file_id=fileIdB)"
    B->>A: responses with input_file
    A-->>B: model response
    B-->>N: 200 (no assertions — PREVIEW)

    N->>B: DELETE /anthropic/v1/files/fileIdA
    B->>A: delete file A
    A-->>B: "{type:"file_deleted", id: fileIdA}"
    B-->>N: 200
    Note over N: asserts type=file_deleted, id matches

    N->>B: DELETE /anthropic/v1/files/fileIdB
    B->>A: delete file B
    A-->>B: "{type:"file_deleted", id: fileIdB}"
    B-->>N: 200
    Note over N: asserts type=file_deleted, id matches

Loading

_{Reviews (2): Last reviewed commit: "chore: adds harness test cases for file ..." | Re-trigger Greptile}

[akshaydeo]

Merge activity

• Jul 6, 6:05 PM UTC: A user started a stack merge that includes this pull request via Graphite.
• Jul 6, 6:06 PM UTC: @akshaydeo merged this pull request with Graphite.