Skip to content

.NET: Fix failing vision integration tests by using local test files#4128

Merged
westey-m merged 4 commits intomainfrom
copilot/fix-vision-integration-tests
Feb 20, 2026
Merged

.NET: Fix failing vision integration tests by using local test files#4128
westey-m merged 4 commits intomainfrom
copilot/fix-vision-integration-tests

Conversation

Copy link
Contributor

Copilot AI commented Feb 20, 2026

Motivation and Context

ValidateFileDataAsync and ValidateFileUploadAsync were intermittently failing because sample-files.com occasionally returns HTML instead of the requested PDF/JPEG, causing the AI service to reject the content with unsupported_file.

Description

Replace all external file downloads in MediaInputTest with local files committed to the repository.

  • Added TestFiles/basic-text.pdf — minimal valid PDF 1.4 (single page, "Hello World" text stream)
  • Added TestFiles/test-image.jpg — minimal valid JPEG (1×1 grayscale pixel)
  • .csproj — added TestFiles\* with CopyToOutputDirectory=Always
  • MediaInputTest.cs:
    • Removed DownloadFileAsync() and System.Net.Http dependency
    • Added ReadLocalFile helper that resolves paths via Path.Combine(AppContext.BaseDirectory, relativePath) and returns byte[] synchronously using File.ReadAllBytes — compatible with all target frameworks including net472
    • ValidateFileDataAsync, ValidateImageFileDataAsync, ValidateFileUploadAsync now use ReadLocalFile with local paths
    • Renamed ImageReferenceImageReferenceUrl to distinguish URL constants from local path constants
    • ValidateFileUrlAsync left unchanged (passes URL directly to UriContent, no download involved)

Contribution Checklist

  • The code builds clean without any errors or warnings
  • The PR follows the Contribution Guidelines
  • All unit tests pass, and I have added new tests where possible
  • Is this a breaking change? If yes, add "[BREAKING]" prefix to the title of the PR.
Original prompt

This section details on the original issue you should resolve

<issue_title>Fix failing vision integration tests</issue_title>
<issue_description>Two integration tests are failing regularly:
Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.MediaInputTest.ValidateFileDataAsync
Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.MediaInputTest.ValidateFileUploadAsync

The content returned by the samples-files.com service is not consistently the data that was requested, specifically, some times it's html instead of an pdf of jpg file.

To improve reliability we should download the files, check them in and use them from the local repository.

Failed output:

  Failed Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.MediaInputTest.ValidateFileDataAsync(fileSource: "https://sample-files.com/downloads/documents/pdf/b"···, mediaType: "application/pdf", useConversation: False) [4 s]
  Error Message:
   Microsoft.Agents.AI.Workflows.Declarative.DeclarativeActionException : Unhandled workflow failure - #invoke_vision (InvokeAzureAgent)
---- System.ClientModel.ClientResultException : HTTP 400 (invalid_request_error: unsupported_file)
Parameter: input

The file type you uploaded is not supported. Please try again with a pdf
  Stack Trace:
     at Microsoft.Agents.AI.Workflows.Declarative.Interpreter.DeclarativeActionExecutor.HandleAsync(ActionExecutorResult message, IWorkflowContext context, CancellationToken cancellationToken) in /home/runner/work/agent-framework/agent-framework/dotnet/src/Microsoft.Agents.AI.Workflows.Declarative/Interpreter/DeclarativeActionExecutor.cs:line 100
   at Microsoft.Agents.AI.Workflows.Declarative.Interpreter.DeclarativeActionExecutor.HandleAsync(ActionExecutorResult message, IWorkflowContext context, CancellationToken cancellationToken) in /home/runner/work/agent-framework/agent-framework/dotnet/src/Microsoft.Agents.AI.Workflows.Declarative/Interpreter/DeclarativeActionExecutor.cs:line 106
   at Microsoft.Agents.AI.Workflows.RouteBuilder.<>c__DisplayClass12_0`1.<<AddHandler>g__WrappedHandlerAsync|0>d.MoveNext() in /home/runner/work/agent-framework/agent-framework/dotnet/src/Microsoft.Agents.AI.Workflows/RouteBuilder.cs:line 250
--- End of stack trace from previous location ---
   at Microsoft.Agents.AI.Workflows.Execution.MessageRouter.RouteMessageAsync(Object message, IWorkflowContext context, Boolean requireRoute, CancellationToken cancellationToken) in /home/runner/work/agent-framework/agent-framework/dotnet/src/Microsoft.Agents.AI.Workflows/Execution/MessageRouter.cs:line 137
----- Inner Stack Trace -----
   at OpenAI.ClientPipelineExtensions.ProcessMessageAsync(ClientPipeline pipeline, PipelineMessage message, RequestOptions options)
   at OpenAI.Responses.ResponsesClient.CreateResponseAsync(BinaryContent content, RequestOptions options)
   at OpenAI.Responses.ResponsesClient.<>c__DisplayClass20_0.<<CreateResponseStreamingAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at OpenAI.AsyncSseUpdateCollection`1.GetRawPagesAsync()+MoveNext()
   at OpenAI.AsyncSseUpdateCollection`1.GetRawPagesAsync()+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at System.ClientModel.AsyncCollectionResult`1.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()
   at System.ClientModel.AsyncCollectionResult`1.GetAsyncEnumerator(CancellationToken cancellationToken)+MoveNext()
   at System.ClientModel.AsyncCollectionResult`1.GetAsyncEnumerator(CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at Microsoft.Extensions.AI.OpenAIResponsesChatClient.FromOpenAIStreamingResponseUpdatesAsync(IAsyncEnumerable`1 streamingResponseUpdates, CreateResponseOptions options, String conversationId, String resumeResponseId, CancellationToken cancellationToken)+MoveNext()
   at Microsoft.Extensions.AI.OpenAIResponsesChatClient.FromOpenAIStreamingResponseUpdatesAsync(IAsyncEnumerable`1 streamingResponseUpdates, CreateResponseOptions options, String conversationId, String resumeResponseId, CancellationToken cancellationToken)+MoveNext()
   at Microsoft.Extensions.AI.OpenAIResponsesChatClient.FromOpenAIStreamingResponseUpdatesAsync(IAsyncEnumerable`1 streamingResponseUpdates, CreateResponseOptions options, String conversationId, String resumeResponseId, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at Microsoft.Agents.AI.AzureAI.AzureAIProjectChatClient.GetStreamingResponseAsync(IEnumerable`1 messages, ChatOptions options, CancellationToken cancellationToken)+MoveNext() in /home/runner/work/agent-framework/agent-framework/dotnet/src/Microsoft.Agents.AI.AzureAI/AzureAIProjectChatClient.cs:line 118
   at Microsoft.Agents.AI.AzureAI.AzureAIProjectChatClient.GetStreamingResponseAsync(IEnumerable`1 messages, ChatOptions options, CancellationToken cancellationToken)+MoveNext() in /home/r...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

- Fixes microsoft/agent-framework#4127

<!-- START COPILOT CODING AGENT TIPS -->
---

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. [Learn more about Advanced Security.](https://gh.io/cca-advanced-security)

Co-authored-by: westey-m <164392973+westey-m@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix failing vision integration tests Fix failing vision integration tests by using local test files Feb 20, 2026
Copilot AI requested a review from westey-m February 20, 2026 12:38
@westey-m westey-m marked this pull request as ready for review February 20, 2026 12:39
Copilot AI review requested due to automatic review settings February 20, 2026 12:39
@github-actions github-actions bot changed the title Fix failing vision integration tests by using local test files .NET: Fix failing vision integration tests by using local test files Feb 20, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to stabilize the vision integration tests by removing reliance on downloading external media during test execution and instead using locally committed test assets.

Changes:

  • Added minimal PDF and JPEG fixtures under TestFiles/ for use by integration tests.
  • Updated MediaInputTest to read file bytes from local assets (removing the HttpClient download helper).
  • Updated the integration test .csproj to copy TestFiles\* into the test output directory.

Reviewed changes

Copilot reviewed 2 out of 4 changed files in this pull request and generated 2 comments.

File Description
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/MediaInputTest.cs Switches file-data and upload tests from remote downloads to local file reads; renames URL constant for clarity.
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.csproj Copies TestFiles/* to output so tests can access fixtures at runtime.
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/TestFiles/basic-text.pdf Adds a minimal PDF fixture for file-data tests.
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/TestFiles/test-image.jpg Adds a minimal JPEG fixture for image file-data tests.

@westey-m
Copy link
Contributor

@copilot, the build is failing. Please fix the build issues and make sure to build the solution to ensure that it builds correctly.

Error: /home/runner/work/agent-framework/agent-framework/dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/MediaInputTest.cs(47,38): error CS0117: 'File' does not contain a definition for 'ReadAllBytesAsync' [/home/runner/work/agent-framework/agent-framework/dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.csproj::TargetFramework=net472]
Error: /home/runner/work/agent-framework/agent-framework/dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/MediaInputTest.cs(62,38): error CS0117: 'File' does not contain a definition for 'ReadAllBytesAsync' [/home/runner/work/agent-framework/agent-framework/dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.csproj::TargetFramework=net472]
Error: /home/runner/work/agent-framework/agent-framework/dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/MediaInputTest.cs(79,38): error CS0117: 'File' does not contain a definition for 'ReadAllBytesAsync' [/home/runner/work/agent-framework/agent-framework/dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.csproj::TargetFramework=net472]

…e helper using AppContext.BaseDirectory

Co-authored-by: westey-m <164392973+westey-m@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Feb 20, 2026

@copilot, the build is failing. Please fix the build issues and make sure to build the solution to ensure that it builds correctly.

...

Fixed in commit 9935b69. Replaced File.ReadAllBytesAsync (not available in net472) with a ReadLocalFileAsync helper that uses File.ReadAllBytes wrapped in Task.FromResult — which is compatible with all target frameworks. The helper also resolves paths from AppContext.BaseDirectory for robustness across different runners/CI configurations.

Co-authored-by: westey-m <164392973+westey-m@users.noreply.github.com>
Copilot AI requested a review from westey-m February 20, 2026 13:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants