.NET: Fix failing vision integration tests by using local test files#4128
.NET: Fix failing vision integration tests by using local test files#4128
Conversation
Co-authored-by: westey-m <164392973+westey-m@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR aims to stabilize the vision integration tests by removing reliance on downloading external media during test execution and instead using locally committed test assets.
Changes:
- Added minimal PDF and JPEG fixtures under
TestFiles/for use by integration tests. - Updated
MediaInputTestto read file bytes from local assets (removing theHttpClientdownload helper). - Updated the integration test
.csprojto copyTestFiles\*into the test output directory.
Reviewed changes
Copilot reviewed 2 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/MediaInputTest.cs | Switches file-data and upload tests from remote downloads to local file reads; renames URL constant for clarity. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.csproj | Copies TestFiles/* to output so tests can access fixtures at runtime. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/TestFiles/basic-text.pdf | Adds a minimal PDF fixture for file-data tests. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/TestFiles/test-image.jpg | Adds a minimal JPEG fixture for image file-data tests. |
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/MediaInputTest.cs
Outdated
Show resolved
Hide resolved
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/MediaInputTest.cs
Show resolved
Hide resolved
|
@copilot, the build is failing. Please fix the build issues and make sure to build the solution to ensure that it builds correctly. |
…e helper using AppContext.BaseDirectory Co-authored-by: westey-m <164392973+westey-m@users.noreply.github.com>
... Fixed in commit 9935b69. Replaced |
dotnet/tests/Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests/MediaInputTest.cs
Outdated
Show resolved
Hide resolved
Co-authored-by: westey-m <164392973+westey-m@users.noreply.github.com>
Motivation and Context
ValidateFileDataAsyncandValidateFileUploadAsyncwere intermittently failing becausesample-files.comoccasionally returns HTML instead of the requested PDF/JPEG, causing the AI service to reject the content withunsupported_file.Description
Replace all external file downloads in
MediaInputTestwith local files committed to the repository.TestFiles/basic-text.pdf— minimal valid PDF 1.4 (single page, "Hello World" text stream)TestFiles/test-image.jpg— minimal valid JPEG (1×1 grayscale pixel).csproj— addedTestFiles\*withCopyToOutputDirectory=AlwaysMediaInputTest.cs:DownloadFileAsync()andSystem.Net.HttpdependencyReadLocalFilehelper that resolves paths viaPath.Combine(AppContext.BaseDirectory, relativePath)and returnsbyte[]synchronously usingFile.ReadAllBytes— compatible with all target frameworks includingnet472ValidateFileDataAsync,ValidateImageFileDataAsync,ValidateFileUploadAsyncnow useReadLocalFilewith local pathsImageReference→ImageReferenceUrlto distinguish URL constants from local path constantsValidateFileUrlAsyncleft unchanged (passes URL directly toUriContent, no download involved)Contribution Checklist
Original prompt
This section details on the original issue you should resolve
<issue_title>Fix failing vision integration tests</issue_title>
<issue_description>Two integration tests are failing regularly:
Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.MediaInputTest.ValidateFileDataAsync
Microsoft.Agents.AI.Workflows.Declarative.IntegrationTests.MediaInputTest.ValidateFileUploadAsync
The content returned by the samples-files.com service is not consistently the data that was requested, specifically, some times it's html instead of an pdf of jpg file.
To improve reliability we should download the files, check them in and use them from the local repository.
Failed output: