[AI] Fix multi-turn tool calling, add chat overlay, upgrade Agent Framework to rc2, and add CI pipelines#34124
Merged
mattleibow merged 75 commits intomainfrom Mar 3, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes Apple Intelligence multi-turn tool-calling transcript/streaming issues and adds an AI chat overlay (with tool calling) to the Essentials.AI sample app, plus small test/dev tooling updates.
Changes:
- Add multi-turn tool-calling device tests and update Apple Intelligence native/managed interop to better handle tool-call/tool-result content.
- Reset streaming chunker state across tool boundaries to avoid dropped post-tool text.
- Introduce a sample chat overlay UI + view model/service wiring, and extend repo scripts/skills to run AI device tests and parameterize Sandbox runs.
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| src/AI/tests/Essentials.AI.DeviceTests/Tests/ChatClientFunctionCallingTests.cs | Adds new multi-turn tool-calling device tests. |
| src/AI/src/Essentials.AI/Platform/StreamChunkerBase.cs | Adds Reset() contract for stream chunkers. |
| src/AI/src/Essentials.AI/Platform/PlainTextStreamChunker.cs | Implements Reset() for plain-text delta chunking. |
| src/AI/src/Essentials.AI/Platform/MaciOS/AppleIntelligenceChatClient.cs | Resets chunker on tool calls; extends content conversion for function call/result history. |
| src/AI/src/Essentials.AI/Platform/JsonStreamChunker.cs | Implements Reset() for JSON delta chunking state. |
| src/AI/src/AppleNative/EssentialsAI/ChatMessageContent.swift | Extends native function-result content with a name property. |
| src/AI/src/AppleNative/EssentialsAI/ChatClient.swift | Fixes prompt selection for tool-loop scenarios; builds transcript entries for tool calls/outputs. |
| src/AI/src/AppleNative/ApiDefinitions.cs | Updates binding for FunctionResultContentNative name and designated initializer signature. |
| src/AI/samples/Essentials.AI.Sample/Views/ChatOverlayView.xaml.cs | Adds overlay show/hide + auto-scroll behavior. |
| src/AI/samples/Essentials.AI.Sample/Views/ChatOverlayView.xaml | Defines overlay UI and chat bubble templates. |
| src/AI/samples/Essentials.AI.Sample/Views/ChatBubbleTemplateSelector.cs | Adds template selector for user/assistant/tool bubbles. |
| src/AI/samples/Essentials.AI.Sample/ViewModels/ChatViewModel.cs | Adds streaming chat view model with tool-call/result bubble handling. |
| src/AI/samples/Essentials.AI.Sample/Services/ChatService.cs | Adds tool-backed chat service (landmarks/weather/tags/language/plan-trip). |
| src/AI/samples/Essentials.AI.Sample/Pages/TripPlanningPage.xaml.cs | Adds FAB + overlay wiring on TripPlanningPage. |
| src/AI/samples/Essentials.AI.Sample/Pages/TripPlanningPage.xaml | Adds FAB button to TripPlanningPage layout. |
| src/AI/samples/Essentials.AI.Sample/Pages/LandmarksPage.xaml.cs | Adds FAB + overlay wiring and “plan trip” navigation hook. |
| src/AI/samples/Essentials.AI.Sample/Pages/LandmarksPage.xaml | Adds FAB button to LandmarksPage layout. |
| src/AI/samples/Essentials.AI.Sample/Models/ChatBubble.cs | Adds chat bubble model + expand/collapse command. |
| src/AI/samples/Essentials.AI.Sample/MauiProgram.cs | Registers ChatViewModel and ChatService in DI. |
| .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 | Adds AI device test project and improves artifact path derivation. |
| .github/skills/run-device-tests/SKILL.md | Documents AI as a valid device test project target. |
| .github/scripts/BuildAndRunSandbox.ps1 | Parameterizes project/bundle/app/test-dir and updates artifact/log handling. |
src/AI/src/Essentials.AI/Platform/MaciOS/AppleIntelligenceChatClient.cs
Outdated
Show resolved
Hide resolved
mattleibow
force-pushed
the
dev/fix-apple-tools
branch
from
1444b8b to
cabef52
Compare
mattleibow
force-pushed
the
dev/fix-apple-tools
branch
from
f7866e8 to
d348317
Compare
Fix two issues that caused multi-turn conversations with tool calling to fail: 1. C# side: Add FunctionCallContent/FunctionResultContent to native conversion and filter empty messages from conversation history. 2. Swift side: Change prepareSession to find the last User message as role message after FunctionInvokingChatClient processes tool results. Also add Name property to FunctionResultContentNative binding and support ToolCall/ToolCalls/ToolOutput transcript entries in Swift. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add ChatClientFunctionCallingTestsBase with three tests: - GetResponseAsync_WithToolCall_ReturnsToolAndTextContent - GetResponseAsync_MultiTurnWithTools_HandlesConversationHistory - GetStreamingResponseAsync_WithToolCall_StreamsToolAndText Tests verify that function call/result content flows correctly through the Apple Intelligence chat client and that follow-up messages after tool execution succeed without errors. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Apple Intelligence's FoundationModels resets the cumulative text stream after native tool execution. The PlainTextStreamChunker was computing deltas assuming monotonic growth, which caused the first N characters of post-tool text to be silently dropped (where N = length of pre-tool text). For example, 'Here are some...' lost 'Here' because the chunker treated it as overlapping with the pre-tool 'null' output. Add Reset() to StreamChunkerBase and call it when a ToolCall update arrives, so post-tool text is treated as a fresh stream. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add a floating action button (FAB) chat interface to the AI sample app with a streaming chat overlay that supports tool calling. Includes: - ChatBubble model with 4 types (User/Assistant/ToolCall/ToolResult) - ChatViewModel with streaming, tool deduplication, and junk bubble cleanup - ChatOverlayView with collapsible tool bubbles and animations - ChatService with 8 tools: search_landmarks, list_landmarks_by_continent, get_landmark_details, search_points_of_interest, get_weather, generate_tags, set_language, plan_trip - FAB buttons on LandmarksPage and TripPlanningPage - Navigation event for plan_trip tool Tools use Apple Intelligence native tool calling (not FunctionInvokingChatClient) since FoundationModels handles the tool loop directly via ToolNative/ToolCallWatcher. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add AI project support to Run-DeviceTests.ps1 and update SKILL.md - Fix artifact path bug in Run-DeviceTests.ps1 for AI project - Parameterize BuildAndRunSandbox.ps1 with -ProjectPath, -BundleId, -AppName, -TestDir to support any MAUI sample app (defaults to Sandbox app for backward compatibility) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- 8 tests for PlainTextStreamChunker.Reset() including tool call boundary simulation, multiple resets, idempotency, and multi-segment concatenation - 7 tests for JsonStreamChunker.Reset() including fresh stream after reset, re-emission of cleared strings, and mid-string reset safety - Fix UnitTests.csproj to exclude ChatBubble.cs from Models/** glob (uses MAUI Command type not available in net10.0 test project) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add New Chat (🗑) button in header to clear conversation - Add NewChatCommand to ChatViewModel (cancels pending, clears messages/history) - Responsive panel: 90% width/height with max 400w x 700h for desktop - Panel now HorizontalOptions=Center so it doesn't stretch full width - Dynamic TranslationY for slide animation based on actual panel height - Replace hardcoded 500px height with computed size via SizeChanged Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Changed chat panel HorizontalOptions to End for right alignment - Added 'Thinking...' bubble immediately after user sends message - Fixed post-tool-call text: create new bubble if thinking bubble was removed - Validated with Appium: Africa landmarks query works end-to-end Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Replace SizeChanged width calculation with XAML MaximumWidthRequest=500 - Keep minimal SizeChanged for height only (MAUI End-aligned elements overflow) - Round all corners for floating bubble appearance - Add MarkdownConverter for assistant bubbles (bold, italic, code spans) - Panel: 500w x 800h max, responsive on small windows Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Replace regex-based markdown converter with Markdig AST walker - Supports bold, italic, code, lists, headings, links, code blocks - Unsupported elements fall back to plain text Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add today's date to system prompt instead of separate GetCurrentDate tool - Change weather date param from string to DateTimeOffset for typed schema - Remove ResolveDate string parsing — AI computes dates from system prompt - Fix responsive panel sizing with SizeChanged (MAUI centers MaxWidth-capped elements) - Validate date range (today through +7 days) with clear error messages Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add FunctionCallContent and FunctionResultContent to conversation history so the Apple Intelligence transcript includes the complete tool interaction chain on subsequent turns. Previously only TextContent was kept, causing the model to lose context about which tools were called and what they returned. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Test the exact pattern ChatViewModel uses: collect streaming content, build history by role (Assistant→FunctionCallContent, Tool→FunctionResultContent, Assistant→TextContent), then send follow-up. Verifies the model can reference specific tool results (47°F) from prior turns. Both tests pass on MacCatalyst: 112 total, 110 passed, 0 failed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Instead of collecting all content into a temp list and grouping by type (which reorders call→result→call→result into call→call→result→result), add each item to _conversationHistory as it streams in. This preserves the natural interleaved order of tool call/result pairs. Removes allContents temp list and post-stream processing block. Same fix applied to device tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix Apple Intelligence 'null' text sentinel: Filter at Swift layer instead of C# — guard in streamResponse loop skips empty/null content before tool calls. Removed all 'null' text string filters from C# and ViewModel. - Fix FunctionResultContent tool name: Build callId→name lookup from FunctionCallContent so FunctionResultContent passes correct tool name to Swift's Transcript.ToolOutput. - Fix thread safety: PlanTripAsync dispatches NavigateToTripRequested to main thread via MainThread.BeginInvokeOnMainThread. - Fix timezone: Use date.DateTime instead of date.LocalDateTime to prevent DateTimeOffset→DateOnly shifting to previous day in non-UTC timezones. - Fix MarkdownConverter bounds: Clamp Markdig Span.Start/Length to text length to prevent IndexOutOfRangeException. - Fix error message: Improved ArgumentException when all messages filtered. - Add ILogger<ChatViewModel> debug logging for conversation flow tracing. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove ILogger<ChatViewModel> and IDispatcher from ChatViewModel. SendMessageAsync runs on UI thread from button click; await foreach preserves SynchronizationContext, making Dispatch() redundant. - Fix toTranscriptEntries: process assistant message contents in order, flushing batches when content type changes. Preserves interleaving instead of grouping all text first then all tool calls. - Inline toTranscriptSegment into toTranscriptEntries (single pass). - Add device test: no 'null' text leaks through streaming pipeline during tool-calling conversations. - Add device test: content order preserved (call→result pairs stay adjacent) through streaming history building. - All tests pass: 290 unit, 112 device (110 passed, 2 skipped). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Removed text != "null" guards from both Swift streaming loops. Device test GetStreamingResponseAsync_WithToolCalling_NoNullTextContent confirms no 'null' text leaks through during tool-calling streams. Investigation with 5 AI models + empirical testing shows: - Text path: StreamResponse<String>.content is String, empty = "" - The original 'null' observation was likely caused by other bugs since fixed (chunker reset, history ordering, prompt extraction) - GPT-5.1-Codex: 'null' only occurs on schema path via .jsonString on empty GeneratedContent — not applicable to text streaming Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Always remove pre-tool assistant bubble when tool calls arrive instead of conditionally keeping it based on text content - Add ViewModelSimulation test that replicates ChatViewModel state machine through full middleware pipeline (wrap→FICC→unwrap) - Add raw client null text test (3 runs) for landmarks/Africa query - Add comprehensive stream capture test for null text detection Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Extract onUpdate logic into static local functions ProcessStreamUpdate and CompleteStream to reduce nesting. Wrap both callbacks in try/catch routing exceptions to channel.Writer.TryComplete(ex) to prevent unhandled managed exceptions from crossing the native Swift boundary. Change all channel.Writer.Complete() calls to TryComplete() so that if onUpdate already completed the channel with an error, the subsequent onComplete callback does not throw ChannelClosedException. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ToNative enumerates messages twice: once for callIdToName lookup and once for conversion. If a caller passes a forward-only IEnumerable, the second enumeration yields nothing. Materialize with .ToList() at method entry, then Prepend the system instruction afterward. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When a ToolCall update arrives during streaming, call chunker.Flush() before chunker.Reset() to emit any buffered content. For JsonStreamChunker, Reset() silently discards pending strings, containers, and open structures that Flush() would have emitted. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- StreamingResponseHandler: encapsulates channel, chunker, ProcessUpdate, Complete - NonStreamingResponseHandler: encapsulates TCS, Complete, CompleteCancelled - StreamUpdate record struct decouples handler from native ResponseUpdateNative - 16 new unit tests (10 streaming, 6 non-streaming) covering: - Content deltas, plain text chunking, tool call emit, tool result role - Flush before tool call, malformed JSON error, double completion safety - Post-tool fresh stream, empty text filtering - Valid/null response, error surfacing, cancellation, multiple messages - InternalsVisibleTo for device test project - All 137 tests pass (135 passed, 2 ignored) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This was referenced Mar 16, 2026
Bump Microsoft.Maui.Controls from 10.0.30 to 10.0.50
TheCodeTraveler/BecomeAnExpertWithAsyncAwait#11
Open
Open
This was referenced Mar 26, 2026
Open
Open
This was referenced Apr 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!
Summary
This PR fixes multi-turn tool calling in
AppleIntelligenceChatClient, adds a chat overlay UI to the AI sample app, upgrades dependencies (Microsoft Agent Framework to rc2, Microsoft.Extensions.AI to 10.3.0), and integrates AI tests into CI pipelines.Changes
1. Fix multi-turn tool calling in AppleIntelligenceChatClient
When a conversation included tool calls, the next user message would fail with
"The content type '...' is not supported by Apple Intelligence chat APIs"becauseFunctionCallContentandFunctionResultContentfrom previous turns were wrapped in opaqueServerFunctionCallContent/ServerFunctionResultContenttypes that the native Apple Intelligence API could not process.Fix: Convert
FunctionCallContentandFunctionResultContentdirectly to their native counterparts (FunctionCallContentNative,FunctionResultContentNative) and pass them to the Swift layer. The Swift layer maps these to properTranscript.Entrytypes (.toolCalls,.toolOutput) that Apple'sLanguageModelSessionunderstands. AcallId → namelookup dictionary is built fromFunctionCallContententries so thatFunctionResultContent(which doesn't carry the tool name) can reference the correct tool.The old approach of wrapping function content in
ServerFunctionCallContent/ServerFunctionResultContentis removed entirely.Refactoring for testability:
StreamingResponseHandler— manages channel, chunker, and update processing for streaming responsesNonStreamingResponseHandler— manages TaskCompletionSource for non-streaming responsesNew public API:
AppleIntelligenceChatClient(ILoggerFactory?, IServiceProvider?)constructor overload for dependency injection of logging and function invocation services.2. Tool call logging (InformationalOnly pattern)
Replace the custom
NonFunctionInvokingChatClientwrapper (deleted) with the built-inFunctionCallContent.InformationalOnlyproperty from Microsoft.Extensions.AI. Tool calls from Apple Intelligence are logged as informational-onlyFunctionCallContententries in the conversation history, making them visible to the caller withoutFunctionInvokingChatClientattempting to invoke them again.Deleted files:
NonFunctionInvokingChatClient.cs(230 lines)NonFunctionInvokingChatClientTests/FunctionLoggingTests.cs(281 lines)New tests:
InformationalOnlyToolCallLoggingTests— verifies informational-only tool calls are logged correctly and not re-invoked by FICCInvocableToolCallLoggingTests— verifies invocable tool calls produce both call and result contentAppleIntelligenceChatClientToolCallLoggingTests— device tests for on-device tool call logging3. Chat overlay UI in sample app
Add a reusable chat overlay with markdown rendering for the AI sample app:
ChatOverlayView— XAML overlay with message bubbles, typing indicatorsChatViewModel/ChatBubbleViewModel— MVVM view modelsChatService— service layer connecting to the AI agentChatBubbleTemplateSelector— user vs. assistant bubble stylingMarkdownConverter— converts markdown to MAUI formatted textDataService— data provider for landmarks and points of interestLandmarksViewModel— updated for chat integrationLandmarksPageandTripPlanningPage4. Multi-turn function calling device tests
New device tests in
ChatClientFunctionCallingTestscovering:New device tests in
AppleIntelligenceChatClientValidationTestsfor input validation.Total new device test lines: ~1,390.
5. Upgrade dependencies
Microsoft.Extensions.AI:
Microsoft.Extensions.AI: 10.0.1 → 10.3.0Microsoft.Extensions.AI.Abstractions: 10.0.1 → 10.3.0Microsoft Agent Framework:
Microsoft.Agents.AI: 1.0.0-preview.251204.1 → 1.0.0-rc2Microsoft.Agents.AI.Workflows: new package, 1.0.0-rc2 (previously bundled with Agents.AI)Microsoft.Agents.AI.Workflows.Generators: new package, 1.0.0-rc2Microsoft.Agents.AI.Hosting: new package, 1.0.0-preview.260225.1 (previously bundled with Agents.AI)rc2 migration changes:
In rc2,
ChatProtocolExecutorsetsAutoSendMessageHandlerResultObject = falseand only declaresList<ChatMessage>andTurnTokenin its protocol. Custom types sent viacontext.SendMessageAsync()must be explicitly declared by overridingConfigureProtocol. Without this,TravelPlanResultwas silently dropped by the edge router (DroppedTypeMismatch) and executor 2 never received it.ConfigureProtocolto addSendsMessage<TravelPlanResult>()inTravelPlannerExecutorAutoSendTurnToken = falsesince downstream executors are typed and don't handleTurnTokenResponseFormat+JsonSerializer.Deserialize<T>()pattern withagent.RunAsync<T>()which handles JSON schema and deserialization automatically (rc2 feature)ItineraryWorkflowExtensionsconfigurationItineraryWorkflowToolshelper class6. Swift native layer improvements
ChatClient.swift: Refactored singletoTranscriptEntrymethod into per-role functions (toUserEntry,toAssistantEntries,toSystemEntry,toToolEntries) for clearer code and proper handling of interleaved content (text + tool calls in assistant messages produce multipleTranscript.Entryitems)JsonSchemaDecoder.swift: ReplacecompactMapwith explicit error handling that surfaces decode failures instead of silently dropping propertiesChatMessageContent.swift: Addnamefield toFunctionResultContentNativeso tool results carry the tool name back to the managed layer7. Stream chunker improvements
Reset()method toJsonStreamChunker,PlainTextStreamChunker, andStreamChunkerBaseStreamingResponseHandler, chunker is flushed before reset on tool calls to prevent partial content lossJsonStreamChunkerTests/ResetandPlainTextStreamChunkerTests/Reset)8. CI pipeline integration for AI tests
Register
Essentials.AI.DeviceTestsandEssentials.AI.UnitTestsin CI pipelines:Device tests (all 4 platforms: iOS, MacCatalyst, Android, Windows):
eng/Build.props— added toBuildDeviceTestsproject listeng/helix_xharness.proj— added MAUIScenario "EssentialsAI" + iOSXHarnessAppBundleToTestentryeng/pipelines/arcade/stage-device-tests.yml— added towindowsDeviceTestProjects,$artifactNames, and$projectszip arrayseng/pipelines/device-tests.yml— added legacy pipeline entry for all 4 platformsProperties/launchSettings.json— required by Windows Cake build script for packaged/unpackaged toggleUnit tests:
eng/helix.proj— addedXUnitProjectfor Helix submissioneng/cake/dotnet.cake— added todotnet-testtarget for local runsTest path fix:
DataStreamsHelper.GetTxtItinerary()— fixed source-tree relative path (../../../../../src/...) to use output-directory relative path (TestData/DataStreams/Itinerary/) for Helix compatibilityRun-device-tests skill:
Run-DeviceTests.ps1andSKILL.mdTest coverage
9. Weak-link FoundationModels.framework for backward compatibility
The device test app (and any app using Apple Intelligence APIs) must weak-link
FoundationModels.frameworkso it can launch on iOS < 26 / macOS < 26 without crashing withLibrary not loaded: /System/Library/Frameworks/FoundationModels.framework/FoundationModels.Problem: The .NET iOS SDK's
CreateXcArchivetask hardcodesOTHER_LDFLAGS="-ObjC"as a readonly constant, completely overriding anyOTHER_LDFLAGSset in the Xcode project's build settings. This means adding-weak_framework FoundationModelstoOTHER_LDFLAGSinproject.pbxprojhas no effect.Fix: Add
FoundationModels.frameworkto thePBXFrameworksBuildPhasesection inproject.pbxprojwithATTRIBUTES = (Weak). Xcode processes framework build phases independently ofOTHER_LDFLAGS, so this producesLC_LOAD_WEAK_DYLIBin the binary (verified withotool -l).10. Test skip infrastructure for Apple Intelligence
Apple Intelligence tests require iOS 26+ / macOS 26+ with on-device model availability. Tests must be skipped on older OS versions and on CI (where models aren't available).
Traits.cs— Central test category constants (AppleIntelligenceChatClient,NLEmbeddingGenerator,OpenAIChatClient,OpenAIEmbeddingGenerator) andGetSkipTraits()which:OperatingSystem.IsIOSVersionAtLeast(26)/IsMacCatalystVersionAtLeast(26)/IsMacOSVersionAtLeast(26)TestFilterenvironment variable fromNSProcessInfo(set by Helix/XHarness via--set-env)Helix configuration (
eng/helix_xharness.proj):AITestCategoriesToSkipOnCIproperty lists categories to skip on CIXHarnessAppBundleToTestentries useCustomCommandswith--set-env="TestFilter=SkipCategories=$(AITestCategoriesToSkipOnCI)"SmokeTests.cs— SingleTestInfrastructureWorkstest ensuring test discovery works on all platforms including Windows (where no Apple Intelligence or OpenAI tests compile).Validation test fixes —
AppleIntelligenceChatClientValidationTestsupdated to match M.E.AI 10.3.0 behavior:TextContent(null).Textreturns""(not null) due to null-coalescing getter, so null text content passes through filtering without throwingFunctionCallContentconstructor validatesname != nullviaThrow.IfNull, so tests use empty string instead ofnull!See also: #34298 (tracking Apple Intelligence CI model availability)