Python: Fix AG-UI reasoning role and multimodal media parsing to follow specification#5389
Conversation
Fix two spec compliance issues in the AG-UI integration: 1. ReasoningMessageStartEvent now uses role='reasoning' instead of role='assistant', matching the AG-UI specification for reasoning messages. 2. _parse_multimodal_media_part now reads the 'value' field from source dicts (with fallback to 'data' for backward compatibility), matching the current AG-UI InputContentSource specification. Bump ag-ui-protocol dependency from ==0.1.13 to >=0.1.16,<0.2 to pick up the SDK fix that accepts role='reasoning' in ReasoningMessageStartEvent. Fix pre-existing pyright reportMissingImports errors for orjson in sample files, and fix import ordering in foundry-hosted-agents sample. Fixes microsoft#5340 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ow specification Fixes microsoft#5340
moonbox3
left a comment
moonbox3
left a comment
There was a problem hiding this comment.
Automated Code Review
Reviewers: 4 | Confidence: 96% | Result: All clear
Reviewed: Correctness, Security Reliability, Test Coverage, Design Approach
Automated review by moonbox3's agents
Python Test Coverage Report •
Python Unit Test Overview
|
|||||||||||||||||||||||||||||||||||
There was a problem hiding this comment.
Pull request overview
Fixes the Python AG-UI adapter to align with the AG-UI specification for reasoning event roles and multimodal media payload parsing, and updates the AG-UI protocol dependency/tests accordingly.
Changes:
- Emit
ReasoningMessageStartEvent(role="reasoning")(instead of"assistant") in both streaming and non-streaming reasoning paths. - Parse multimodal media payloads from
source.valuefirst (with fallback to deprecatedsource.data) for base64/data/unknown source types. - Update
ag-ui-protocoldependency (and lockfile) and add regression tests for both fixes.
Reviewed changes
Copilot reviewed 9 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
python/packages/ag-ui/agent_framework_ag_ui/_run_common.py |
Sets reasoning message start role to "reasoning" in both flow and non-flow paths. |
python/packages/ag-ui/agent_framework_ag_ui/_message_adapters.py |
Prefers source.value over source.data when parsing multimodal media payloads. |
python/packages/ag-ui/pyproject.toml |
Relaxes ag-ui-protocol pin to >=0.1.16,<0.2 to pick up upstream fix. |
python/uv.lock |
Locks ag-ui-protocol to 0.1.17 and updates metadata specifier accordingly. |
python/packages/ag-ui/tests/ag_ui/test_run.py |
Updates existing reasoning role assertion and adds targeted flow/non-flow role regression tests. |
python/packages/ag-ui/tests/ag_ui/test_message_adapters.py |
Adds regression tests ensuring value is used (and preferred) for multimodal media parsing. |
python/samples/04-hosting/foundry-hosted-agents/responses/02_local_tools/main.py |
Import ordering adjustment (moves Annotated import into the stdlib section). |
python/samples/02-agents/conversations/file_history_provider.py |
Adds inline Pyright ignore for optional orjson import. |
python/samples/02-agents/conversations/file_history_provider_conversation_persistence.py |
Adds inline Pyright ignore for optional orjson import. |
.maf-runtime-ready |
Adds a new repo-root marker file. |
moonbox3
left a comment
moonbox3
left a comment
There was a problem hiding this comment.
Automated Code Review
Reviewers: 4 | Confidence: 94% | Result: All clear
Reviewed: Correctness, Security Reliability, Test Coverage, Design Approach
Automated review by moonbox3's agents
Address PR review feedback: the .maf-runtime-ready file is not referenced anywhere in the repo and was left over from automation. Fixes microsoft#5340
moonbox3
force-pushed
the
agent/fix-5340-2
branch
from
408b9d0 to
372f31f
Compare
|
To: @moonbox3 I do want to note that this does not fix the issue completely and with this fix, another part of the bug has popped up. Thank you in advance. |
Thanks for the callout here, @Rickyneer. You're right that we can improve this. Making the fix now. |
Resolve conflicts in: - python/packages/ag-ui/pyproject.toml: keep ag-ui-protocol>=0.1.16,<0.2 (the fix this PR ships) alongside upstream's agent-framework-core>=1.1.1 bump. - python/packages/ag-ui/tests/ag_ui/test_run.py: keep both TestReasoningEventRole and the new session_id tests from upstream.
The snapshot normalization path used a second copy of the multimodal source
parsing logic that still read the deprecated 'data' field. When clients sent
base64 media with source={"type": "base64", "value": ...}, the snapshot event
emitted by the server dropped the payload, causing AG-UI-compatible clients
to crash on ingest.
Extract the shared source-field extraction into _extract_multimodal_source_fields
so both _parse_multimodal_media_part and the snapshot _legacy_binary_part stay
in sync with the AG-UI spec. Add snapshot-path regression tests covering
value-only, value-preferred-over-data, and the legacy data-field fallback.
Addresses review feedback on microsoft#5389 from @Rickyneer.
Previous commit incorrectly renamed the [1.1.1] header to [1.2.0], which wiped the historical 1.1.1 entries and wrongly attributed them to 1.2.0. This restores [1.1.1] to its origin/main content and adds a new [1.2.0] section above containing only the commits in python-1.1.1..HEAD: - microsoft#4238 functional workflow API - microsoft#5142 GitHub Copilot OpenTelemetry - microsoft#2403 A2A bridge support - microsoft#5070 oauth_consent_request events in Foundry clients - microsoft#5447 FoundryAgent hosted agent sessions - microsoft#5459 hosting server dependency upgrade + types - microsoft#5389 AG-UI reasoning/multimodal parsing fix - microsoft#5440 stop [TOOLBOXES] warning spam - microsoft#5455 user agent prefix fix Also corrects the [1.2.0] compare base to python-1.1.1 (not 1.1.0) and adds the missing [1.1.1] reference link.
* Bump Python package versions for 1.2.0 release Released tier bumps 1.1.1 -> 1.2.0 (core, openai, foundry, root) to reflect additive public APIs landed since 1.1.0: functional workflow API (#4238) and FunctionTool SKIP_PARSING sentinel (#5424). All beta packages stamped 1.0.0b260424, alpha packages 1.0.0a260424. All 26 non-core agent-framework-core floors raised to >=1.2.0,<2. CHANGELOG consolidates the never-tagged 1.1.1 entries with the post-merge additions into [1.2.0]. * Update CHANGELOG footer links for 1.2.0 Advance [Unreleased] comparison base from python-1.1.0 to python-1.2.0 and add a [1.2.0] reference link comparing python-1.1.0...python-1.2.0 so the heading links resolve correctly. * Fix CHANGELOG: restore [1.1.1] section and add proper [1.2.0] Previous commit incorrectly renamed the [1.1.1] header to [1.2.0], which wiped the historical 1.1.1 entries and wrongly attributed them to 1.2.0. This restores [1.1.1] to its origin/main content and adds a new [1.2.0] section above containing only the commits in python-1.1.1..HEAD: - #4238 functional workflow API - #5142 GitHub Copilot OpenTelemetry - #2403 A2A bridge support - #5070 oauth_consent_request events in Foundry clients - #5447 FoundryAgent hosted agent sessions - #5459 hosting server dependency upgrade + types - #5389 AG-UI reasoning/multimodal parsing fix - #5440 stop [TOOLBOXES] warning spam - #5455 user agent prefix fix Also corrects the [1.2.0] compare base to python-1.1.1 (not 1.1.0) and adds the missing [1.1.1] reference link.
Motivation and Context
The AG-UI adapter emitted
role="assistant"onReasoningMessageStartEventinstead of the spec-requiredrole="reasoning", causing reasoning messages to render incorrectly in AG-UI clients. Additionally,_parse_multimodal_media_partread the deprecated"data"field instead of the current spec's"value"field, silently dropping all user-submitted multimodal media.Fixes #5340
Description
Changed
ReasoningMessageStartEventrole from"assistant"to"reasoning"in both the flow and non-flow code paths of_emit_text_reasoningin_run_common.py. Updated_parse_multimodal_media_partin_message_adapters.pyto readsource_dict.get("value")first (with fallback to"data"for backward compatibility) for base64/data source types and the unknown-type fallback branch. Bumped theag-ui-protocoldependency from pined0.1.13to>=0.1.16,<0.2to pick up the upstream SDK fix for the reasoning role type. Added targeted regression tests for both the reasoning role and multimodal value-field parsing.Contribution Checklist