Skip to content

feat(agent-runtime): add typed agent lifecycle - #350

Merged
nvddr merged 4 commits into
mainfrom
agent/native-tool-dispatcher
Aug 12, 2026
Merged

feat(agent-runtime): add typed agent lifecycle#350
nvddr merged 4 commits into
mainfrom
agent/native-tool-dispatcher

Conversation

@nvddr

@nvddr nvddr commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add the dependency-light xr-ai-agent-runtime package with one addressable Agent lifecycle
  • expose call(ref, message) for bounded request/response and publish(topic, event) for typed fan-out
  • let resident agents start runtime-owned background tasks while bounded mailbox handlers serialize private state
  • migrate simple-vlm-example finite vision requests to an addressable SimpleVlmAgent

Architecture

Raw media stays on XR-Media-Hub. The runtime carries typed semantic messages with participant, correlation, source, and parent metadata. The dispatcher stays private; model loops, tools, planning, and memory remain agent implementation details. Relay remains the execution and telemetry boundary around model and tool calls inside agents. A future Fabric adapter can host or bridge this lifecycle without changing application-level call and publish topology.

The simple VLM worker now registers LiveVisionTool behind the runtime. Its bounded agent returns one complete VisionResponse; streaming voice behavior is intentionally layered in #351.

Review feedback addressed

  • every call and subscriber receives a deep-isolated Pydantic payload
  • a call timeout covers mailbox backpressure and handler execution as one operation
  • participant scope is runtime metadata and is preserved across nested calls and publications
  • runtime ownership replaces the former manual periodic-source lifecycle

Validation

  • runtime and simple-VLM focused suite: 18 passed
  • targeted Pyright: 0 errors
  • Ruff, SPDX, DCO, worker wheel build, and worker lock resolution passed

Rebased onto main after #349 merged. #351 is stacked on this PR.

@nvddr
nvddr deployed to github-pages August 11, 2026 21:36 — with GitHub Actions Active
github-actions Bot added a commit that referenced this pull request Aug 11, 2026
@nvddr
nvddr force-pushed the agent/relay-simple-vlm-native-vision branch from 0abbb52 to ad1d15c Compare August 11, 2026 22:09
@nvddr
nvddr force-pushed the agent/native-tool-dispatcher branch from 0fdb44f to 40c11d5 Compare August 11, 2026 22:13
@nvddr
nvddr deployed to github-pages August 11, 2026 22:14 — with GitHub Actions Active
github-actions Bot added a commit that referenced this pull request Aug 11, 2026
@nvddr
nvddr force-pushed the agent/relay-simple-vlm-native-vision branch from ad1d15c to 05c2619 Compare August 11, 2026 22:16
@nvddr
nvddr force-pushed the agent/native-tool-dispatcher branch from 40c11d5 to 64ead4c Compare August 11, 2026 22:17
@nvddr
nvddr deployed to github-pages August 11, 2026 22:18 — with GitHub Actions Active
github-actions Bot added a commit that referenced this pull request Aug 11, 2026
@nvddr
nvddr force-pushed the agent/relay-simple-vlm-native-vision branch from 05c2619 to 3dc056e Compare August 11, 2026 22:43
@nvddr
nvddr force-pushed the agent/native-tool-dispatcher branch from 64ead4c to 2da244a Compare August 12, 2026 00:40
@nvddr
nvddr deployed to github-pages August 12, 2026 00:41 — with GitHub Actions Active
github-actions Bot added a commit that referenced this pull request Aug 12, 2026
@nvddr
nvddr force-pushed the agent/native-tool-dispatcher branch from 2da244a to 89aec18 Compare August 12, 2026 00:57
@nvddr
nvddr deployed to github-pages August 12, 2026 00:58 — with GitHub Actions Active
github-actions Bot added a commit that referenced this pull request Aug 12, 2026
@nvddr
nvddr force-pushed the agent/native-tool-dispatcher branch from 89aec18 to 51131e5 Compare August 12, 2026 01:05
@nvddr
nvddr deployed to github-pages August 12, 2026 01:05 — with GitHub Actions Active
github-actions Bot added a commit that referenced this pull request Aug 12, 2026
@nvddr
nvddr force-pushed the agent/native-tool-dispatcher branch from 51131e5 to 2107dd8 Compare August 12, 2026 01:33
@nvddr nvddr changed the title feat(native-tools): dispatch voice and background triggers feat(native-tools): add typed event mailboxes Aug 12, 2026
@nvddr
nvddr deployed to github-pages August 12, 2026 01:33 — with GitHub Actions Active
github-actions Bot added a commit that referenced this pull request Aug 12, 2026
Base automatically changed from agent/relay-simple-vlm-native-vision to main August 12, 2026 01:40
@nvddr
nvddr force-pushed the agent/native-tool-dispatcher branch from 2107dd8 to a11fcb4 Compare August 12, 2026 01:41
@nvddr
nvddr deployed to github-pages August 12, 2026 01:42 — with GitHub Actions Active
github-actions Bot added a commit that referenced this pull request Aug 12, 2026
Comment thread agent-sdk/xr-ai-nat/xr_ai_nat/events/dispatcher.py Fixed
Comment thread tests/test_native_events.py Fixed
Comment thread tests/test_native_events.py Fixed
Comment thread tests/test_native_events.py Fixed
Comment thread agent-sdk/xr-ai-nat/xr_ai_nat/events/periodic.py Fixed
github-actions Bot added a commit that referenced this pull request Aug 12, 2026
Comment thread tests/test_agent_runtime.py Fixed
Comment thread agent-sdk/xr-ai-agent-runtime/xr_ai_runtime/runtime.py Fixed
Comment thread agent-sdk/xr-ai-agent-runtime/xr_ai_runtime/runtime.py Fixed
@nvddr
nvddr force-pushed the agent/native-tool-dispatcher branch from 2b6808d to cea8f26 Compare August 12, 2026 20:39
@nvddr
nvddr deployed to github-pages August 12, 2026 20:40 — with GitHub Actions Active
github-actions Bot added a commit that referenced this pull request Aug 12, 2026
@nvddr
nvddr force-pushed the agent/native-tool-dispatcher branch from cea8f26 to 1264a59 Compare August 12, 2026 20:45
@nvddr
nvddr deployed to github-pages August 12, 2026 20:46 — with GitHub Actions Active
github-actions Bot added a commit that referenced this pull request Aug 12, 2026
@nvddr
nvddr deployed to github-pages August 12, 2026 21:22 — with GitHub Actions Active
Comment thread tests/test_agent_runtime.py Fixed
Comment thread tests/test_agent_runtime.py Fixed
github-actions Bot added a commit that referenced this pull request Aug 12, 2026
@wenxind-nvidia

Copy link
Copy Markdown
Collaborator

Blocker

  1. Runtime-owned failures are silently downgraded to event-loop log messages. AgentContext._background_done() lets a failed background task disappear while runtime.running remains true, and AgentRuntime._deliver() suppresses subscriber exceptions so publish() returns successfully. I reproduced both behaviors: essential work can stop while the worker still appears healthy. Please let subscriber failures propagate after all fan-out deliveries have settled, and make a runtime-owned background failure fail or explicitly transition the runtime instead of only invoking the loop exception handler.

@nvddr
nvddr deployed to github-pages August 12, 2026 21:32 — with GitHub Actions Active
github-actions Bot added a commit that referenced this pull request Aug 12, 2026
@nvddr

nvddr commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

@wenxind-nvidia Addressed in 182a4fc. Subscriber exceptions now propagate from publish() only after every fan-out delivery has settled. A failed runtime-owned background task now transitions the runtime to failed, sets running false, cancels remaining owned work, causes later operations to raise RuntimeFailedError from the original exception, and surfaces that original failure again during shutdown. Added regression coverage for both cases. The 28 focused runtime/tool tests pass on Python 3.11 and 3.12; Ruff and Pyright also pass.

@nvddr
nvddr deployed to github-pages August 12, 2026 21:39 — with GitHub Actions Active
github-actions Bot added a commit that referenced this pull request Aug 12, 2026
@nvddr
nvddr force-pushed the agent/native-tool-dispatcher branch from 182a4fc to 337b2d0 Compare August 12, 2026 21:42
@nvddr
nvddr deployed to github-pages August 12, 2026 21:42 — with GitHub Actions Active
github-actions Bot added a commit that referenced this pull request Aug 12, 2026
Comment thread tests/test_agent_runtime.py Fixed
nvddr added 4 commits August 12, 2026 15:42
Signed-off-by: Devdeep Ray <devdeepr@nvidia.com>
Signed-off-by: Devdeep Ray <devdeepr@nvidia.com>
Signed-off-by: Devdeep Ray <devdeepr@nvidia.com>
Signed-off-by: Devdeep Ray <devdeepr@nvidia.com>
@nvddr
nvddr force-pushed the agent/native-tool-dispatcher branch from d299ef8 to 7dbd9b4 Compare August 12, 2026 22:43
@nvddr
nvddr deployed to github-pages August 12, 2026 22:44 — with GitHub Actions Active
github-actions Bot added a commit that referenced this pull request Aug 12, 2026
@nvddr
nvddr merged commit 79aa65d into main Aug 12, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants