Skip to content

AMDGPU EP: register umbrella plugin EP and forward profile options - #2

Open
zz002 wants to merge 1 commit into
aditya-dl:amd/dev/adilohia/migraphx_supportfrom
zz002:oga-amdgpu-hip-ep-registration
Open

AMDGPU EP: register umbrella plugin EP and forward profile options#2
zz002 wants to merge 1 commit into
aditya-dl:amd/dev/adilohia/migraphx_supportfrom
zz002:oga-amdgpu-hip-ep-registration

Conversation

@zz002

@zz002 zz002 commented Jul 22, 2026

Copy link
Copy Markdown

Summary

Makes the AMDGPU umbrella EP work end-to-end through OGA's own model_benchmark with profile=hip. Three OGA-side fixes in src/amdgpu; no umbrella EP change required.

Why

Testing with OGA's bundled C model_benchmark (which has no --ep_library/-p/--profile flag; profile is passed via genai_config.json provider options) surfaced three gaps:

  1. Plugin EP not registered. The AMDGPU umbrella is a plugin EP, so its OrtEpDevice is only discoverable after RegisterExecutionProviderLibrary. Legacy in-proc EPs skip this, but the umbrella does not, so AppendExecutionProvider_V2 finds no device and allocation fails ("no requested allocator available"). The RyzenAI interface already self-registers its DLL; the AMDGPU path did not.
  2. Profile not reaching the umbrella. The umbrella reads its provider options (notably profile) from session-config entries prefixed ep.amdgpuexecutionprovider., not the AppendExecutionProvider_V2 ep_options channel, so profile=hip never selected the hipgpu backend.
  3. Trivial init session had no profile. EnsureDeviceOrtInit builds a trivial device-init session to set up the device allocator. After the Route init-session provider-option shaping through DeviceInterface microsoft/onnxruntime-genai#2232 refactor, its provider options come solely from the per-EP ShapeInitSessionProviderOptions hook (default no-op), so the umbrella had no backend to create for that session.

What

  1. EnsureUmbrellaEpRegistered() in src/amdgpu/session_options.cpp — self-registers amdgpu-ep.dll on the OrtEnv, mirroring the RyzenAI pattern (resolve the DLL next to the genai/ort module or the executable; re-registration is benign).
  2. ForwardUmbrellaProviderOptions() in src/amdgpu/session_options.cpp — bridges the genai_config provider options into ep.amdgpuexecutionprovider.* config entries so the umbrella selects the requested backend.
  3. ShapeInitSessionProviderOptions() override in src/amdgpu/interface.cpp — forwards the user's provider options to the trivial init session so it carries the profile too.

Test plan

  • Built model_benchmark.exe + onnxruntime-genai.dll locally with the umbrella amdgpu-ep.dll + hip-backend.dll.
  • profile=hip runs end-to-end on Llama-3.1-8B (int4-awq) via model_benchmark.exe on gfx1151 (StxHalo); prefill + decode complete, output correct.
  • Perf within ~3% of the 2194 baseline (TTFT/TPS); no regression from the added registration/forwarding.

Notes for reviewers

  • Umbrella EP allocator memory-info name is irrelevant to allocator lookup (matched by device attributes: GPU + AMD vendor id + device index + memory type), so no umbrella-side change is needed here.
  • StxHalo default-heuristic enablement in the umbrella EP is a separate follow-up.

@zz002
zz002 force-pushed the oga-amdgpu-hip-ep-registration branch 2 times, most recently from 24ad31f to de1f8c9 Compare July 27, 2026 02:00
The AMDGPU umbrella is a plugin EP, so its OrtEpDevice is only visible after
RegisterExecutionProviderLibrary. The C model_benchmark has no --ep_library flag,
so self-register amdgpu-ep.dll on the OrtEnv the way the RyzenAI interface does,
resolving the DLL next to the genai/ort module or the executable.

The umbrella reads its provider options (e.g. profile) from session config entries
prefixed "ep.amdgpuexecutionprovider.", not the AppendExecutionProvider_V2 ep_options
channel, so bridge the genai_config provider options into that prefixed form. This is
what makes profile=hip actually reach the umbrella and select the hipgpu backend.

Override ShapeInitSessionProviderOptions so the trivial device-init session that
EnsureDeviceOrtInit builds also carries the profile; otherwise the umbrella has no
backend to create for that init session and device/allocator setup fails.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant