cuda : skip UMA override for HIP builds - #27083
Conversation
github-actions
Bot
added
ggml
Geramy
left a comment
Geramy
left a comment
There was a problem hiding this comment.
I don't see anything wrong with this myself the Strix Halo UMA is not one mode it has the ability to configure the memory to system ram or vram, some configurations may only see 32GB System Ram because the bios has 96GB configured to GPU VRAM instead of using GTT, since AMD function hipMemGetInfo supports both configurations while reporting 100% accurate information I think this is a good patch generally.
AMD APUs report accurate memory via hipMemGetInfo. Using MemAvailable over-promises on small-carveout systems. fixes ggml-org#18159
|
@Geramy i dont have any uma machine, but is that true also of windows rocm. I know that for some other cases the memory reporting works differently on windows. |
@IMbackK from what I know, windows has the same memory scheme as linux since its bios based and the architecture of the halo strix, the only difference between linux and windows in this case is on linux you can dedicate more system memory to the GTT table than windows, on windows the limit is 96GB which is a operating system limit. @superm1 correct me if I'm wrong but I believe on windows the outcome is the same as linux right? |
I'll have this checked for you. |
|
@ggml-org/amd please see above ^ |
No; It works differently on Windows. Here is the documentation for it. https://rocm.docs.amd.com/projects/HIP/en/latest/doxygen/html/group___memory.html The code change I submitted is only for Linux. |
correct. |
|
@am17an could you give this PR a quick gander, my PR approvals are only for emotional support haha. |
* origin/master: (86 commits) ui: enforce alphabetical enum member ordering (ggml-org#27272) ui: Refactor Built-In Tools naming (Server/Browser) (ggml-org#27271) ci: more optimizations (ggml-org#26983) doc: document MCP stdio servers and CORS defaults in the server README [no release] [no ci] (ggml-org#26847) server: save processed mtmd chunks as placeholder (ggml-org#27278) mtmd: use sha256 for input hashing (ggml-org#27274) vocab : support integer tokenizer scores (ggml-org#27260) mtmd : skip thumbnail for non-tiled LFM2 images (ggml-org#27246) cuda : skip UMA override for HIP builds (ggml-org#27083) vendor: move hash to vendor (ggml-org#27262) ci : push release tag explicitly in release.yml (ggml-org#27261) ui: move get_datetime tool to frontend (ggml-org#27255) ci : reduce builds in build-xcframework.sh (ggml-org#27252) model: support speculators-format checkpoints for DSpark (ggml-org#26275) ui: add browser get_info tool (ggml-org#27251) ci : restore release.yml check during make-release.yml (ggml-org#27247) mtmd: harden preprocessor_granite (ggml-org#27235) ci : allow make-release to target a specific commit (ggml-org#27234) ci : make release workflows use a deploy key (ggml-org#27229) convert: add @ModelBase.example (ggml-org#27208) ...
AMD APUs report accurate memory via hipMemGetInfo. Using MemAvailable over-promises on small-carveout systems. fixes ggml-org#18159
4 participants
AMD APUs report accurate memory via hipMemGetInfo. Using MemAvailable over-promises on small-carveout systems.
fixes #18159
Overview
AMD iGPU systems report the correct value through
hipMemGetInfo(). Currently the support added for DGX spark in #17368 mispromises memory that isn't there or not enough (depending upon what carveout is set to).Additional information
I tried the max() approach mentioned in #18159 and it overpromises.
Requirements