Skip to content

UPSTREAM PR #18919: metal : support virtual devices#960

Open
loci-dev wants to merge 2 commits intomainfrom
upstream-PR18919-branch_ggml-org-gg/metal-virtual-devices
Open

UPSTREAM PR #18919: metal : support virtual devices#960
loci-dev wants to merge 2 commits intomainfrom
upstream-PR18919-branch_ggml-org-gg/metal-virtual-devices

Conversation

@loci-dev
Copy link

Mirrored from ggml-org/llama.cpp#18919

Support virtual Metal devices. Allows simulating multi-GPU environments on Mac using the new GGML_METAL_DEVICES environment variable.

GGML_METAL_DEVICES=4 ./bin/llama-completion -m [model.gguf]

...

0.02.020.033 I llama_memory_breakdown_print: | memory breakdown [MiB]    |  total     free    self   model   context   compute    unaccounted |
0.02.020.034 I llama_memory_breakdown_print: |   - MTL0 (Apple M2 Ultra) | 165150 = 158091 + (1916 =   780 +    1024 +     112) +        5143 |
0.02.020.034 I llama_memory_breakdown_print: |   - MTL1 (Apple M2 Ultra) | 165150 = 158091 + (1738 =   780 +     896 +      62) +        5320 |
0.02.020.036 I llama_memory_breakdown_print: |   - MTL2 (Apple M2 Ultra) | 165150 = 158091 + (1198 =   240 +     896 +      62) +        5861 |
0.02.020.037 I llama_memory_breakdown_print: |   - MTL3 (Apple M2 Ultra) | 165150 = 158091 + (2205 =  1137 +     768 +     300) +        4853 |
0.02.020.037 I llama_memory_breakdown_print: |   - Host                  |                     364 =   296 +       0 +      68                |

@loci-review
Copy link

loci-review bot commented Jan 18, 2026

Explore the complete analysis inside the Version Insights

@loci-dev loci-dev force-pushed the main branch 8 times, most recently from ddecb43 to fac93a3 Compare January 20, 2026 12:17
@loci-dev loci-dev force-pushed the upstream-PR18919-branch_ggml-org-gg/metal-virtual-devices branch from e644345 to 06ac1f7 Compare January 20, 2026 15:43
@loci-review
Copy link

loci-review bot commented Jan 20, 2026

Explore the complete analysis inside the Version Insights

Based on the analysis, no functions were identified with measurable performance changes between the base and target versions. This indicates no meaningful performance impact from the code changes.

@loci-dev loci-dev force-pushed the main branch 15 times, most recently from 238591d to f76b51a Compare January 22, 2026 19:12
@loci-dev loci-dev force-pushed the main branch 30 times, most recently from fec6630 to 62c23fc Compare January 28, 2026 21:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants