Skip to content

metal : support virtual devices#18919

Merged
ggerganov merged 4 commits intomasterfrom
gg/metal-virtual-devices
Feb 2, 2026
Merged

metal : support virtual devices#18919
ggerganov merged 4 commits intomasterfrom
gg/metal-virtual-devices

Conversation

@ggerganov
Copy link
Member

@ggerganov ggerganov commented Jan 18, 2026 •
edited
Loading

Support virtual Metal devices. Allows simulating multi-GPU environments on Mac using the new GGML_METAL_DEVICES environment variable.

GGML_METAL_DEVICES=4 ./bin/llama-completion -m [model.gguf]

...

0.02.020.033 I llama_memory_breakdown_print: | memory breakdown [MiB]    |  total     free    self   model   context   compute    unaccounted |
0.02.020.034 I llama_memory_breakdown_print: |   - MTL0 (Apple M2 Ultra) | 165150 = 158091 + (1916 =   780 +    1024 +     112) +        5143 |
0.02.020.034 I llama_memory_breakdown_print: |   - MTL1 (Apple M2 Ultra) | 165150 = 158091 + (1738 =   780 +     896 +      62) +        5320 |
0.02.020.036 I llama_memory_breakdown_print: |   - MTL2 (Apple M2 Ultra) | 165150 = 158091 + (1198 =   240 +     896 +      62) +        5861 |
0.02.020.037 I llama_memory_breakdown_print: |   - MTL3 (Apple M2 Ultra) | 165150 = 158091 + (2205 =  1137 +     768 +     300) +        4853 |
0.02.020.037 I llama_memory_breakdown_print: |   - Host                  |                     364 =   296 +       0 +      68                |

@github-actions github-actions bot added ggml changes relating to the ggml tensor library for machine learning Apple Metal https://en.wikipedia.org/wiki/Metal_(API) labels Jan 18, 2026
@loci-dev loci-dev mentioned this pull request Jan 18, 2026
Open
@ggerganov ggerganov force-pushed the gg/metal-virtual-devices branch from e644345 to 06ac1f7 Compare January 20, 2026 15:40
This was referenced Jan 20, 2026
Merged
Merged
@ggerganov ggerganov force-pushed the gg/metal-virtual-devices branch from 06ac1f7 to 4d19e61 Compare January 31, 2026 08:05
@loci-dev loci-dev mentioned this pull request Jan 31, 2026
Open
@ggerganov ggerganov merged commit 6fdddb4 into master Feb 2, 2026
73 of 78 checks passed
@ggerganov ggerganov deleted the gg/metal-virtual-devices branch February 2, 2026 12:29
@ggerganov ggerganov mentioned this pull request Feb 3, 2026
Merged
1 task
shaofeiqi pushed a commit to qualcomm/llama.cpp that referenced this pull request Feb 6, 2026
@ggerganov @shaofeiqi
metal : support virtual devices (ggml-org#18919)
f6cc6fb 
* metal : support virtual devices

* cont : manage buffer type context memory

* metal : add events

* cont : implement cpy_tensor_async
liparetejas pushed a commit to liparetejas/llama.cpp that referenced this pull request Feb 23, 2026
@ggerganov @liparetejas
metal : support virtual devices (ggml-org#18919)
74b3495 
* metal : support virtual devices

* cont : manage buffer type context memory

* metal : add events

* cont : implement cpy_tensor_async
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Apple Metal https://en.wikipedia.org/wiki/Metal_(API) ggml changes relating to the ggml tensor library for machine learning

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@ggerganov