You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ggml_backend_cuda_init seems to be called with device 0 as a constant.
It then crashes with out of memory because it only has 8 GB, while another device contains 16.
For the benefit of anyone who finds this issue: you can use CUDA_VISIBLE_DEVICES=x to force a different GPU to be "index 0", I am running successfully on 2 GPUs just have to load balance externally.
@evcharger , The Vulkan equivalent is GGML_VK_VISIBLE_DEVICES; for instance, GGML_VK_VISIBLE_DEVICES=1 will make only the second card available to sd.cpp.
I have mutliple GPUs, that only have enough VRAM together. Is there some way to load different layers on different cards? Or use one GPU but only have a small number of layers in RAM?
Was this ever sorted out ? I have to 12G GPUs, one is not sufficient but if i could spread it across both its plenty ... Regular llama.ccp does this for LLM use.
Activity
Green-Sky commentedon Aug 28, 2024
the-crypt-keeper commentedon Sep 4, 2024
For the benefit of anyone who finds this issue: you can use CUDA_VISIBLE_DEVICES=x to force a different GPU to be "index 0", I am running successfully on 2 GPUs just have to load balance externally.
evcharger commentedon Sep 28, 2025
@the-crypt-keeper is there any way to make this work with multi-gpu on Vulkan?
wbruna commentedon Sep 28, 2025
@evcharger , The Vulkan equivalent is
GGML_VK_VISIBLE_DEVICES; for instance,GGML_VK_VISIBLE_DEVICES=1will make only the second card available to sd.cpp.atrejo3 commentedon Oct 19, 2025
I have mutliple GPUs, that only have enough VRAM together. Is there some way to load different layers on different cards? Or use one GPU but only have a small number of layers in RAM?
Nurb4000 commentedon Mar 17, 2026
Was this ever sorted out ? I have to 12G GPUs, one is not sufficient but if i could spread it across both its plenty ... Regular llama.ccp does this for LLM use.
Qarqor5555555 commentedon Apr 7, 2026
where this file
CUDA_VISIBLE_DEVICES=x
ayttop commentedon Apr 11, 2026
0.txt
ayttop commentedon Apr 11, 2026
xsuuc_stable-diffusion.cpp.txt