[Enhancement] GPU selection for multi-GPU setups #19

Activity

1. william-murray1204 commented on Jan 31, 2026

   

   william-murray1204

   on Jan 31, 2026

   Owner

   More actions

   Hi @MrDrMcCoy, there are a few scenarios where multi-GPU setups work and are limited, depending on your hardware. The simplest and currently supported scenario is using multiple GPUs running the same backend, for example two CUDA GPUs. In that case, you can force which GPU is visible via environment variables, as described in this issue. I have an example of how you might go about implementing something like this multi-AMD GPU inference (using a HIP backend) in my Python tests.

   If, on the other hand, you want to run multi-backend GPU hardware simultaneously (e.g. a CUDA and an AMD GPU), there is no easy way to do this built into the library, as each build can only support a single backend. It is not impossible, but it would require running separate Python environments with separate installs of stable-diffusion-cpp-python each targeting a different backend. You could then coordinate inference externally, for example, via a task queue that dispatches jobs to each environment.
2. abhinandan-chakraborty commented on Feb 17, 2026

   

   abhinandan-chakraborty

   on Feb 17, 2026

   More actions

   Hi @william-murray1204 , in my case, I am running a single GRPC server which requires access to both GPU's at the same time. I can sure launch a new process just for stable-diffusion-cpp-python with CUDA_VISIBLE_DEVICES or I can use Ray Serve to distribute my GPU processes, but I don't want to overcomplicate this yet. I wish there was something like llama-cpp-python has tensor_split to split GPU usage. I know it's a lot to ask from you but if it is possible then please consider adding this feature, if not, then also it's no big deal.

   Thank you for this awesome library anyway! Cheers!
3. william-murray1204 commented on Feb 17, 2026

   

   william-murray1204

   on Feb 17, 2026

   Owner

   More actions

   Hi @abhinandan-chakraborty, you don't need to launch a seperate process to use CUDA_VISIBLE_DEVICES as you can set the environment variable in code as shown in the example I linked to above.

   Note that this repo is merely a Python interface (bindings) for the original stable-diffusion.cpp internal C API. The llama-cpp-python like functionality you are describing would first need to be requested/implemented in the upstream stable-diffusion.cpp repo for me to be able to support it here.
4. ayttop commented on Apr 11, 2026

   

   ayttop

   on Apr 11, 2026

   More actions

   https://github.com/william-murray1204/stable-diffusion-cpp-python/blob/main/tests/test_multi_gpu.py
   i ask it run with 2gpu rtx4060ti
   or not??????????