Feat: Automatic GPU Switch #845
Conversation
gpu switch version: 0.1 # Without GPU: NVIDIA_VISIBLE_DEVICES="" ./gpu-switch.sh start container_name # With single GPU: NVIDIA_VISIBLE_DEVICES="0" ./gpu-switch.sh start container_name # With multiple GPUs: NVIDIA_VISIBLE_DEVICES="0,1" ./gpu-switch.sh start container_name # With PCI addresses: NVIDIA_VISIBLE_DEVICES="0000:03:00.0,0000:04:00.0" ./gpu-switch.sh start container_name # Explicitly disable GPU: NVIDIA_VISIBLE_DEVICES="none" ./gpu-switch.sh start container_name
|
have to modify the docker compose side as I was under the impression it supported pre-start and post-stop scripts but I misread and its post-start and pre-stop, ill need to find a new way to work this, script still works and can be implemented using user scripts in unraid. [again tho im waiting on a gpu so i haven't been able to fully test it] |
|
Very interesting work!! Did you already receive your GPU to test it? |
|
Very good |
|
Hmm! Interesting |
| if ! check_gpu_needed; then | ||
| log "Continuing without GPU management" | ||
| exit 0 | ||
| fi |
There was a problem hiding this comment.
Instead of listing all containers, you can directly check the existence of the container using docker container inspect, which is more efficient since it only checks the specified container without scanning the entire list. Here’s how to replace that line:
if ! docker container inspect "$CONTAINER_NAME" > /dev/null 2>&1; then
error_exit "Container $CONTAINER_NAME does not exist"
fi
The docker container inspect command returns an error if the container does not exist, so you can use it to directly verify the container’s existence without listing all containers.
There was a problem hiding this comment.
ill take a look at implementing this thanks for the ideas
| } | ||
|
|
||
| # Convert any GPU identifier to PCI address | ||
| convert_to_pci_address() { |
There was a problem hiding this comment.
Incorporating these improvements, here’s the final optimized convert_to_pci_address fu
convert_to_pci_address() {
local device="$1"
local gpu_address=""
if [[ "$device" =~ ^[0-9]+$ || "$device" =~ ^GPU-.*$ ]]; then
# Convert GPU index or UUID to PCI address
gpu_address=$(nvidia-smi --id="$device" --query-gpu=gpu_bus_id --format=csv,noheader 2>/dev/null | tr -d '[:space:]')
else
# Direct PCI address provided
gpu_address="$device"
fi
# Check for valid output
if [ -z "$gpu_address" ]; then
error_exit "Failed to get PCI address for device: $device"
fi
# Standardize format
echo "$gpu_address" | sed -e 's/0000://' -e 's/\./:/g'
}
There was a problem hiding this comment.
ill take a look at implementing this thanks for the ideas on this as well
|
share failed? is there something wrong? |
sadly no the one I ordered from ebay was extremely unstable (kept crashing my server when using it with ollama) so im waiting for my money back |
you will have to expand on this, i dont understand. |
Update gpu-switch.sh
|
Looking forward to your completion, thank you, I hope there will be a complete tutorial in the future |
| local gpu_address=$(convert_to_pci_address "$device") | ||
| if [ -z "$gpu_address" ]; then | ||
| error_exit "Failed to get PCI address for device: $device" | ||
| } |
There was a problem hiding this comment.
./gpu-switch.sh: line 64: syntax error near unexpected token `}'
./gpu-switch.sh: line 64: ` }'
| if [ -z "$gpu_audio_address" ]; then | ||
| log "Warning: No audio device found for GPU $gpu_address" | ||
| continue | ||
| } |
There was a problem hiding this comment.
./gpu-switch.sh: line 75: syntax error near unexpected token `}'
./gpu-switch.sh: line 75: ` }'
|
I got the following error after running Here's the output of |
Docker Windows GPU Passthrough
[this is not fully tested as im waiting for a gpu to come in]
Automated GPU management solution for Windows in Docker containers with NVIDIA GPU passthrough support. This project provides scripts and configurations to dynamically manage GPU binding between host and Docker containers, with support for multiple GPUs and audio devices.
Prerequisites
Quick Start
git clone https://github.com/yourusername/docker-windows-gpu.git cd docker-windows-gpu# Set to your GPU ID(s), PCI address(es), or 'none' add NVIDIA_VISIBLE_DEVICES=0Configuration
Environment Variables
NVIDIA_VISIBLE_DEVICES: Specify GPU(s) to useNVIDIA_VISIBLE_DEVICES=0NVIDIA_VISIBLE_DEVICES=0,1NVIDIA_VISIBLE_DEVICES=0000:03:00.0,0000:04:00.0NVIDIA_VISIBLE_DEVICES=noneDocker Compose
The provided
docker-compose.ymlincludes all necessary configurations for:Usage
Manual GPU Management (until I find a way to run pre and post stop, use it with user scripts)
Bind GPU to container:
Release GPU:
Script Details
The
gpu-switch.shscript handles: