One GPU leveraged while 8 exposed

[aimanakheel]

Has this issue been opened before?

• It is not in the FAQ, I checked.
• It is not in the issues, I searched.

Describe the bug

Docker implementation is really great!!

Is there a way to leverage all GPUs? I have 8 Nvidia GPUs exposed. I can see all the GPUs when i run the "nvidia-smi" in the docker container.

My docker compose looks like

x-base_service: &base_service
    ports:
      - "${WEBUI_PORT:-7860}:7860"
    volumes:
      - &v1 ./data:/data
      - &v2 ./output:/output
    stop_signal: SIGKILL
    tty: true
    deploy:
      resources:
        reservations:
          devices:
              - driver: nvidia
                count: all
                capabilities: [gpu]
    runtime: nvidia
    environment:
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility
      - NVIDIA_VISIBLE_DEVICES=all

name: webui-docker

services:
  download:
    build: ./services/download/
    profiles: ["download"]
    volumes:
      - *v1

  auto: &automatic
    <<: *base_service
    profiles: ["auto"]
    build: ./services/AUTOMATIC1111
    image: sd-auto:78
    environment:
      - CLI_ARGS=--allow-code --medvram --xformers --enable-insecure-extension-access --api
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility
      - NVIDIA_VISIBLE_DEVICES=all

  auto-cpu:
    <<: *automatic
    profiles: ["auto-cpu"]
    deploy: {}
    environment:
      - CLI_ARGS=--no-half --precision full --allow-code --enable-insecure-extension-access --api

  comfy: &comfy
    <<: *base_service
    profiles: ["comfy"]
    build: ./services/comfy/
    image: sd-comfy:7
    environment:
      - CLI_ARGS=
      - NVIDIA_DRIVER_CAPABILITIES=compute,utility
      - NVIDIA_VISIBLE_DEVICES=all

  comfy-cpu:
    <<: *comfy
    profiles: ["comfy-cpu"]
    deploy: {}
    environment:
      - CLI_ARGS=--cpu

Which UI

comfy

Hardware / Software

• OS : Ubuntu 22.04
• Docker Version:

Client: Docker Engine - Community
 Version:           27.3.1
 API version:       1.47
 Go version:        go1.22.7
 Git commit:        ce12230
 Built:             Fri Sep 20 11:40:59 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          27.3.1
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.22.7
  Git commit:       41ca978
  Built:            Fri Sep 20 11:40:59 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.22
  GitCommit:        7f7fdf5fed64eb6a7caf99b3e12efcf9d60e311c
 runc:
  Version:          1.1.14
  GitCommit:        v1.1.14-0-g2c9f560
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

• Docker compose version: Docker Compose version v2.29.7

• Repo version: 802d0bc

• RAM: 512GB

• GPU/VRAM: 8 Nvidia A16 GPUs & 128GB Vram

[DevilaN]

If all GPUs are available and visible inside container, then it is probably ComfyUI thing to make proper use of them.
Make sure using multiple GPUs is supported by Comfy in the first place.

[nwatab]

@aimanakheel found a solution?

[aimanakheel]

i did not. Perhaps this may not be the correct forum for this ticket. May be need to open a ticket with comfyUI team.

[nwatab]

I'm still not sure if it is a comfyUI problem. I tried with auto . nvidia-smi shows all GPUs and I rewrote docker compose to be count: all. However, only one GPU is used.

[LoopControl]

Comfy only uses 1 GPU for most nodes. If you want to use multiple GPUs you'd have to run multiple Comfy instances.

There are also some custom_nodes for Comfy that support offloading to a 2nd GPU but that's node specific.