Docker MIG Manager

Unofficial minimal docker instructions for managing NVIDIA Multi-Instance GPU (MIG) in containers.

Prerequisites:

• NVIDIA Driver
• Docker
• NVIDIA Container Toolkit
• Enable MIG Mode

Take A100 as an example, run:

docker run --rm -it --gpus all \
    --cap-add=SYS_ADMIN \
    -e NVIDIA_MIG_CONFIG_DEVICES=all \
    ubuntu

# in the container
# Create two `3g.20gb` GPU instances (GI) and corresponding compute instances (CI)
nvidia-smi mig -cgi 9,3g.20gb -C
# List the available CIs and GIs
nvidia-smi mig -lgi; nvidia-smi mig -lci;
# Destroy all the CIs and GIs
nvidia-smi mig -dci; nvidia-smi mig -dgi;

This should also work on A30/H100/H200 by substituting the MIG profile to a supported one.

Note: --runtime=nvidia, -e NVIDIA_VISIBLE_DEVICES=all, and -e NVIDIA_DRIVER_CAPABILITIES=all may be required depending on your environment and use cases.

References

Some references I found useful during the investigation.

• MIG User Guide
  • 9.4. Creating GPU Instances
  • 9.6. Destroying GPU Instances
  • 10. Device Nodes and Capabilities
• The NVIDIA Container Runtime
  • NVIDIA_MIG_CONFIG_DEVICES
• NVIDIA MIG Manager For Kubernetes

Acknowledgement

Thanks Hsu-Tzu Ting for discussions.