ROCm

[ehartford]

I tried to install llm-compressor on AMD mi300x
it installed a bunch of nvidia packages like nvidia-ml-py, nvidia-cusparselt-cu12, nvidia-nvtx-cu12

[brian-dellabetta]

Hi @ehartford , llm-compressor doesn't list any specific CUDA dependencies (requirements here). you can do a pip show nvidia-ml-py to see what dependency is requiring those, my guess is torch defaults to CUDA? See vllm docs here for recommended install on ROCm:

# Install PyTorch
pip uninstall torch -y
pip install --no-cache-dir --pre torch --index-url https://download.pytorch.org/whl/nightly/rocm6.3