Can't install. Error

[AfterHAL]

Hi. I've been trying to install torch-cubas-hgemm (for flux-fp8-api) on ubuntu without success.

The same error ModuleNotFoundError: No module named 'torch', whether with
pip install git+https://github.com/aredden/torch-cublas-hgemm.git@master
or with
python -m pip install -U -v . (after cloning the repository).

--> Of course, torch is installed.

Regards.

python -m pip install -U -v .


Using pip 24.2 from /mnt/w/TDocker/gradio1/flux-fp8-api/venv/lib/python3.12/site-packages/pip (python 3.12)
Processing /mnt/w/TDocker/gradio1/flux-fp8-api/torch-cublas-hgemm
  Running command pip subprocess to install build dependencies
  Using pip 24.2 from /mnt/w/TDocker/gradio1/flux-fp8-api/venv/lib/python3.12/site-packages/pip (python 3.12)
  Collecting setuptools>=40.8.0
    Obtaining dependency information for setuptools>=40.8.0 from https://files.pythonhosted.org/packages/ff/ae/f19306b5a221f6a436d8f2238d5b80925004093fa3edea59835b514d9057/setuptools-75.1.0-py3-none-any.whl.metadata
    Using cached setuptools-75.1.0-py3-none-any.whl.metadata (6.9 kB)
  Using cached setuptools-75.1.0-py3-none-any.whl (1.2 MB)
  Installing collected packages: setuptools
  Successfully installed setuptools-75.1.0
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Traceback (most recent call last):
    File "/mnt/w/TDocker/gradio1/flux-fp8-api/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/mnt/w/TDocker/gradio1/flux-fp8-api/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/mnt/w/TDocker/gradio1/flux-fp8-api/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
             ^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-a9kwdlcq/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 332, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=[])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/tmp/pip-build-env-a9kwdlcq/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 302, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-a9kwdlcq/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 503, in run_setup
      super().run_setup(setup_script=setup_script)
    File "/tmp/pip-build-env-a9kwdlcq/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 318, in run_setup
      exec(code, locals())
    File "<string>", line 3, in <module>
  ModuleNotFoundError: No module named 'torch'
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /mnt/w/TDocker/gradio1/flux-fp8-api/venv/bin/python /mnt/w/TDocker/gradio1/flux-fp8-api/venv/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpnp9reoxk
  cwd: /mnt/w/TDocker/gradio1/flux-fp8-api/torch-cublas-hgemm
  Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[aredden]

That is very strange- the only thing I can think of would be that python isn't pointing to the right python. You're sure that when you do python -m pip list | grep torch - torch shows up?

[AfterHAL]

That is very strange- the only thing I can think of would be that python isn't pointing to the right python. You're sure that when you do python -m pip list | grep torch - torch shows up?

Hi @aredden. Thanks for helping out.
Nothing special for me about the torch installation :

python -m pip list | grep torch
pytorch-triton           3.1.0+5fe38ffd73
torch                    2.6.0.dev20240917+cu124
torchaudio               2.5.0.dev20240917+cu124
torchvision              0.20.0.dev20240917+cu124

I'm trying to run flux-fp8-api from a fresh virtual environment. I've been trying updating and checking everything I could thing of without success.

By the way, I'm looking for way to use a fast loading Flux-dev-fp8 model like the "easy to use ComfyUI fp8 checkpoint for the Flux dev (safetensors file here)" of ComfyUI (with the --fast argument). It loads in 30 to 40 seconds on my 64GBram/4090-24GBvram PC, and generates a 1280x1280px image in less then 20 secs, even with 4 or 5 LoRAs. This setup uses around 20 GBvram.
I've been looking inside the the code of ComfyUI and the official Flux.1 repository, but it is a little too hard for me.

You're code looks good and I would like to know how to do it, but I'm stuck ;)

Let me know if you have any idea.
Thanks again.

[AfterHAL]

Hi @aredden.
I finally started from scratch with a fresh installation of Ubuntu and it's ok now.
By the way, I don't know the origin of the error I had before.

[aredden]

Ah okay cool! Yeah I'm unsure either. I'm glad you got it sorted :)

[MichaelP84]

I have the exact same issue, so let me know if you get any clarity on it, thanks!

[mikhail-shevtsov-wiregate]

Just in case if someone faces this issue:

python -m pip install --no-build-isolation -U -v .

Adding --no-build-isolation solved the problem.

[spejamas]

I think this happens because the package depends on torch/cuda during its setup.py execution, not only at runtime.
--no-build-isolation will work for some (by allowing the installation to look globally for torch/cuda I guess?). Some environments though—like when trying to deploy with some cloud providers—won't have torch/cuda available during the build phase, so it won't work for everyone.

[fyepi]

I have encountered the same problem when I was trying flux-fp8-api.

I used python -m venv venv to create a virtual environment, activated it and then followed the installation steps to install torch then install requirements.txt then this problem happened.

I used ubuntu 22.04 and python 3.12. To reinstall ubuntu is way too costly for me.

@spejamas
Some environments though—like when trying to deploy with some cloud providers—won't have torch/cuda available during the build phase seems to be the root cause. Any way to get torch ready in build phase?

Thanks.

[SlizenDize]

Anyone got a solution to this issue? --no-build-isolation didn't work for me and I have been searching for hours now trying to understand how pips and environments work. In the end I realized the more I read the more I understand that I don't understand anything. I feel like a stranded whale.

[JonSingleton]

If you're running ComfyUI portable like me (and installing to use some node that can use it) then my issue was that following the directions as provided in the readme don't point to your python embedded folder. So for example what I had to run was:

cd C:\ComfyUI_windows_portable_nvidia\ComfyUI_windows_portable\python_embeded

git clone https://github.com/aredden/torch-cublas-hgemm.git

python.exe -m pip install -U -v torch-cublas-hgemm\.

notice I didn't cd into the cublas folder. I guess you could and then point to the correct python.exe using the full path too though. Either way the ultimate issue was it was trying to use my system python and not the embedded.

Side note, anyone else notice the team at comfyui misspelled embedded as "embeded"? that's going to bother the shit out of me.

[ZedOud]

What worked for me, Windows 11, having installed Forge Classic via Stability Matrix, and I was previously following Forge Classic's install guide, (which didn't work for me).

1. activate your virtual environment (make sure you then cd into the build directory):
   venv\Scripts\activate
2. bypass the distutils hack:
   $env:SETUPTOOLS_USE_DISTUTILS = "stdlib"
3. install it:
   python -m pip install --no-build-isolation -U -v .