RuntimeError: CUDA error: no kernel image is available for execution on the device #19863

donhuvy · 2025-03-25T13:36:33Z

donhuvy
Mar 25, 2025

Microsoft Windows [Version 10.0.26100.3476]
(c) Microsoft Corporation. All rights reserved.

C:\vy>pip install ultralytics
Collecting ultralytics
  Downloading ultralytics-8.3.96-py3-none-any.whl.metadata (35 kB)
Collecting numpy<=2.1.1,>=1.23.0 (from ultralytics)
  Downloading numpy-2.1.1-cp311-cp311-win_amd64.whl.metadata (59 kB)
Requirement already satisfied: matplotlib>=3.3.0 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from ultralytics) (3.10.1)
Collecting opencv-python>=4.6.0 (from ultralytics)
  Downloading opencv_python-4.11.0.86-cp37-abi3-win_amd64.whl.metadata (20 kB)
Requirement already satisfied: pillow>=7.1.2 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from ultralytics) (11.0.0)
Requirement already satisfied: pyyaml>=5.3.1 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from ultralytics) (6.0.2)
Requirement already satisfied: requests>=2.23.0 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from ultralytics) (2.32.3)
Collecting scipy>=1.4.1 (from ultralytics)
  Downloading scipy-1.15.2-cp311-cp311-win_amd64.whl.metadata (60 kB)
Requirement already satisfied: torch>=1.8.0 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from ultralytics) (2.6.0+cu126)
Requirement already satisfied: torchvision>=0.9.0 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from ultralytics) (0.21.0+cu126)
Requirement already satisfied: tqdm>=4.64.0 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from ultralytics) (4.67.1)
Requirement already satisfied: psutil in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from ultralytics) (7.0.0)
Collecting py-cpuinfo (from ultralytics)
  Downloading py_cpuinfo-9.0.0-py3-none-any.whl.metadata (794 bytes)
Requirement already satisfied: pandas>=1.1.4 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from ultralytics) (2.2.3)
Collecting seaborn>=0.11.0 (from ultralytics)
  Downloading seaborn-0.13.2-py3-none-any.whl.metadata (5.4 kB)
Collecting ultralytics-thop>=2.0.0 (from ultralytics)
  Downloading ultralytics_thop-2.0.14-py3-none-any.whl.metadata (9.4 kB)
Requirement already satisfied: contourpy>=1.0.1 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from matplotlib>=3.3.0->ultralytics) (1.3.1)
Requirement already satisfied: cycler>=0.10 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from matplotlib>=3.3.0->ultralytics) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from matplotlib>=3.3.0->ultralytics) (4.56.0)
Requirement already satisfied: kiwisolver>=1.3.1 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from matplotlib>=3.3.0->ultralytics) (1.4.8)
Requirement already satisfied: packaging>=20.0 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from matplotlib>=3.3.0->ultralytics) (24.2)
Requirement already satisfied: pyparsing>=2.3.1 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from matplotlib>=3.3.0->ultralytics) (3.2.1)
Requirement already satisfied: python-dateutil>=2.7 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from matplotlib>=3.3.0->ultralytics) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from pandas>=1.1.4->ultralytics) (2025.1)
Requirement already satisfied: tzdata>=2022.7 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from pandas>=1.1.4->ultralytics) (2025.2)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from requests>=2.23.0->ultralytics) (3.4.1)
Requirement already satisfied: idna<4,>=2.5 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from requests>=2.23.0->ultralytics) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from requests>=2.23.0->ultralytics) (2.3.0)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from requests>=2.23.0->ultralytics) (2025.1.31)
Requirement already satisfied: filelock in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from torch>=1.8.0->ultralytics) (3.13.1)
Requirement already satisfied: typing-extensions>=4.10.0 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from torch>=1.8.0->ultralytics) (4.12.2)
Requirement already satisfied: networkx in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from torch>=1.8.0->ultralytics) (3.3)
Requirement already satisfied: jinja2 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from torch>=1.8.0->ultralytics) (3.1.6)
Requirement already satisfied: fsspec in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from torch>=1.8.0->ultralytics) (2024.6.1)
Requirement already satisfied: sympy==1.13.1 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from torch>=1.8.0->ultralytics) (1.13.1)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from sympy==1.13.1->torch>=1.8.0->ultralytics) (1.3.0)
Requirement already satisfied: colorama in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from tqdm>=4.64.0->ultralytics) (0.4.6)
Requirement already satisfied: six>=1.5 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from python-dateutil>=2.7->matplotlib>=3.3.0->ultralytics) (1.17.0)
Requirement already satisfied: MarkupSafe>=2.0 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from jinja2->torch>=1.8.0->ultralytics) (3.0.2)
Downloading ultralytics-8.3.96-py3-none-any.whl (949 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 949.8/949.8 kB 14.6 MB/s eta 0:00:00
Downloading numpy-2.1.1-cp311-cp311-win_amd64.whl (12.9 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 12.9/12.9 MB 23.1 MB/s eta 0:00:00
Downloading opencv_python-4.11.0.86-cp37-abi3-win_amd64.whl (39.5 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 39.5/39.5 MB 54.6 MB/s eta 0:00:00
Downloading scipy-1.15.2-cp311-cp311-win_amd64.whl (41.2 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.2/41.2 MB 12.7 MB/s eta 0:00:00
Downloading seaborn-0.13.2-py3-none-any.whl (294 kB)
Downloading ultralytics_thop-2.0.14-py3-none-any.whl (26 kB)
Downloading py_cpuinfo-9.0.0-py3-none-any.whl (22 kB)
Installing collected packages: py-cpuinfo, numpy, scipy, opencv-python, ultralytics-thop, seaborn, ultralytics
  Attempting uninstall: numpy
    Found existing installation: numpy 2.1.2
    Uninstalling numpy-2.1.2:
      Successfully uninstalled numpy-2.1.2
Successfully installed numpy-2.1.1 opencv-python-4.11.0.86 py-cpuinfo-9.0.0 scipy-1.15.2 seaborn-0.13.2 ultralytics-8.3.96 ultralytics-thop-2.0.14

C:\vy>yolo track source=a.mp4
Creating new Ultralytics Settings v0.0.6 file ✅
View Ultralytics Settings with 'yolo settings' or at 'C:\Users\ADMIN\AppData\Roaming\Ultralytics\settings.json'
Update Settings with 'yolo settings key=value', i.e. 'yolo settings runs_dir=path/to/dir'. For help see https://docs.ultralytics.com/quickstart/#ultralytics-settings.
WARNING ⚠️ 'model' argument is missing. Using default 'model=yolo11n.pt'.
Downloading https://github.com/ultralytics/assets/releases/download/v8.3.0/yolo11n.pt to 'yolo11n.pt'...
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5.35M/5.35M [00:00<00:00, 7.05MB/s]
requirements: Ultralytics requirement ['lap>=0.5.12'] not found, attempting AutoUpdate...
Collecting lap>=0.5.12
  Downloading lap-0.5.12-cp311-cp311-win_amd64.whl.metadata (6.3 kB)
Requirement already satisfied: numpy>=1.21.6 in c:\users\admin\appdata\local\programs\python\python311\lib\site-packages (from lap>=0.5.12) (2.1.1)
Downloading lap-0.5.12-cp311-cp311-win_amd64.whl (1.5 MB)
   ---------------------------------------- 1.5/1.5 MB 19.6 MB/s eta 0:00:00
Installing collected packages: lap
Successfully installed lap-0.5.12

requirements: AutoUpdate success ✅ 16.4s, installed 1 package: ['lap>=0.5.12']
requirements: ⚠️ Restart runtime or rerun command for updates to take effect

C:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\cuda\__init__.py:235: UserWarning:
NVIDIA GeForce RTX 5090 with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_61 sm_70 sm_75 sm_80 sm_86 sm_90.
If you want to use the NVIDIA GeForce RTX 5090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

  warnings.warn(
Ultralytics 8.3.96 🚀 Python-3.11.9 torch-2.6.0+cu126 CUDA:0 (NVIDIA GeForce RTX 5090, 32607MiB)
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Scripts\yolo.exe\__main__.py", line 7, in <module>
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Lib\site-packages\ultralytics\cfg\__init__.py", line 987, in entrypoint
    getattr(model, mode)(**overrides)  # default args from model
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Lib\site-packages\ultralytics\engine\model.py", line 594, in track
    return self.predict(source=source, stream=stream, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Lib\site-packages\ultralytics\engine\model.py", line 543, in predict
    self.predictor.setup_model(model=self.model, verbose=is_cli)
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Lib\site-packages\ultralytics\engine\predictor.py", line 378, in setup_model
    self.model = AutoBackend(
                 ^^^^^^^^^^^^
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\utils\_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Lib\site-packages\ultralytics\nn\autobackend.py", line 163, in __init__
    model = model.fuse(verbose=verbose)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Lib\site-packages\ultralytics\nn\tasks.py", line 205, in fuse
    m.conv = fuse_conv_and_bn(m.conv, m.bn)  # update conv
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python311\Lib\site-packages\ultralytics\utils\torch_utils.py", line 259, in fuse_conv_and_bn
    w_bn = torch.diag(bn.weight.div(torch.sqrt(bn.eps + bn.running_var)))
                                               ~~~~~~~^~~~~~~~~~~~~~~~
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.


C:\vy>

Answered by Y-T-G

Mar 27, 2025

You just install PyTorch nightly and then Ultralytics

View full answer

UltralyticsAssistant · 2025-03-25T13:37:10Z

UltralyticsAssistant
Mar 25, 2025
Maintainer

👋 Hello @donhuvy, thank you for bringing this to our attention! 🚀 This is an automated response to help guide you while an Ultralytics engineer will assist you soon.

To address the RuntimeError: CUDA error: no kernel image is available for execution on the device, here are some steps you can follow:

Upgrade: Ensure you’re using the latest version of ultralytics and all relevant dependencies:
```
pip install -U ultralytics
```
Additionally, verify your PyTorch installation is compatible with your GPU and CUDA version. You can find the proper installation instructions for PyTorch here.
Minimum Reproducible Example (MRE): If this issue persists, please share a minimum reproducible example that demonstrates the error. This includes:
- The exact command or script you’re running.
- Any custom configurations or modifications made.
- Environment details (Python version, CUDA version, GPU model, etc.).
Environment Verification: YOLO is verified to work in the following environments. You can try running it in one of these to help isolate the issue:
- Notebooks with free GPU:
- Google Cloud Deep Learning VM. See GCP Quickstart Guide
- Amazon Deep Learning AMI. See AWS Quickstart Guide
- Docker Image. See Docker Quickstart Guide
Docs and Resources: Check out our Docs for guidance, especially the Python and CLI usage guides.

💡 If you want to connect with the community for additional troubleshooting, join:

Discord 🎧 for real-time support.
Discourse for detailed discussions.
Subreddit to share and learn with others.

Let us know if you’ve already tried any of the above steps or require further clarification. An engineer will follow up soon to assist you further. Thank you for your patience! 😊

0 replies

donhuvy · 2025-03-25T13:37:13Z

donhuvy
Mar 25, 2025
Author

3 replies

glenn-jocher Mar 25, 2025
Maintainer

@donhuvy the CUDA error occurs because your NVIDIA GeForce RTX 5090 GPU's compute capability (sm_120) isn't supported by your current PyTorch/CUDA installation. This typically happens with newer GPUs before official PyTorch support is added.

Two solutions:

Install PyTorch nightly build which often adds support for newer GPUs first:

pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu121

Temporarily use CPU by adding device=cpu to your command:

yolo track source=a.mp4 device=cpu

For full compatibility guidance, see PyTorch's official installation instructions.

donhuvy Mar 26, 2025
Author

Thank you so much! When YOLO use PyTorch version what support CUDA sm_120 , you can please notice me because I need custom training object detection for my thesis at my university, and I want use this new GPU NVIDIA RTX 5090.

glenn-jocher Mar 26, 2025
Maintainer

We recommend monitoring the official PyTorch release notes for CUDA compute capability updates, as Ultralytics YOLO inherits GPU support from PyTorch versions. When PyTorch adds sm_120 support in a stable release, it will automatically be available in subsequent Ultralytics updates. You can track PyTorch's CUDA compatibility progress at PyTorch Get Started.

donhuvy · 2025-03-25T13:38:03Z

donhuvy
Mar 25, 2025
Author

Please help me run success. Thank you!

1 reply

glenn-jocher Mar 25, 2025
Maintainer

@donhuvy the CUDA error occurs because your NVIDIA GeForce RTX 5090's compute capability (sm_120) isn't supported by your current PyTorch/CUDA installation. This is typically caused by driver/software mismatches. Let's resolve this:

Update NVIDIA Drivers: Get the latest drivers from NVIDIA's driver portal

Verify CUDA Compatibility: Install PyTorch with CUDA 12.x using:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121

Temporary Workaround: Force CPU usage by adding device='cpu' to your command:
```
yolo track source=a.mp4 device=cpu
```

For full troubleshooting steps, see the YOLO Common Issues guide. Let us know if you still encounter issues after trying these steps.

Y-T-G · 2025-03-26T00:39:08Z

Y-T-G
Mar 26, 2025
Maintainer

You can try installing nightly PyTorch

8 replies

donhuvy Mar 26, 2025
Author

Hi @Y-T-G , I think it is impossible because YOLO depends on stable PyTorch version (not nightly version).

Y-T-G Mar 26, 2025
Maintainer

Ultralytics works with nightly PyTorch

donhuvy Mar 27, 2025
Author

Can you teach me how to do with YOLO12 and nightly PyTorch? or send me links for my self-research.

Y-T-G Mar 27, 2025
Maintainer

You just install PyTorch nightly and then Ultralytics

Answer selected by donhuvy

donhuvy Mar 28, 2025
Author

Thank you so much!

glenn-jocher Mar 28, 2025
Maintainer

This CUDA compatibility error typically occurs when your GPU architecture (sm_120) isn't supported by your current PyTorch/CUDA version. For NVIDIA's latest GPUs like RTX 5090, you'll need to install PyTorch nightly with CUDA 12.1+ support using:

pip3 install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu121

After installation, verify CUDA compatibility with import torch; print(torch.cuda.is_available()). For full details see PyTorch's Get Started guide.

naptax Jun 6, 2025

Hello @glenn-jocher,

Thank you very much for your detailed explanations.

As the proud owner of an RTX 5090 (sm_120), I'm unfortunately waiting for the various solution providers to integrate version 2.8 :-(

I can't find anywhere to follow YOLO's release forecasts so I can watch for the integration of PyTorch 2.8.x into the product. Furthermore, I'm also looking for nightly builds to get this integration as soon as possible.

Thank you so much for your help.
Eric

glenn-jocher Jun 10, 2025
Maintainer

@naptax YOLO doesn't need to "integrate" specific PyTorch versions - it automatically works with whatever PyTorch installation you have. The sm_120 support depends entirely on PyTorch, not YOLO releases.

You can try PyTorch nightly right now for RTX 5090 support:

pip install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu121
pip install ultralytics

For YOLO releases, track the GitHub releases page, though PyTorch nightly should resolve your GPU compatibility immediately without waiting for any YOLO updates.

Ultralytics

RuntimeError: CUDA error: no kernel image is available for execution on the device #19863

Uh oh!

donhuvy Mar 25, 2025

Replies: 4 comments · 12 replies

Uh oh!

UltralyticsAssistant Mar 25, 2025 Maintainer

Uh oh!

Uh oh!

donhuvy Mar 25, 2025 Author

Uh oh!

glenn-jocher Mar 25, 2025 Maintainer

Uh oh!

donhuvy Mar 26, 2025 Author

Uh oh!

glenn-jocher Mar 26, 2025 Maintainer

Uh oh!

donhuvy Mar 25, 2025 Author

Uh oh!

glenn-jocher Mar 25, 2025 Maintainer

Uh oh!

Y-T-G Mar 26, 2025 Maintainer

Uh oh!

donhuvy Mar 26, 2025 Author

Uh oh!

Y-T-G Mar 26, 2025 Maintainer

Uh oh!

donhuvy Mar 27, 2025 Author

Uh oh!

Y-T-G Mar 27, 2025 Maintainer

Uh oh!

donhuvy Mar 28, 2025 Author

Uh oh!

glenn-jocher Mar 28, 2025 Maintainer

Uh oh!

naptax Jun 6, 2025

Uh oh!

glenn-jocher Jun 10, 2025 Maintainer

donhuvy
Mar 25, 2025

Replies: 4 comments 12 replies

UltralyticsAssistant
Mar 25, 2025
Maintainer

donhuvy
Mar 25, 2025
Author

glenn-jocher Mar 25, 2025
Maintainer

donhuvy Mar 26, 2025
Author

glenn-jocher Mar 26, 2025
Maintainer

donhuvy
Mar 25, 2025
Author

glenn-jocher Mar 25, 2025
Maintainer

Y-T-G
Mar 26, 2025
Maintainer

donhuvy Mar 26, 2025
Author

Y-T-G Mar 26, 2025
Maintainer

donhuvy Mar 27, 2025
Author

Y-T-G Mar 27, 2025
Maintainer

donhuvy Mar 28, 2025
Author

glenn-jocher Mar 28, 2025
Maintainer

glenn-jocher Jun 10, 2025
Maintainer