KataGo v1.16.3 cuGetProcAddress_v2 Returns CUDA_ERROR_NOT_FOUND for D3D/WGL Functions and cuGraphInstantiateWithParams_ptsz

[KurococLiu]

zluda_trace logs (tarball/zip file)

zluda_log_katago.zip

Description

Description

When running KataGo (version katago-v1.16.3-cuda12.1-cudnn8.9.7-windows-x64) with ZLUDA as the CUDA compatibility layer on an AMD Radeon RX 7800 XT, the cuGetProcAddress_v2 function returns CUDA_ERROR_NOT_FOUND for several functions, primarily Direct3D (D3D9, D3D10, D3D11) and WGL interoperability functions, as well as cuGraphInstantiateWithParams_ptsz. Additionally, an empty symbol query (symbol: "") fails, which may indicate a minor issue in the application.

Expected Behavior

All queried CUDA functions should return CUDA_SUCCESS with valid function pointers (pfn), including graphics interoperability functions if supported by ZLUDA. KataGo should fully utilize GPU acceleration for neural network inference and self-play training without errors or performance degradation.

Actual Behavior

Most core CUDA functions (e.g., cuInit, cuMemcpy, cuLaunchKernel) are loaded successfully with CUDA_SUCCESS.

Graphics interoperability functions (e.g., cuWGLGetDevice, cuD3D11GetDevice, cuD3D9GetDevice) fail with CUDA_ERROR_NOT_FOUND, returning pfn: 0x0.

The function cuGraphInstantiateWithParams_ptsz fails with CUDA_ERROR_NOT_FOUND, which may impact CUDA Graph functionality.

An empty symbol query (symbol: "") fails with CUDA_ERROR_NOT_FOUND, which is expected but could indicate a minor bug in KataGo's function querying logic.

Steps to reproduce

1. Download KataGo v1.16.3 for Windows with Zluda 5-preview.136.
2. Extract the zip file to a directory (e.g., C:\KataGo).
3. Download a pre-trained model (e.g., from https://katagotraining.org/) and place it in the KataGo directory (e.g., kata1-b18c384nbt-20230523.bin.gz).
4. Copy nvcuda.dll(rename to cublas64_12.dll) and cudnn64_9.dll(rename to cudnn64_8.dll) to the KataGo directory
5. Run KataGo with GTP mode:

cd C:\KataGo
katago.exe gtp -model kata1-b18c384nbt-20230523.bin.gz -config gtp_example.cfg

ZLUDA version

5-preview.136

Operating System

Windows 11 (version 24H2 26100.3476)

GPU

AMD Radeon RX 7800 XT

[vosen]

Thanks for taking your time to collect traces. The CUDA_ERROR_NOT_FOUND in the log are benign.
I've reproed your set up and the logs are cut off due to poor Windows support in ZLUDA. We will be working on improving it (#522) so we can get all the necessary logs, but I can't give any timelines.
Is it possible for you to collect the logs on Linux in the meantime?

[KurococLiu]

Thanks for the response @vosen. As requested, I've collected the trace logs on Linux (WSL Ubuntu 24.04 LTS). The cuDNN panic persists with "not yet implemented" in zluda_dnn/src/impl.rs:25:5. Attached katago_log.zip containing trace logs, full backtrace.

Steps to reproduce on Linux:

• Environment: WSL2 Ubuntu 24.04 LTS, AMD Radeon RX 7800 XT, ROCm 6.4.2-120, ZLUDA 5-preview.139.
• Command: LD_LIBRARY_PATH="/mnt/f/Go/wsl/ZLUDA:$LD_LIBRARY_PATH" ./katago benchmark -model [model path] -config config.cfg
• GPU detected as "AMD Radeon RX 7800 XT [ZLUDA]", but crashes during neural net initialization.

katago_log.zip

[vosen]

Thanks, I've traced it and this is the list of CUDA functions we need for that workload

cublasCreate_v2
cublasHgemm
cudnnConvolutionForward
cudnnCreateFilterDescriptor
cudnnDestroyConvolutionDescriptor
cudnnSetTensor4dDescriptor
cudnnDestroyTensorDescriptor
cudnnDestroyFilterDescriptor
cudnnCreate
cublasSgemm_v2
cudnnDestroy
cudnnSetFilter4dDescriptor
cudnnSetConvolutionMathType
cublasDestroy_v2
cudnnGetConvolutionForwardWorkspaceSize
cudnnGetConvolutionForwardAlgorithm_v7
cudnnGetErrorString
cudnnSetConvolution2dDescriptor
cudnnCreateTensorDescriptor
cudnnCreateConvolutionDescriptor

In addition we need to split cudnn8 and cudnn9 (tey are slightly incompatible). I'll bump this issue when these land in ZLUDA

[KurococLiu]

Thanks, I've traced it and this is the list of CUDA functions we need for that workload

cublasCreate_v2
cublasHgemm
cudnnConvolutionForward
cudnnCreateFilterDescriptor
cudnnDestroyConvolutionDescriptor
cudnnSetTensor4dDescriptor
cudnnDestroyTensorDescriptor
cudnnDestroyFilterDescriptor
cudnnCreate
cublasSgemm_v2
cudnnDestroy
cudnnSetFilter4dDescriptor
cudnnSetConvolutionMathType
cublasDestroy_v2
cudnnGetConvolutionForwardWorkspaceSize
cudnnGetConvolutionForwardAlgorithm_v7
cudnnGetErrorString
cudnnSetConvolution2dDescriptor
cudnnCreateTensorDescriptor
cudnnCreateConvolutionDescriptor

In addition we need to split cudnn8 and cudnn9 (tey are slightly incompatible). I'll bump this issue when these land in ZLUDA

Thanks again for the update. To help move things forward, is there anything else I can do right now to assist with improvements, like providing more traces or testing specific setups?
Also, once these changes land, will the updated version support running on both Windows and Linux platforms seamlessly? If a version that resolves this issue gets released, could you let me know so I can test it and provide feedback?

[vosen]

I'll let you know when we have those features merged. We automatically build a new binary package after every change that gets merged, so you will be able to use that.
It won't be seamless on Windows. While it should work, there are two problems with the integration:

• Aligning all the libraries (nvcuda.dll, rocblas.dll, etc) in the right directory so they get loaded. This can be resolved manually
• MIOpen. Last time I checked it's not shipped with HIP SDK. I think there are unofficial Windows builds floating around. It's on you to find it and put in the right path