wgpu vs. tch-gpu performance on ms windows and 3080 RTX

[dmk978]

Tried text-classification examples with different backends, and found out that wgpu training is going at about 0.25 iters/sec, CPU (Ryzen 9) - at about 1.3 iters/sec, and tch-gpu is about 15-17 iters/sec.

Is it OK? Tried with different env vars, but no luck.

Also to compile db-pedia-train with wgpu there is a need to specify:
#![recursion_limit = "256"]

[laggui]

Sorry for the delayed response!

wgpu training is going at about 0.25 iters/sec, CPU (Ryzen 9) - at about 1.3 iters/sec, and tch-gpu is about 15-17 iters/sec.

Wgpu is the most portable, but not the fastest. Though it certainly should be performing much better than the reported numbers here. Are you sure wgpu is detecting you GPU correctly and not using the CPU?

On my machine (laptop with 4050 GTX notebook), I get the following training stats on ag-news-train:

wgpu

Train epoch 10.12 iter/sec - batch 11.50 iter/sec

tch-gpu

Train epoch 12.34 iter/sec - batch 12.01 iter/sec

Also, the initial iter/sec reportings might be skewed by autotune when using wgpu.

[dmk978]

wgpu:

tch-gpu:

ndarray:

tch-cpu:

[laggui]

That's weird. Your iteration speed is closer to an integrated GPU or even CPU, but the reportage GPU memory usage seems to indicate GPU is used 🤔

What if you try with vulkan? It still uses the wgpu runtime but we have a SPIR-V compiler.

[dmk978]

I also tried with vulkan and cuda features, but they are basically the same - both in speed, displayed GPU number and GPU load.

My system does have integrated AMD GPU (built-in in Ryzen 9), but I believe it is not in use.

[laggui]

The reason why I asked is because Vulkan uses the wgpu runtime, but with our own compiler to generate SPIR-V compute shaders. So it is expected to be much better than just wgpu (using wgsl shaders).

Still not sure why you're seeing such slow iteration speed with wgpu though.

My system does have integrated AMD GPU (built-in in Ryzen 9), but I believe it is not in use.

Doesn't look like it either, at least not from the provided screenshots. The GPU memory is used and usage is 36% at the time of the screenshot.