ryzen ai HX 370 (Radeon 890m) + ROCm 6.3.2, Am I using it wrong?

[bear8203]

I compared the 4060ti 16GB eGPU (self-made with TH3P4G2), the 7840HS (Firebat MN56), and the newly purchased Ryzen AI HX370 (Beelink SER9).

I'm comparing these two because their combined prices are similar. Also, I'm using the eGPU (the Firebat) for Deepseek R1 or IBM Granite tests, and the newly purchased HX370 has faster memory speeds than the Firebat (5600 vs. 7500 MT/s). Even though it is a bit slower than the RTX eGPU combination when using ROCm GPU, I ended up buying it because I was swayed by AMD's flashy advertising (bought it on discount for $899)...

The environment is the first workflow of the comfyUI (512x512px + SD1.5 base model - v1-5-pruned-maonly.ckpt)

• 1. RTX4060ti + 7840HS (Cuda & CPU Offloading) = takes 2 seconds
• 2. (I bought 2nd hands) 7840HS (Micron Crucial DDR5-5600 CL46 - 32GB dual channel) = takes 40~50 seconds
• 3. Pretty New Ryzen 9 AI - Our Shiny Expensive HX370 (LPDDR5X 7500MHz 32GB(8GB 4 Channel!!) = takes 30~40 seconds

Be honest (From today's price)

• 1. Price is $750.00 (the RTX $480.00 + egpu $120.00 + 2nd hands 7840HS $150.00) = took 2 seconds
• 2. Price is $150.00 (the 2nd hands 7840HS $150.00) = took, let's say 45 seconds
• 3. Price is $999.00 (beelink SER9 shiny pretty new Ryzen 9 AI HX370 with more memory with channels) = took, 35 seconds

---

What is better for a typical user right now? I'll share my service recipe, and I'd love to hear your creative and challenging ideas! (for my shame $899.00 😭 )

[Unit]
Description=ComfyUI Service
After=network.target

[Service]
Type=simple
User=root
WorkingDirectory=/root/ComfyUI
Environment=VIRTUAL_ENV=/root/ComfyUI/.venv
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin:/root/ComfyUI/.venv/bin
Environment=HSA_OVERRIDE_GFX_VERSION=11.0.1
Environment=GIT_PYTHON_GIT_EXECUTABLE=/usr/bin/git
Environment=GIT_PYTHON_REFRESH=quiet
ExecStartPre=/bin/chmod 666 /dev/dri/card1 /dev/dri/renderD128 /dev/kfd
ExecStart=/root/ComfyUI/.venv/bin/python3 /root/ComfyUI/main.py --force-fp32 --use-split-cross-attention --listen 0.0.0.0 --port 8188 --enable-cors-header "*"
Restart=always

[Install]
WantedBy=multi-user.target

Capture

???

I've been thinking, if I buy four used 7840HS devices ($150.00 x 4 = $600.00) and try Kubernetes on them, which is known to be challenging, would that be a more cost-effective way to experiment?

[ivoras]

Looks like AMD HX370 is a dud as far as AI tasks are concerned: https://www.reddit.com/r/LocalLLaMA/comments/1i7cj11/amd_hx370_llm_performance/ . Neither the NPU or GPU are supported by official AMD ROCm drivers at this time.

[zakyhassani]

looks like i bought a dud too hhhhh

[liszper]

What a dud 🥹

[abraxaswd]

very sad... yes. Too BAD AMD hypes this as their AI Chip and then not even supporting it with drivers.

[zakyhassani]

anyone tried the new XDNA driver in 6.14 kernel?

[RobertoMaurizzi]

I compiled it on a 6.12 kernel.. .useless. Nothing interesting uses it.

[stopandgo-44]

Can you try to compare the 4060ti without offloading to CPU/RAM at all? I'm curious if you only use that Nvidia GPU how that would work. Of course, unless the model literally doesn't fit on the GPU, but if it can fully fit on VRAM, it'd be interesting.

[hetzbh]

Seen this?
https://github.com/likelovewant/ROCmLibs-for-gfx1103-AMD780M-APU

[fighter3005]

I compared the 4060ti 16GB eGPU (self-made with TH3P4G2), the 7840HS (Firebat MN56), and the newly purchased Ryzen AI HX370 (Beelink SER9).

I'm comparing these two because their combined prices are similar. Also, I'm using the eGPU (the Firebat) for Deepseek R1 or IBM Granite tests, and the newly purchased HX370 has faster memory speeds than the Firebat (5600 vs. 7500 MT/s). Even though it is a bit slower than the RTX eGPU combination when using ROCm GPU, I ended up buying it because I was swayed by AMD's flashy advertising (bought it on discount for $899)...

The environment is the first workflow of the comfyUI (512x512px + SD1.5 base model - v1-5-pruned-maonly.ckpt)

• 1. RTX4060ti + 7840HS (Cuda & CPU Offloading) = takes 2 seconds
• 2. (I bought 2nd hands) 7840HS (Micron Crucial DDR5-5600 CL46 - 32GB dual channel) = takes 40~50 seconds
• 3. Pretty New Ryzen 9 AI - Our Shiny Expensive HX370 (LPDDR5X 7500MHz 32GB(8GB 4 Channel!!) = takes 30~40 seconds

Are these speeds on the iGPU (780M I believe)? I was wondering how you got it working. I have a 5750G with a Vega 8 GPU and I struggled to get it working (crashes as soon as CLIP should be loaded).

[RobertoMaurizzi]

On my Ryzen HX370:
On 980m iGPU it isn't hard if you use Vulkan. It's just work using llama.cpp or lmstudio (well, it uses llama.cpp).

Interesting, I also didn't know you can use Vulkan with Comfy (or maybe you're referring to llama.cpp only?). I'll see what I can find. Ollama, as I said, has a compatible ROCm runtime that it downloads on installation... I now wonder if I can use it 🤔

On NPU tho, I tried building XDNA1 driver and failed. I can't even detect the NPU.

I was able to get it to compile and load, with test passing and I see the device using xrt-smi... but then there's nothing generally useful that can use it on Linux from what I was able to find :-(

[zakyhassani]

Interesting, I also didn't know you can use Vulkan with Comfy (or maybe you're referring to llama.cpp only?). I'll see what I can find. Ollama, as I said, has a compatible ROCm runtime that it downloads on installation... I now wonder if I can use it 🤔

No, I've used only llama.cpp. not ComfyUI. But AFAIK ComfyUI can be configured to use Openai API backend too, that means lmstudio backend will work perfectly.

I was able to get it to compile and load, with test passing and I see the device using xrt-smi... but then there's nothing generally useful that can use it on Linux from what I was able to find :-(

At least it's a progress. May I know the steps? I'd like to try it again. Not that I need to use the local NPU, but it's still a part of this product and I feel like it's a waste not to use it. Last time I tried it was at February, I'd like to try again sometime later.

[RobertoMaurizzi]

@zakyhassani sorry for the late reply, I do have some docs from when I built the driver, mostly useful for Debian I guess.
I cleaned them up a bit and uploaded them in a gist here: https://gist.github.com/RobertoMaurizzi/8613dd0aae85877ab4f1ba391ca7c26a

[RobertoMaurizzi]

I tried updating the repository to the current main and it doesn't compile anymore, I'm getting an error on a missing include (that is in the repo but in a different directory... -__-' )

[RobertoMaurizzi]

update: the change I made around xdna-driver/xrt/src/CMake/cpackLin.cmake:139 blocked the git submodule update --init --recursive from updating the xrt repository... once I reverted it and updated, following the instructions produced a working driver. It's likely that they removed or fixed the problematic "ubuntu 23.10" check (there were about 160 commits from last time I built it... 😬 )

[LuXuxue]

i failed to run comfyui on linux on 780m, and now i use it on windows by therock. it can use fp16 and faster than fp32.
it seems that therock project have done some work on gfx1151, and nightly builds have been released(https://d2awnip2yjpvqn.cloudfront.net/v2/gfx1151/), but no official support on 780m(gfx1103) and 890m(gfx1150)
we still need to wait