configuration ideas

[bert269]

I have a 1080ti laying around - water-cooled. What is the suggested/tested hardware to run this on - intel or AMD platform?
I am an AMD fan, but need to know a little more about the WIS configuration and what is suggested - I could go either way, depending on the majority of platforms this was tested on.
Also, (again - if I will encounter this during the setup/config, I do appologize beforehand) - but what else is reguired for the WIS to run in terms of configuration / setup? Debian, headless I asume or does it need X as a desktop?
Again, if I asked questions that will be answered during the config, I appologize - I just need a proven starting point to get WIS up and running on the best tested platform.

Thanks.

[hamishcunningham]

I've got a similar gpu talking to an i7 laptop over thunderbird 3, ubuntu 22.04, works fine If you want to do Vicuna then things get tricky but for speech the CPU isn't too critical if you have >= 8gb VRAM I think Hth

…

On Sun, 25 Jun 2023, 15:49 bert269, ***@***.***> wrote: I have a 1080ti latying around - water-cooled. What is the suggested/tested hardware to run this on - intel or AMD platform? I am an AMD fan, but need to know a little more about the WIS configuration and what is suggested - I could go either way, depending on the majority of platforms this was tested on. Also, (again - if I will encounter this during the setup/config, I do appologize beforehand) - but what else is reguired for the WIS to run in terms of configuration / setup? Debian, headless I asume or does it need X as a desktop? Again, if I asked questions that will be answered during the config, I appologize - I just need a proven starting point to get WIS up and running on the best tested platform. Thanks. — Reply to this email directly, view it on GitHub <#92>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABBRBFFUB6UYZFNKAZY3MFLXNBFVNANCNFSM6AAAAAAZTGG6DI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[nikito]

As @hamishcunningham said the VRAM and number of CUDA cores is mainly what determines performance. That GPU will perform well for STT and TTS, bit I think at minimum you'd need an RTX 3090 with 24GB VRAM to start playing with LLMs. That said, cpu isn't really a big factor from what I have seen. I personally am running with an older i5 processor and a GTX 1070 and performance for STT and TTS is excellent, and I'm using the large model with a beam size of 5. You'd also want a decent amount of system RAM as well, I have 8GB on my server and that works fine. As far as Intel/AMD I Believe people here are using both and there's no issues there. Hope this is helpful!

[kristiankielhofner]

@nikito is correct - CPU isn't a big factor. Generally speaking anything that has the hardware support for an x16 slot of a decent PCIe version won't have a perceptible/meaningful impact on response times.

We also test with Intel and AMD CPUs - even ancient ones that don't even support AVX. The Nvidia container will print a scary warning but we haven't observed any illegal instructions, etc in GPU mode on these CPUs.