AVX support

[KSepetanc]

Does ExaModels use CPU AVX instruction set?
AVX is SIMD technique. ExaModels is conceptually based on SIMD.
How to select AVX version etc.?

[sshin23]

Hmm... I have to confess that I'm not familiar with AVX instruction set. Is this something automatically taken care of by the julia compiler or at the LLVM level or is there something we need to do at the ExaModels level?

[KSepetanc]

"Advanced Vector Extensions (AVX) extensions are SIMD extensions to the x86 instruction set architecture for microprocessors." Basically it enables a processor to execute instruction on multiple data (simultaneously). AVX does 128 bit operations (two doubles), AVX2 does 256 bit operations and AVX512 does 512 bit operations. Support for it depends on CPU architecture. Zen 5 (by AMD) supports full native AVX512, Zen 4 supports AVX512 by using double cycles then AVX2, Zen 3 and previous do not have AVX512 support. Intel currently has only support for up-to AVX2 on consumer PC's, but should have support for it in future CPU releases (under name AVX-10). Basically if this is not in use already, having 512 bit operations can speed-up factor 8 computation as executing 512 bits simultaneously is 8 times more than 64 bits.

I don't know if this is handled by Julia compiler. Maybe it is. However, due to potential big performance advantage, I think we need to find out if it is used and how can we exploit that if it is not. Zen5 should provide masive performance advantage e.g. compared to zen3 if compiler handles this correctly.

[KSepetanc]

What worries me is that it is not used as when I start Julia (on Zen3 CPU) with command line argument to target some very old CPU such as pentium4 (julia -C pentium4) or to disable avx instructions (julia -C znver3,-avx,-avx2,-avx512bitalg), or even to enable them (julia znver3,+avx2) I see no difference. If used difference should be significant.

[frapac]

I agree that in theory, we could vectorize the operations in ExaModels using the AVX instruction sets. However, I am not sure it's a good idea to support AVX explicitly:

• KernelAbstractions.jl is optimized mostly for acceleration on GPUs, and offers poor performance on CPU.
• the packages supporting vectorization in Julia are currently under funded and slowly falling behind
• we cannot trust the Julia compiler to vectorize the operations automatically
• it's hard to predict if AVX instructions will be supported long term with the current situation at Intel (see also the debate around the AVX512 instruction set)

For the reasons listed above, and the current funding situation, I don't think it would be a good idea to support AVX instructions in ExaModels. If funding allows, I would be more in favor of improving the support of multithreading in ExaModels.

[KSepetanc]

Thank you for joining conversation.

• I read that KA.jl is changing CPU() backend for OpenCL backend. Do we expect CPU performance uplift from this? Base ExaModels I think do no use KA.jl, only in extension. Currently I do not see any benefit of using CPU() backend (see discussion).
• It is poor condition for Julia that emphasizes speed to not support AVX. I've tried forking ExaModels using @turbo macro on supported Julia 1.10, but LoopVectorization.jl seems does not support custom structs so it fails (ERROR: LoadError: Don't know how to handle expression. gradient!). This point is very significant and I agree that until better funding is provided current solution is good.
• Not sure why Julia compiler can not do it automatically, but I am accepting it as such.
• With AMD now supporting full native AVX512 (with Zen5) and stating that future development will use Zen5 for building upon, AVX512 is probably set to stay or be just modified by instruction set unification (there are multiple versions of AVX512). Intel dropped support due to E cores lacking them. AFAIK Intel is also set to reintroduce it under name AVX10 when they implement it in E cores.