Replies: 1 comment
-
GCC13 vs GCC12: According to: https://community.arm.com/arm-community-blogs/b/tools-software-ides-blog/posts/gcc-13 There were a number of improvements for aarch64. Anyone know if that is the only or at least primary reason for the performance improvement? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I thought arm64 and aarch64 were the same thing, but . . .
Using VMware on my Mac Mini m2 Pro 16GB, I find when running tests inference for TG (text generation) speeds, devoting 6 of the 10 Mac Mini processors and 10240MB (10GB) to the VM:
Compiling with GCC13 on Ubuntu:
(Ubuntu 24.04 VM) UNAME_M=aarch64 make: 17 t/s
(Ubuntu 24.04 VM) UNAME_M=arm64 make: 13 t/s
Compiling with GCC12 on stable Debian 12:
(Debian 12 VM) UNAME_M=arm64 make: 13 t/s
(Debian 12 VM) UNAME_M=aarch64 make: ERROR MESSAGES - due to missing aarch64 packages not part of GCC12
If Compiling with GCC13 on Debian 13 (testing channel so Debian 13 is risky to use):
(Debian 13 VM) UNAME_M=aarch64 make: 19 t/s
Why all these different speeds? Is this something specific to llama.cpp implementation or makefile instructions? Or is there something new about GCC13 that makes compiled code for Linux VMs running on Apple Silicon much faster?
Bonus question: And how could Debian 13 VM be any faster than Ubuntu 24.04 VM (19 t/s vs 17 t/s)?
Beta Was this translation helpful? Give feedback.
All reactions