Significantly slower response following rename of main to llama-cli. #8148
-
|
I might be missing something obvious, but I happen to have both an old build which was running main.exe (I now have deleted that directory not knowing that it would be faster), and it ran about 5x faster than my current build which runs llama-cli.exe. Was this a simple rename or is there more to it? I can't seem to understand why the speed dropped so significantly. Thanks in advance! |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
I have not figured out the reason with 100% certainty, but I think it must have something to do with my CMake settings. I seem to have managed to replicate similar performance now by rebuilding with w64devkit. I tried rebuilding with CMake but it kept yielding the same speed, about 0.5 tokens/s. New build with w64devkit yields 2.68 tokens/s. Previously my main.exe file was in build/Release if that is helpful to anyone. I wonder whether the Release config build wasn't working in CMake for some reason. I'll leave it open in case anyone has anything else to add regarding insights, otherwise please feel free to close it. |
Beta Was this translation helpful? Give feedback.
I have not figured out the reason with 100% certainty, but I think it must have something to do with my CMake settings. I seem to have managed to replicate similar performance now by rebuilding with w64devkit.
I tried rebuilding with CMake but it kept yielding the same speed, about 0.5 tokens/s.
New build with w64devkit yields 2.68 tokens/s.
Previously my main.exe file was in build/Release if that is helpful to anyone. I wonder whether the Release config build wasn't working in CMake for some reason. I'll leave it open in case anyone has anything else to add regarding insights, otherwise please feel free to close it.