Make llama_samplers.cpp throw an error/exception on lack of choices during sampling #9952
SteelPh0enix
started this conversation in
Ideas
Replies: 1 comment
-
Open an issue with repro steps and this will get fixed. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I've recently noticed, while testing samplers options in llama.cpp, that current implementation aborts and crashes when sampling code (specifically,
llama_sampler_softmax_impl
in my case) encounters a situation where the length of tokens array is 0.IMO this should not be a critical error of this magnitude, and instead it should exit graciously, with either an exception or some other kind of error-propagation mechanism. However, i'm currently unfamiliar with llama.cpp codebase so i don't know what kind of approach would be preferred, hence i started this discussion.
Beta Was this translation helpful? Give feedback.
All reactions