Guide or Script to “tune” args/parameters to main? #1372
Unanswered
Free-Radical
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
As a noob to ML/AI I am not finding good examples (or simple explanations) showing what values to set the Args/parameters (top_p, ctx, etc) to main.
I have read the doc but it's really not explanatory enough for me to tune and run the different llms efficiently on my hardware.
And from the questions I have seen, many people are having the same problem.
Is there a decent comprehensive guide with examples somewhere that explains how to pass parameters to main per model and hardware? or perhaps a script that does an "auto-tune" ?
If not, someone who can do this ( I would be very happy to help) really needs to setup such a resource. It will be a tremendous benefit to promoting "edge" inference.
Look forward to responses. Thank you everyone and especially @ggerganov for your excellent work.
Beta Was this translation helpful? Give feedback.
All reactions