how to generate longer outputs with what models of llama that is within 64gb cpu ram and 8gb vram?
#5224
Replies: 1 comment
-
|
anyone has answer to this? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
i saw on github issues that can generate 400k context but using gguf, i cant generate anything longer than 2048 no matter what parameters i put even if i put in -n 32400 -c 32400.
so how to generate longer outputs?
Beta Was this translation helpful? Give feedback.
All reactions