You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, some questions:
(1) is there a straightforward way to dump logits for a new model being added to ggml? I read through the eval callback and the output building, but it's a bit opaque to me at what point the logits are actually materialized into the lctx.logits array.
(2) is there a way to constrain the input tokens per forward to 1? It would help if the input shapes are relatively static. I noticed ubatch appears to change the sequence length for the first forward pass, but it's not clear to me whether that behaves as I'd want it to (I more or less don't need batching at the moment)
(3) I'd like to disable most/all samplers, but I noticed that's a bit tricky. It seems to always include...
sampler chain: logits -> logit-bias -> penalties -> softmax -> dist
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
Hi, some questions:
(1) is there a straightforward way to dump logits for a new model being added to ggml? I read through the eval callback and the output building, but it's a bit opaque to me at what point the logits are actually materialized into the lctx.logits array.
(2) is there a way to constrain the input tokens per forward to 1? It would help if the input shapes are relatively static. I noticed ubatch appears to change the sequence length for the first forward pass, but it's not clear to me whether that behaves as I'd want it to (I more or less don't need batching at the moment)
(3) I'd like to disable most/all samplers, but I noticed that's a bit tricky. It seems to always include...
sampler chain: logits -> logit-bias -> penalties -> softmax -> dist
Thanks!
Beta Was this translation helpful? Give feedback.
All reactions