You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current default is 99 (e.g. in llama-bench). Some models already have around half of this.
99 also suggests to some people that this parameter could be a percentage or similar.
It being set to 99 confused me for a bit. Since it is stored as int32, I would suggest setting it to std::numeric_limits<int32>::max.
This would avoid confusion now, and incorrect testing once models grow to around 99 layers in the future.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
The current default is 99 (e.g. in
llama-bench
). Some models already have around half of this.99 also suggests to some people that this parameter could be a percentage or similar.
It being set to 99 confused me for a bit. Since it is stored as
int32
, I would suggest setting it tostd::numeric_limits<int32>::max
.This would avoid confusion now, and incorrect testing once models grow to around 99 layers in the future.
Beta Was this translation helpful? Give feedback.
All reactions