-
|
I was thinking about making a PR that adds a parameter to the server to customize the tokens that DRY will use when computing its penalties. Something like #3727, but for DRY. It will solve the same problems. However, when inspecting the current code I now see that the original penalty prompt functionality is removed. Seems like it was done by #9294. I didn't find any explanation on why it was removed. I assume it was too cumbersome to port it to the new refactored code in that PR? Does it mean that I can submit a new PR that re-adds the penalty prompt? What about a penalty prompt for DRY? Or does it mean that penalty prompt functionality will not be accepted into this project anymore? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
The main reason to remove this functionality was I had a hard time to adapt it to the changes in the PR. If there are server tests for this, then it would be easier to accept. My opinion is that repetition penalties are generally always wrong to use. #3727 highlights and attempts to solve one of the main problems with them. But IMO it's simply easier to not use repetition penalties all together. I might be biased, since the main LLM applications that I am interested are in information retrieval and code completion and these kind of penalties (together with many other sampling techniques) are irrelevant. |
Beta Was this translation helpful? Give feedback.
The main reason to remove this functionality was I had a hard time to adapt it to the changes in the PR. If there are server tests for this, then it would be easier to accept.
My opinion is that repetition penalties are generally always wrong to use. #3727 highlights and attempts to solve one of the main problems with them. But IMO it's simply easier to not use repetition penalties all together.
I might be biased, since the main LLM applications that I am interested are in information retrieval and code completion and these kind of penalties (together with many other sampling techniques) are irrelevant.