Replies: 1 comment
-
Never mind, it was my mistake: it was one of those silly small little things you spent hours on. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
I have a prompt and a grammar that basically constraints the LLM to chose from a list of options.
Today I noticed that it doesn't work quite as it should in a sense that from time to time if outputs text that is not in the list from the grammar. I don't have clear steps to replicate (this is why I didn't opened an issue).
I don't remember this happening ~2 weeks ago (worked as expected meaning is was always reliable).
I do sampling on GPU (I mentioned this because there are multiple GPU related commits in the last 2 weeks).
Does anyone also have this issue?
Is it a regression or grammar works by just increasing probabilities without fully enforcing the constraints?
Beta Was this translation helpful? Give feedback.
All reactions