Langfuse Prompt Management - Ability to send label

[phasmax]

Langfuse prompt management (prompt retrieval by ID) works well. The only thing missing is the ability to also specify the prompt_label - ie in langfuse you can have "production", "stage" labels for versions of a prompt.

[krrishdholakia]

hey @phasmax how would you want to specify this?

[phasmax]

Hey @krrishdholakia.

Given the current naming standard (prompt_id, prompt_variables), I'd say "prompt_label" would be appropriate. If omitted or blank, just do what's currently done. If specified, pass label through to langfuse when retrieving prompt.

[mg-dev25]

@krrishdholakia Hi, I have maybe miss the doc information, but can we also passe the request message from a client like "openwebui" to the "prompt_variables" setting dynamically ? I didn't find a way to do that with litellm proxy.

Also if we could route the result to another LLM model before sending the response to the client.
I didn't find an example for this case. Thanks

[krrishdholakia]

Hey @mg-dev25 - you can specify the model in langfuse - https://docs.litellm.ai/docs/proxy/prompt_management#set-the-model-in-langfuse

For prompt_variables via openwebui - you just need to pass it in the request body, so as long as you can do that on openwebui (i think pipelines do this), that should work

curl -L -X POST 'http://0.0.0.0:4000/v1/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-d '{
    "model": "my-langfuse-model",
    "messages": [
        {
            "role": "user",
            "content": "THIS WILL BE IGNORED"
        }
    ],
    "prompt_variables": {
        "key": "this is used"
    }
}'

[mg-dev25]

Thanks, I will look how I can make this work.