Override PromptValues: Using multi-step LLMChains

[sirsimonson]

I have the following question:

Given a Flow with 2 LLMChains (one of them beeing the output prediction) and 2 ChatPromptTemplates. In the 2nd LLMChain I use the output-prediction value as a promptValue in the 2nd PromptTemplate. When using the overrideConfig I must override ALL PromptValues and can not reuse the output_prediction. I get the error please provide promptValues for "x". What am I doing wrong?
Is it possible to override only a few PromptValues?

Here is my flow: (I want to reuse inference_result but override the context)

[Jaredude]

At issue is that using promptValues will replace the whole object rather than spread the two objects together. What you can do instead is place a custom js function in between the LLM Chain and the Chat Prompt Template and have the custom JS pull the value from overrideConfig.vars.context instead of using overrideConfig.promptValues.getPromptTemplate_0.context.

A flow along these lines:

will allow you to use both the inference and the context from overrideConfig.vars

Hope that helps

[sirsimonson]

Thanks, that's a neat workaround. I already tried similar approaches but tried to override the function's input parameters with no success since then all input Params must be overridden.

.. and thanks for pointing that out. I already looked into the code and debugged a bit in the replacer function inside server/src/utils/indes.ts. (I don't want to manage some fork)

Do you see any reason why it's not spreaded? Otherwise I would give it a try to contribute.

[Jaredude]

I think the issue with performing a spread within the existing util is unintended consequences. For instance, let's say you have variables and two of them are mutually exclusive. Spreading the two objects would result in a conflict between the mutually exclusive variables and force the user to come up with a workaround to solve that problem. In essence, we'd be solving this issue in this thread that you're encountering and creating a problem for someone else.

EDIT: Since both can be solved the same way by using a middleware function, to me it makes sense to leave it as is and solve this issue using this method and not create a potential breaking change to existing flows that are expecting the full overwriting of an object.

[sirsimonson]

@Jaredude agree, can you provide docs to this Middleware approach you're talking about?

I guess it would only make sense if you spread them only to the node-id notation? So only if one would have defined an object for one specific node.

[Jaredude]

By "middleware" function, I was meaning the example above where you place a custom JS function between the two nodes (LLM Chain and Chat Prompt Template so that you can check for the existence of $vars.context and pass that along. Hope that makes sense.

[HenryHengZJ]

I think we've put in a bugfix - #1645 are you on latest version?