Feedback on LCEL: it's difficult to read

[gardner]

It's almost been two months now. I thought I would get used to it, but I haven't. LCEL is difficult to read. I understand that it is declarative and composable and async native and objectively better. My subjective experience is that every time I see it I have a really hard time understanding what is going on.

It seems simple and elegant up to a point, and then, in real world situations the code looks like a shotgun spat some spaghetti on the wall.

[tigerinus]

Not only it's difficult to read, it is also hard to debug.

[eyurtsev]

Any ideas on what could help in improving readability or ability to debug?

if you (or other community members) have good examples of complex lcel chains that you're willing to share, please post in this discussion. It'll be helpful to see them to understand what improvements we can make.

[tigerinus]

I would argue if somehow it can have some discoverability, or some VSCode intellisense completion, that would be perfect. But honestly I don't know how exactly the development experience would look like.

[gardner]

Hi @eyurtsev, good question. I should have included some in the original post.

Generally, because they are so expressive and not a construct that I am used to seeing in python, it get's a bit weird when objects are constructed in the chain definition, or when an object literal is defined to pass in.

Embarrasingly, I went looking for the python documentation on pipeline operations. It took me a minute to realise that the magic happens in RunnableSequence.__or__(). Once we understand that it's domain specific operator overloading, it then means that understanding LCEL requires a level of understanding of the langchain API that most newcomers will not have. I still inspect return values to understand what it being returned by methods; this is difficult to do when constructing a RunnableSequence, at least for me.

Please keep in mind that I don't use python in my everyday work. While I do feel comfortable with it, some of the advanced features that might be comfortable to developers capable of producing langchain, might be a bit out of reach for casual users.

With that caveat in mind, please don't take this list as things that need to be fixed. Please instead read this as user feedback from someone that enjoys using langchain and wants to help it become better. How can we make the very expressive DSL easier to pick apart and understand?

Some difficult to understand examples:

From docs/docs/modules/chains/foundational/sequential_chains.ipynb

llm = ChatOpenAI()
chain = (
    {"synopsis": synopsis_prompt | llm | StrOutputParser()}
    | review_prompt
    | llm
    | StrOutputParser()
)
chain.invoke({"title": "Tragedy at sunset on the beach"})

• The above example appears to construct a chain inside the {} braces, which is then used to construct another chain. I don't understand why the {} braces are required. Does this create a dict or a set? Are the braces overloaded?
• It is unclear if the () parenthesis are overloaded. Is it just standard implicit line joining?

synopsis_chain = synopsis_prompt | llm | StrOutputParser()
review_chain = review_prompt | llm | StrOutputParser()
chain = {"synopsis": synopsis_chain} | RunnablePassthrough.assign(review=review_chain)
chain.invoke({"title": "Tragedy at sunset on the beach"})

In the above example we need to understand what RunnablePassthrough.assign() returns and then what that object's invoke() method returns. It is also unclear why chain.invoke({"title": "Tragedy at sunset on the beach"}) is explicitly called instead of being part of a RunnableSequence.

---

From: cookbook/advanced_rag_eval.ipynb

summarize_chain = {"element": lambda x: x} | prompt | model | StrOutputParser()

• When does the lambda get invoked? What is passed to it? We can piece it together if we inspect the code in a non-linear way. This is a small example of something that adds to confusion.

---

From: docs/docs/expression_language/cookbook/prompt_llm_parser.ipynb

chain = prompt | model.bind(function_call={"name": "joke"}, functions=functions)

• The above example could be more clear if we bind the model first, instead of in the LCEL constructor:

joke_model = model.bind(function_call={"name": "joke"}, functions=functions)
# a curous newbie could inspect the joke_model variable at this point
chain = prompt | joke_model

---

From: docs/docs/modules/chains/document/map_reduce.ipynb

# The final full chain
map_reduce = (map_as_doc_chain.map() | collapse | reduce_chain).with_config(
    run_name="Map reduce"
)

This is a great example of more than one thing happening as the RunnableSequence is being defined. It's difficult to understand what is happening at every stage of the pipe. What object is being returned? How does it decide what to call? What is with_config() and what class is it defined in?

---

Thanks for taking the time to read this. My intuition is that as LCEL continues to evolve, keeping feedback like this in mind when making changes or documenting it will help nudge it in the right directions. Some ideas:

• I could submit some PRs that unwind the documentation into more explicit steps.
• It might be useful to have "tabbed" code examples in the docs so that we can see dense examples and then a more verbose example as well. (This sounds like a lot of maintenance)
• Another thing would be improving inspectability / types as @tigerinus mentioned above.

[hwchase17]

Thanks for the feedback! What do you thinking of the following options:

1. focus on adding better documentation for LCEL
2. Focus on adding more “creator functions” in langchain, eg “create_sql_db_chain” that return standard LCEL chains (so you don’t have to write the basic ones)
3. Focus on adding more of the “old” styles of chains (eg the objects)
4. Other?

[tigerinus]

Thinking out loud here:

What do you say about building chain using https://en.m.wikipedia.org/wiki/Fluent_interface

[HaritzPuerto]

For me it's quite difficult to debug too. In particular, I would like to double check that the input prompt is correct but I don't know how to do this. I also couldn't find the way to access the prompt template from the RunnableSequence