ChatGPT API and Chat models in general

[hwchase17]

The ChatGPT API came out today and had a pretty different interface than before. Rather than being "text in, text out" it was exposed with a "list of messages in, list of messages out" format.

The first integration we did was to create a wrapper that just treated ChatGPT API as a normal LLM: #1367

However, this seems a bit limiting in allowing users to take advantage of the full functionality of ChatGPT API (or other chat APIs in the future). Therefore, it probably makes sense to rethink some of the lower level abstractions. This will likely have the consequence of changing a lot of the internals of the chains, although the high level interface should remain similar

An attempt to start doing this is here: #1375

If you just want to use the code, I cut a prerelease to allow people to do so

pip install --pre langchain==0.0.99rc0

So far, some of the more common chains that seem appropriate for chat have been implemented:

• ConversationChain
• QAChain
• VectorDBQAChain
• ChatVectorDBChain

I am extremely interested in hearing others opinions on both the low level abstractions that will be needed to support this new type of API as well as the chain types we should prioritize adding support for these types of interfaces.

[whitead]

Memory design looks seamless for users! Amazed how fast you got that worked out and functional.

Not a fan of having constructor being same name - OpenAI. I imagine there are times we do actually want to have vanilla LLMs and Chat LLMs in the same namespace. I know we can just import module, but I can see this being confusing if someone just copy/pastes the code from examples.

So instead of from langchain.chat_models.openai import OpenAI would prefer from langchain.chat_models.openai import OpenAIChat

For chain types - will this work with agents? Can we have more than one role there beside system? Curious about thoughts on that.

[hwchase17]

looking into agents tmrw

fair point on naming

[FILINIAL]

Hm... Its interesting! You guys are cool that you can write programs) I love such people!You guys are cool, you do chat, it's probably difficult to do such things! But you can also do simple games for example)

[not-poma]

I think OpenAIChat.generate() function should accept the complete list of messages, similar to how in regular OpenAI class it accepts the entire prompt.

Even for non-chat applications I found that splitting typical prompts into a conversation improves the output. For example my summary prompt:

user: Write the summary according to the following instructions ...
assistant: Ok I remembered the instructions and will closely follow them. Now provide some examples of how the the result should be formatted.
user: {examples}
assistant: Ok I remembered the examples and will use similar formatting in my answer, but will not usethe example content. Now provide me with the context
user: {context}

So I think langchain should treat the entire message array as a prompt. It can still accept a single string for simplicity by internally checking whether the argument type is List or str to conveniently accept simple queries.

[hwchase17]

agree, take a look at the pr linked below for updated thoughts on prompting!

[tmanager22]

I have a question.

from langchain.prompts.prompt import PromptTemplate
from langchain.llms import OpenAI
from langchain.chains import ChatVectorDBChain

_template = """Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question.
You can assume the question is about 

Chat History:
{chat_history}
Follow Up Input: {question}
Standalone question:"""
CONDENSE_QUESTION_PROMPT = PromptTemplate.from_template(_template)

template = """You are an AI assistant for the . The documentation is located at .
You are given the following extracted parts of a long document and a question. Provide a conversational answer with a hyperlink to the documentation.
Always use urls that are given in 'source' in the context. Do NOT change the url that is give as source in the context.
If the question includes a request for code, provide a code block directly from the documentation.
If you don't know the answer, just say "Hmm, I'm not sure." Don't try to make up an answer.
If the question is not about, politely inform them that you are tuned to only answer questions about .

Question: {question}
=========
{context}
=========
Answer in Markdown:"""
QA_PROMPT = PromptTemplate(template=template, input_variables=["question", "context"])

print(QA_PROMPT)


def get_chain(vectorstore):
    llm = OpenAI(temperature=0)
    qa_chain = ChatVectorDBChain.from_llm(
        llm,
        vectorstore,
        qa_prompt=QA_PROMPT,
        condense_question_prompt=CONDENSE_QUESTION_PROMPT,
        return_source_documents=True
    )
    print(qa_chain)
    return qa_chain

Q1. How to print the prompt with each query?
Q2. How to pass sources(hyperlink) in context for AI to return correct hyperlink. Currently it's giving wrong madeup hyperlink

[hwchase17]

updated a pr with more thoughts on how we'll introduce the abstractions

pr: #1424

docs: https://langchain.readthedocs.io/en/harrison-chat-models/modules/chat.html

[not-poma]

What's the motivation to introduce separate classes like HumanMessage and AIMessage rather than making ChatMessage with .role or .type enum property? All the message classes behave exactly the same and have no additional logic to differentiate them.

Could look like msg = ChatMessage(role="assistant", content="You are a helpful assistant that translates English to French."). It's also closer to how API works.

[phiweger]

updated a pr with more thoughts on how we'll introduce the abstractions

pr: #1424

docs: https://langchain.readthedocs.io/en/harrison-chat-models/modules/chat.html

link to docs broken

[mehdifanai]

would appreciate if someone could kindly clarify for a beginner about the status of chatgpt API integration and why the docs link is broken . Thank you so much!