More on LLM message history and system instruction (neo4j#240)

* Add examples for LLM message history and system instructions

* Fix for when system_message is None

* Ruff

* Better (?) system prompt for RAG

* Update system_instruction

* Mypy

* Mypy

* Fix ollama test

* Fix anthropic test

* Fix cohere test

* Fix vertexai test

* Fix mistralai test

* Fix graphrag test

* Ruff

* Mypy

* Variable is not used

* Ruff...

* Mypy + e2e tests

* Ruffffffff

* CHANGELOG

* Fix examples

* Remove useless commented api_key from examples

Author: stellasia

CHANGELOG.md

@@ -4,8 +4,8 @@
 
 ### Added
 - Support for conversations with message history, including a new `message_history` parameter for LLM interactions.
-- Ability to include system instructions and override them for specific invocations.
-- Summarization of chat history to enhance query embedding and context handling.
+- Ability to include system instructions in LLM invoke method.
+- Summarization of chat history to enhance query embedding and context handling in GraphRAG.
 
 ### Changed
 - Updated LLM implementations to handle message history consistently across providers.

examples/README.md

@@ -51,6 +51,7 @@ are listed in [the last section of this file](#customize).
 ## Answer: GraphRAG
 
 - [End to end GraphRAG](./answer/graphrag.py)
+- [GraphRAG with message history](./question_answering/graphrag_with_message_history.py)
 
 
 ## Customize
@@ -73,6 +74,9 @@ are listed in [the last section of this file](#customize).
 - [Ollama](./customize/llms/ollama_llm.py)
 - [Custom LLM](./customize/llms/custom_llm.py)
 
+- [Message history](./customize/llms/llm_with_message_history.py)
+- [System Instruction](./customize/llms/llm_with_system_instructions.py)
+
 
 ### Prompts
 

examples/customize/embeddings/cohere_embeddings.py

@@ -2,7 +2,6 @@
 
 # set api key here on in the CO_API_KEY env var
 api_key = None
-# api_key = "sk-..."
 
 embeder = CohereEmbeddings(
     model="embed-english-v3.0",

examples/customize/embeddings/mistalai_embeddings.py

@@ -6,7 +6,6 @@
 
 # set api key here on in the MISTRAL_API_KEY env var
 api_key = None
-# api_key = "sk-..."
 
 embeder = MistralAIEmbeddings(model="mistral-embed", api_key=api_key)
 res = embeder.embed_query("my question")

examples/customize/embeddings/openai_embeddings.py

@@ -6,7 +6,6 @@
 
 # set api key here on in the OPENAI_API_KEY env var
 api_key = None
-# api_key = "sk-..."
 
 embeder = OpenAIEmbeddings(model="text-embedding-ada-002", api_key=api_key)
 res = embeder.embed_query("my question")

examples/customize/llms/anthropic_llm.py

@@ -2,8 +2,6 @@
 
 # set api key here on in the ANTHROPIC_API_KEY env var
 api_key = None
-# api_key = "sk-..."
-
 
 llm = AnthropicLLM(
     model_name="claude-3-opus-20240229",

examples/customize/llms/cohere_llm.py

@@ -2,7 +2,6 @@
 
 # set api key here on in the CO_API_KEY env var
 api_key = None
-# api_key = "sk-..."
 
 llm = CohereLLM(
     model_name="command-r",

examples/customize/llms/llm_with_message_history.py

@@ -0,0 +1,42 @@
+"""This example illustrates the message_history feature
+of the LLMInterface by mocking a conversation between a user
+and an LLM about Tom Hanks.
+
+OpenAILLM can be replaced by any supported LLM from this package.
+"""
+
+from neo4j_graphrag.llm import LLMResponse, OpenAILLM
+
+# set api key here on in the OPENAI_API_KEY env var
+api_key = None
+
+llm = OpenAILLM(model_name="gpt-4o", api_key=api_key)
+
+questions = [
+    "What are some movies Tom Hanks starred in?",
+    "Is he also a director?",
+    "Wow, that's impressive. And what about his personal life, does he have children?",
+]
+
+history: list[dict[str, str]] = []
+for question in questions:
+    res: LLMResponse = llm.invoke(
+        question,
+        message_history=history,  # type: ignore
+    )
+    history.append(
+        {
+            "role": "user",
+            "content": question,
+        }
+    )
+    history.append(
+        {
+            "role": "assistant",
+            "content": res.content,
+        }
+    )
+
+    print("#" * 50, question)
+    print(res.content)
+    print("#" * 50)

examples/customize/llms/llm_with_system_instructions.py

@@ -0,0 +1,22 @@
+"""This example illustrates how to set system instructions for LLM.
+
+OpenAILLM can be replaced by any supported LLM from this package.
+"""
+
+from neo4j_graphrag.llm import LLMResponse, OpenAILLM
+
+# set api key here on in the OPENAI_API_KEY env var
+api_key = None
+
+llm = OpenAILLM(
+    model_name="gpt-4o",
+    api_key=api_key,
+)
+
+question = "How fast is Santa Claus during the Christmas eve?"
+
+res: LLMResponse = llm.invoke(
+    question,
+    system_instruction="Answer with a serious tone",
+)
+print(res.content)

examples/customize/llms/mistalai_llm.py

@@ -2,8 +2,6 @@
 
 # set api key here on in the MISTRAL_API_KEY env var
 api_key = None
-# api_key = "sk-..."
-
 
 llm = MistralAILLM(
     model_name="mistral-small-latest",