GoogleAiGeminiStreamingChatModel - Gemini AI #1574
Replies: 2 comments 10 replies
-
|
Currently we don't use the upstream implementations, but our own. It should be easy to introduce a streaming implemenation, would you be interested in contributing that? |
Beta Was this translation helpful? Give feedback.
-
|
I am not sure where to start 😃 |
Beta Was this translation helpful? Give feedback.
-
|
The implementation for ChatModel can be found here. An example of a StreamingChatModel implemenation (for Azure in this case) can be found here. If you need any more info, we'll gladly help out. |
Beta Was this translation helpful? Give feedback.
-
|
@geoand Can you please have a look here: https://github.com/KaiSuchomel/quarkus-langchain4j/tree/aigemini/streamingChatModel @SystemMessage("""
You are a professional ChatBot that can communicate with a Oncite DPS System
Context:
{chat_memory.extractDialogue}
""")
@McpToolBox("oncite")
String chat(@MemoryId int memoryId, @UserMessage String usermessage);
@SystemMessage("""
You are a professional ChatBot
Context:
{chat_memory.extractDialogue}
""")
@McpToolBox("oncite")
Multi<String> chatWithStream(@MemoryId int memoryId, @UserMessage String usermessage);
@SystemMessage("""
You are a professional ChatBot that does not use tools
Context:
{chat_memory.extractDialogue}
""")
Multi<String> chatWithStreamWOTools(@MemoryId int memoryId, @UserMessage String usermessage);The corresponding Resource: @Inject
GecAiService aiService;
@PUT
@Path("/chat/{memoryId}")
@Produces(MediaType.TEXT_PLAIN)
@Authenticated
public String chat(
@PathParam("memoryId") int aMemoryId,
String aUsermessage) {
return aiService.chat(aMemoryId, aUsermessage);
}
@PUT
@Path("/chat/{memoryId}/multi/tool")
@Produces(MediaType.TEXT_PLAIN)
@Blocking
@Authenticated
public Multi<String> chatWithMulti(@PathParam("memoryId") int aMemoryId, String aUsermessage) {
return aiService.chatWithStream(aMemoryId, aUsermessage);
}
@PUT
@Path("/chat/{memoryId}/multi")
@Produces(MediaType.TEXT_PLAIN)
@Blocking
@Authenticated
public Multi<String> chatWithMultiWOTool(@PathParam("memoryId") int aMemoryId, String aUsermessage) {
return aiService.chatWithStreamWOTools(aMemoryId, aUsermessage);
}I am using the following dependencies: <dependency>
<groupId>io.quarkiverse.langchain4j</groupId>
<artifactId>quarkus-langchain4j-ai-gemini</artifactId>
<version>${quarkus.langchain4j.version}</version>
</dependency>
<dependency>
<groupId>io.quarkiverse.langchain4j</groupId>
<artifactId>quarkus-langchain4j-mcp</artifactId>
<version>${quarkus.langchain4j.version}</version>
</dependency>
<dependency>
<groupId>io.quarkiverse.langchain4j</groupId>
<artifactId>quarkus-langchain4j-oidc-mcp-auth-provider</artifactId>
<version>${quarkus.langchain4j.version}</version>
</dependency>The MCP Server is also developed by me. I have attached the Logs of the different APIs. Call1-Tool.log As you can see in Call3 Log there is some illegal Thread Blocking..... |
Beta Was this translation helpful? Give feedback.
-
|
Okay, i will give it a try 👍 |
Beta Was this translation helpful? Give feedback.
-
|
🙏🏽 |
Beta Was this translation helpful? Give feedback.
-
|
@geoand Same Issue at openai: Result after Timeout: It seems there was an issue retrieving your information. Please try again later or let me know how else I can assist you! |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for checking. That's what I expected. Feel free to open a PR with your implementation and just keep the failing test disabled for now |
Beta Was this translation helpful? Give feedback.
-
|
I have created PR #1578 |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
With langchain4j/langchain4j#1951 GoogleAiGeminiStreamingChatModel was introduced.
Is this also possible to add here??
Beta Was this translation helpful? Give feedback.
All reactions