OllamaEndpointNotFoundError: Ollama call failed with status code 404. Maybe your model is not found and you should pull the model with ollama pull qwen:14b.

[guiniao]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

ollama list
NAME                                   ID                    SIZE             MODIFIED   
llama2:13b                           990f930d55c5            7.4 GB           7 days ago
qwen:14b                             80362ced6553            8.2 GB           6 days ago
The model is already running.

I successfully called using the following method.
`client = OpenAI(
    base_url = 'http://ip:11434/v1',
    api_key='ollama', # required, but unused
)
 
response = client.chat.completions.create(
  model="llama2:13b",
  messages=[
    # {"role": "system", "content": "You are a helpful assistant."},
    # {"role": "user", "content": "Who won the world series in 2020?"},
    # {"role": "assistant", "content": "The LA Dodgers won in 2020."},
    {"role": "user", "content": "how are you"}
  ]
)
print(response.choices[0].message.content)`

But when calling olama in langchain, an error was reported.
`from langchain_community.llms.ollama import Ollama

import json

base_url = 'http://ip:11434/'
api_key='ollama'

# model="llama2:13b"
model="qwen:14b"

chat_model = Ollama(model=model,base_url=base_url)

response = chat_model("how are you?")
print(response)`

The error is as follows:
OllamaEndpointNotFoundError               Traceback (most recent call last)
Cell In[3], [line 15](vscode-notebook-cell:?execution_count=3&line=15)
     [11](vscode-notebook-cell:?execution_count=3&line=11) # chat_model = ChatOpenAI(model_name=model,openai_api_key=api_key,openai_api_base=base_url,temperature=0.3)
     [12](vscode-notebook-cell:?execution_count=3&line=12) # chat_model = OpenAI(model_name=model,openai_api_key=api_key,openai_api_base=base_url,temperature=0.3)
     [13](vscode-notebook-cell:?execution_count=3&line=13) chat_model = Ollama(model=model,base_url=base_url)
---> [15](vscode-notebook-cell:?execution_count=3&line=15) response = chat_model("how are you")
     [16](vscode-notebook-cell:?execution_count=3&line=16) print(response)

File [~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/_api/deprecation.py:148](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/_api/deprecation.py:148), in deprecated.<locals>.deprecate.<locals>.warning_emitting_wrapper(*args, **kwargs)
    [146](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/_api/deprecation.py:146)     warned = True
    [147](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/_api/deprecation.py:147)     emit_warning()
--> [148](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/_api/deprecation.py:148) return wrapped(*args, **kwargs)

File [~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1086](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1086), in BaseLLM.__call__(self, prompt, stop, callbacks, tags, metadata, **kwargs)
   [1079](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1079) if not isinstance(prompt, str):
   [1080](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1080)     raise ValueError(
   [1081](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1081)         "Argument `prompt` is expected to be a string. Instead found "
   [1082](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1082)         f"{type(prompt)}. If you want to run the LLM on multiple prompts, use "
   [1083](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1083)         "`generate` instead."
   [1084](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1084)     )
   [1085](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1085) return (
-> [1086](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1086)     self.generate(
   [1087](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1087)         [prompt],
   [1088](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1088)         stop=stop,
...
    [248](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_community/llms/ollama.py:248)         )
    [249](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_community/llms/ollama.py:249)     else:
    [250](https://vscode-remote+ssh-002dremote-002bjumpser-005fml-005f145.vscode-resource.vscode-cdn.net/home/lproot/ml/lanchain/~/.conda/envs/openai/lib/python3.10/site-packages/langchain_community/llms/ollama.py:250)         optional_detail = response.text

OllamaEndpointNotFoundError: Ollama call failed with status code 404. Maybe your model is not found and you should pull the model with `ollama pull qwen:14b`.

Is there a way to solve it,thank you

Description

I call olama llm server (openai API) using langchain,unsuccessful

System Info

Langchain Version: 0.1.16
langchain-community: 0.0.34
langchain-core: 0.1.45
langchain-text-splitters: 0.0.1
Python : 3.10
centos : 7.9

[dosubot[bot]]

To address the OllamaEndpointNotFoundError you're encountering with the qwen:14b model, follow these steps:

1. Confirm the Model Name: Make sure qwen:14b is correctly spelled and matches the model name listed by ollama list.
2. Pull the Model Again: Execute ollama pull qwen:14b to ensure the model is properly loaded on your Ollama server.
3. Verify the Base URL: Ensure the base_url in your code matches the Ollama server's address where qwen:14b is hosted. The default is usually http://localhost:11434, but it seems you're using a different URL.
4. Compatibility Check: Ensure there's no version mismatch between LangChain and the Ollama server that might cause the qwen:14b model to be unrecognized.
5. Server Logs: If the issue persists, consulting the Ollama server logs might provide more insight into the problem.

These steps should help you resolve the issue and successfully call the qwen:14b model using LangChain.

Details

Sources

• Ability to set ollama serve url
• OllamaEndpointNotFoundError: Ollama call failed with status code 404. Maybe your model is not found and you should pull the model with ollama pull llama2.
• langchain_community.llms.ollama.OllamaEndpointNotFoundError: Ollama call failed with status code 404
• Error while loading the Llama 2 70B chat model
• libs/community/langchain_community/llms/ollama.py
• libs/community/langchain_community/embeddings/ollama.py
• docs/docs/integrations/llms/ollama.ipynb

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[guiniao]

The ollama service is hosted on a different machine.
I have set the olama serve URL and RE ollama pull qwen:14b.It still doesn't work
I conducted the following test on another machine deploying ollama server and the result was successful
curl http://1.1.1.1:11434/api/chat -d '{
"model": "qwen:14b",
"messages": [
{ "role": "user", "content": "why is the sky blue?" }
]
}';
1.1.1.1 is the IP address of my ollama server deployment.

But,I tried to call Ollma server using Langchain, but an error occurred.
`from langchain_community.llms.ollama import Ollama

import json

base_url = 'http://1.1.1.1:11434/'
api_key='ollama'

model="llama2-chinese:13b"

model="qwen:14b"

chat_model = ChatOpenAI(model_name=model,openai_api_key=api_key,openai_api_base=base_url,temperature=0.3)

chat_model = OpenAI(model_name=model,openai_api_key=api_key,openai_api_base=base_url,temperature=0.3)

chat_model = Ollama(model=model,base_url=base_url)

response = chat_model("how are you")
print(response)`

OllamaEndpointNotFoundError Traceback (most recent call last)
Cell In[7], line 15
11 # chat_model = ChatOpenAI(model_name=model,openai_api_key=api_key,openai_api_base=base_url,temperature=0.3)
12 # chat_model = OpenAI(model_name=model,openai_api_key=api_key,openai_api_base=base_url,temperature=0.3)
13 chat_model = Ollama(model=model,base_url=base_url)
---> 15 response = chat_model("how are you")
16 print(response)

File ~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/_api/deprecation.py:148, in deprecated..deprecate..warning_emitting_wrapper(*args, **kwargs)
146 warned = True
147 emit_warning()
--> 148 return wrapped(*args, **kwargs)

File ~/.conda/envs/openai/lib/python3.10/site-packages/langchain_core/language_models/llms.py:1086, in BaseLLM.call(self, prompt, stop, callbacks, tags, metadata, **kwargs)
1079 if not isinstance(prompt, str):
1080 raise ValueError(
1081 "Argument prompt is expected to be a string. Instead found "
1082 f"{type(prompt)}. If you want to run the LLM on multiple prompts, use "
1083 "generate instead."
1084 )
1085 return (
-> 1086 self.generate(
1087 [prompt],
1088 stop=stop,
...
248 )
249 else:
250 optional_detail = response.text

OllamaEndpointNotFoundError: Ollama call failed with status code 404. Maybe your model is not found and you should pull the model with ollama pull qwen:14b.

I have tried many methods, such as modifying the version of Langchain and changing methods, but none of them worked.

[thedarklordkitten]

I have a very similar issue with the following code.

This works:

from langchain_openai import ChatOpenAI
ollama_llm = ChatOpenAI( ...

This does not:

from langchain_community.chat_models import ChatOllama
ollama_llm = ChatOllama( ...

[SiyuanLi-Sven]

you solved my problem, many thanks. I tried the following code and it works.

from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI


output_parser = StrOutputParser()

ollama_llm = ChatOpenAI(
    model='llama3.2:latest',
    api_key="ollama", 
    base_url="http://yourDomainOrIP:11434/v1/"
)

prompt = 'say and only say hello'

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant."),
    ("user", f"{prompt}")
])

chain = prompt | ollama_llm | output_parser

chain.invoke({})