Won't connect to local LLM (ollama)

[Packsolite]

Hello, i'm having trouble getting aider compose in vscode to connect to my ollama.

2025-03-01 04:39:56.698 [info] aider-chat: [2025-03-01 04:39:56,689] ERROR in app: Exception on /api/chat/setting [POST]
2025-03-01 04:39:56.699 [info] aider-chat: Traceback (most recent call last):
2025-03-01 04:39:56.699 [info] aider-chat:   File "C:\Users\TH\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_transports\default.py", line 101, in map_httpcore_exceptions
2025-03-01 04:39:56.699 [info] aider-chat:     yield
2025-03-01 04:39:56.700 [info] aider-chat:   File "C:\Users\TH\AppData\Local\Programs\Python\Python310\lib\site-packages\httpx\_transports\default.py", line 250, in handle_request
2025-03-01 04:39:56.700 [info] aider-chat:     resp = self._pool.handle_request(req)
2025-03-01 04:39:56.700 [info] aider-chat:   File "C:\Users\TH\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_sync\connection_pool.py", line 256, in handle_request
2025-03-01 04:39:56.700 [info] aider-chat:     raise exc from None
2025-03-01 04:39:56.701 [info] aider-chat:   File "C:\Users\TH\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_sync\connection_pool.py", line 236, in handle_request
2025-03-01 04:39:56.701 [info] aider-chat:     response = connection.handle_request(
2025-03-01 04:39:56.702 [info] aider-chat:   File "C:\Users\TH\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_sync\connection.py", line 101, in handle_request
2025-03-01 04:39:56.702 [info] aider-chat:     raise exc
2025-03-01 04:39:56.702 [info] aider-chat:   File "C:\Users\TH\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_sync\connection.py", line 78, in handle_request
2025-03-01 04:39:56.702 [info] aider-chat:     stream = self._connect(request)
2025-03-01 04:39:56.703 [info] aider-chat:   File "C:\Users\TH\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_sync\connection.py", line 124, in _connect
2025-03-01 04:39:56.703 [info] aider-chat:     stream = self._network_backend.connect_tcp(**kwargs)
2025-03-01 04:39:56.703 [info] aider-chat:   File "C:\Users\TH\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_backends\sync.py", line 207, in connect_tcp
2025-03-01 04:39:56.704 [info] aider-chat:     with map_exceptions(exc_map):
2025-03-01 04:39:56.704 [info] aider-chat:   File "C:\Users\TH\AppData\Local\Programs\Python\Python310\lib\contextlib.py", line 153, in __exit__
2025-03-01 04:39:56.705 [info] aider-chat:     self.gen.throw(typ, value, traceback)
2025-03-01 04:39:56.705 [info] aider-chat:   File "C:\Users\TH\AppData\Local\Programs\Python\Python310\lib\site-packages\httpcore\_exceptions.py", line 14, in map_exceptions
2025-03-01 04:39:56.705 [info] aider-chat:     raise to_exc(exc) from exc
2025-03-01 04:39:56.705 [info] aider-chat: httpcore.ConnectError: [WinError 10061] \u5bfe\u8c61\u306e\u30b3\u30f3\u30d4\u30e5\u30fc\u30bf\u30fc\u306b\u3088\u3063\u3066\u62d2\u5426\u3055\u308c\u305f\u305f\u3081\u3001\u63a5\u7d9a\u3067\u304d\u307e\u305b\u3093\u3067\u3057\u305f\u3002

Aider via cli works perfectly fine however:

any ideas?

[lee88688]

ollama's default port is 11434. so I think the port you set may wrong.

[Packsolite]

ollama's default port is 11434. so I think the port you set may wrong.

I double checked the port number is correct. After all, terminal works with the same port.

[lee88688]

httpcore.ConnectError: [WinError 10061] \u5bfe\u8c61\u306e\u30b3\u30f3\u30d4\u30e5\u30fc\u30bf\u30fc\u306b\u3088\u3063\u3066\u62d2\u5426\u3055\u308c\u305f\u305f\u3081\u3001\u63a5\u7d9a\u3067\u304d\u307e\u305b\u3093\u3067\u3057\u305f\u3002

this error shows that connection was refused by the target computer, so I believe there must be some mismatch between plugin and the ollama.
have you used this in the vscode remote?

[lee88688]

I see the python path (C:\Users\TH\AppData\Local\Programs\Python\Python310) in plguin is not the same as aider you display. so you can cd to the path used in plugin and use the aider in it to start in terminal and test if this aider is ok.

[Packsolite]

Aider is using the python installation from C:\Users\TH\AppData\Local\Programs\Python\Python310\

2025-03-03 02:36:06.765 [info] Extension "aider-composer" is now active!
2025-03-03 02:36:06.765 [info] Starting aider-chat service...
2025-03-03 02:36:06.765 [info] aider-chat process args: C:\Users\TH\AppData\Local\Programs\Python\Python310\python.exe -m flask -A c:\Users\TH\.vscode\extensions\lee2py.aider-composer-1.11.0\server\main.py run --port 13397
2025-03-03 02:36:11.785 [info] aider-chat: WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
2025-03-03 02:36:11.785 [info] aider-chat:  * Running on http://127.0.0.1:13397
2025-03-03 02:36:11.785 [info] server started: http://127.0.0.1:13397
2025-03-03 02:36:11.786 [info] aider-chat: Press CTRL+C to quit
2025-03-03 02:36:23.024 [info] aider-chat: 127.0.0.1 - - [03/Mar/2025 02:36:23] "POST /api/chat/setting HTTP/1.1" 200 -
2025-03-03 02:36:27.863 [info] From Webview: sendChatMessage: hello world

That is the same python as in path / used by the terminal:

And also the same as set in vscode settings for aida:

I also changed it to the default port you mentioned:

Still aider-compose fails to connect:

Again, standalone aida via terminal (both vscode terminal and standalone command line application) work fine.

[lee88688]

the issue you post is weird. I have no problem in my windows OS.

but I think the path Development\Python is not the same as Local\Programs\Python\Python310. they may have different packages and config. you may switch to Development\Python and start aider in command line to test if it had some errors.

[Packsolite]

the issue you post is weird. I have no problem in my windows OS.

but I think the path Development\Python is not the same as Local\Programs\Python\Python310. they may have different packages and config. you may switch to Development\Python and start aider in command line to test if it had some errors.

Development\Python is just my project folder with some python scripts i'm developing. It is not a python installation. I already started aider in that folder in the original post.

I don't know what i'm doing wrong. The extension just won't connect and i don't see any reason why...

[lee88688]

do you use the python version installed in the app store? I don't know if this could have problems.

[Packsolite]

do you use the python version installed in the app store? I don't know if this could have problems.

I used the official installer from python.org. I'll try a different version/distribution when i get to it.

[j2l]

I also can't reach my local ollama (docker) on http://0.0.0.0:11434 using Ubuntu 22.04 as host (my desktop).
It does send the POST request to aider, but it doesn't reach the ollama container:
2025-03-11 21:01:57.609 [info] aider-chat: 127.0.0.1 - - [11/Mar/2025 21:01:57] "POST /api/chat HTTP/1.1" 200 -

When saving the settings:

The ollama container has a request on api/show:
[GIN] 2025/03/11 - 20:07:04 | 200 | 13.354919ms | 172.17.0.1 | POST "/api/show"

Only the request doesn't go through.

Using "openai compatible" with local ollama throws another type of request to the container:
[GIN] 2025/03/11 - 20:11:55 | 404 | 17.843µs | 172.17.0.1 | POST "/chat/completions"
but it's a 404.

Aside ollama, aider-composer works well with other settings like openai for GPT4o.

Also my ollama container works well with aider:

aider --model ollama_chat/deepseek-coder-v2:latest
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Aider v0.76.2
Model: ollama_chat/deepseek-coder-v2:latest with whole edit format
Git repo: .git with 0 files
Repo-map: using 4096 tokens, auto refresh
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
> ls                                                                                                                                                  

Sure, please provide the filenames and paths for the files you'd like me to update.                                                                   


Tokens: 593 sent, 18 received.

and on the container:

[GIN] 2025/03/11 - 20:18:25 | 200 |         1m18s |      172.17.0.1 | POST     "/api/chat"

[GIN] 2025/03/11 - 20:18:25 | 200 |   21.938278ms |      172.17.0.1 | POST     "/api/show"

[GIN] 2025/03/11 - 20:18:25 | 200 |   12.604036ms |      172.17.0.1 | POST     "/api/show"

[GIN] 2025/03/11 - 20:18:25 | 200 |   13.046015ms |      172.17.0.1 | POST     "/api/show"

[GIN] 2025/03/11 - 20:18:25 | 200 |   13.492094ms |      172.17.0.1 | POST     "/api/show"

it needed time to load the model, but it went through.

My python path is the same for aider and aider-composer (/usr/bin/python)

[lee88688]

@j2l http://0.0.0.0:11434/ is not a valid url, OS can't find where the 0.0.0.0 is. you should bind a port in host from docker's ollama. then you should use this local port as base url.

[j2l]

@lee88688 http://0.0.0.0:11434 is totally valid URL, on Linux at least. https://en.wikipedia.org/wiki/0.0.0.0
0.0.0.0 means "any IPs assigned to this machine" (127.0.0.1, 192.168.0.2, 10.0.0.2, 172.17.0.2, ...), the OS (Ubuntu) resolves it.

I also tried with 127.0.0.1 and localhost, same issue.*

Hmm, on Linux, python packages are local (in home/) and python is global:
which aider
/home/pm/.local/bin/aider

which python
/usr/bin/python

OK, I made a virtual environment, now everything is local:

Successfully installed Werkzeug-3.1.3 aider-chat-0.76.2 aiohappyeyeballs-2.5.0 aiohttp-3.11.13 aiosignal-1.3.2 annotated-types-0.7.0 anyio-4.8.0 async-timeout-5.0.1 attrs-25.1.0 backoff-2.2.1 beautifulsoup4-4.13.3 blinker-1.9.0 certifi-2025.1.31 cffi-1.17.1 charset-normalizer-3.4.1 click-8.1.8 configargparse-1.7 diff-match-patch-20241021 diskcache-5.6.3 distro-1.9.0 exceptiongroup-1.2.2 filelock-3.17.0 flake8-7.1.2 flask-3.1.0 frozenlist-1.5.0 fsspec-2025.3.0 gitdb-4.0.12 gitpython-3.1.44 grep-ast-0.7.2 h11-0.14.0 httpcore-1.0.7 httpx-0.28.1 huggingface-hub-0.29.2 idna-3.10 importlib-metadata-7.2.1 importlib-resources-6.5.2 itsdangerous-2.2.0 jinja2-3.1.6 jiter-0.9.0 json5-0.10.0 jsonschema-4.23.0 jsonschema-specifications-2024.10.1 litellm-1.63.5 markdown-it-py-3.0.0 markupsafe-3.0.2 mccabe-0.7.0 mdurl-0.1.2 mixpanel-4.10.1 monotonic-1.6 multidict-6.1.0 networkx-3.2.1 numpy-1.26.4 openai-1.65.5 packaging-24.2 pathspec-0.12.1 pexpect-4.9.0 pillow-11.1.0 pip-25.0.1 posthog-3.19.1 prompt-toolkit-3.0.50 propcache-0.3.0 psutil-7.0.0 ptyprocess-0.7.0 pycodestyle-2.12.1 pycparser-2.22 pydantic-2.10.6 pydantic-core-2.27.2 pydub-0.25.1 pyflakes-3.2.0 pygments-2.19.1 pypandoc-1.15 pyperclip-1.9.0 python-dateutil-2.9.0.post0 python-dotenv-1.0.1 pyyaml-6.0.2 referencing-0.36.2 regex-2024.11.6 requests-2.32.3 rich-13.9.4 rpds-py-0.23.1 scipy-1.13.1 six-1.17.0 smmap-5.0.2 sniffio-1.3.1 socksio-1.0.0 sounddevice-0.5.1 soundfile-0.13.1 soupsieve-2.6 tiktoken-0.9.0 tokenizers-0.21.0 tqdm-4.67.1 tree-sitter-0.24.0 tree-sitter-c-sharp-0.23.1 tree-sitter-embedded-template-0.23.2 tree-sitter-language-pack-0.6.0 tree-sitter-yaml-0.7.0 typing-extensions-4.12.2 urllib3-2.3.0 watchfiles-1.0.4 wcwidth-0.2.13 yarl-1.18.3 zipp-3.21.0
(aidervenv) pm@pop-os:~$ which aider
/home/pm/aidervenv/bin/aider
(aidervenv) pm@pop-os:~$ which python
/home/pm/aidervenv/bin/python

I also tried with and without ollama_chat/ in settings:

in VSCode output I get:

2025-03-12 09:53:10.019 [info] aider-chat: �[31m�[1mWARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.�[0m
2025-03-12 09:53:10.020 [info] aider-chat:  * Running on http://127.0.0.1:16219
2025-03-12 09:53:10.020 [info] server started: http://127.0.0.1:16219
2025-03-12 09:53:10.020 [info] aider-chat: �[33mPress CTRL+C to quit�[0m
2025-03-12 09:53:57.627 [info] Add button clicked!
2025-03-12 09:53:57.643 [info] aider-chat: 127.0.0.1 - - [12/Mar/2025 09:53:57] "DELETE /api/chat HTTP/1.1" 200 -
2025-03-12 09:54:06.998 [info] From Webview: sendChatMessage: Create a small browser based JS tool in ES2022 format (index.html) that has:
...
2025-03-12 09:54:07.297 [info] aider-chat: 127.0.0.1 - - [12/Mar/2025 09:54:07] "POST /api/chat HTTP/1.1" 200 -
2025-03-12 09:54:16.691 [info] Setting button clicked!
2025-03-12 09:54:47.655 [info] aider-chat: 127.0.0.1 - - [12/Mar/2025 09:54:47] "POST /api/chat/setting HTTP/1.1" 200 -
2025-03-12 09:54:51.214 [info] Add button clicked!
2025-03-12 09:54:51.227 [info] aider-chat: 127.0.0.1 - - [12/Mar/2025 09:54:51] "DELETE /api/chat HTTP/1.1" 200 -

Still no luck, the request doesn't reach out (or is ignored by) the ollama container.

[butaca]

Same issue here. aider in command line works flawlessly with local ollama.

o4-mini works in aider-composer though, so this seems to be a specific issue with local ollama.

[butaca]

After some trial and error I managed to make it work with local ollama.

1. In the model field use the model name without the prefix ollama_chat/. For example: deepcoder
2. In the Base URL field don't use a trailing slash

This is working for me: