Should we have a ollama dev services?

[emmanuelbernard]

When starting with ollama from a quarkus app, I ended up with "cannot connect to ...". So I figured it out and installed ollama on my machine. Then I realized my ollama did not have downloaded the models I wanted and I did a ollama pull.

A devservices and a way to read the config could make that transparent for the developer.

Here are possible cons though:

• ollama is relatively easy to install so no biggie?
• if in a container, how to retain the downloaded models inbetween restarts. On a (linux) host, ollama stores things in ~/.ollama
• the devservice would need to recognize that a local ollama is running and not start it? I don't know how other devservices recognize a local run and ignore the dev service styrat? Just the explicit URL presence in the config?

CC @geoand @dandreadis

[geoand]

We tried something like that and it was not a good experience.

What we have settled on (fairly recently) is that all you have to do as a developer is have Ollama installed and Quarkus takes care of installing the configured model

[emmanuelbernard]

watch for .n2 firectories though ;P

[geoand]

watch for .n2 firectories though ;P

I checked those this morning :)

[geoand]

Running your reproducer now.

The first comment is that Ollama does have to be running.

Second, I can indeed reproduce the case where the embedding model is not pulled, I'll have a closer look.

[geoand]

The fix is: quarkiverse/quarkus-langchain4j#773

You can try it by building the project locally (mvn clean install -DskipTests) and using version 999-SNAPSHOT

[emmanuelbernard]

🙏