Skip to content

Conversation

@WashingtonKK
Copy link

What type of PR is this?

What does this do?

Which issue(s) does this PR fix/relate to?

Have you included tests for your changes?

Did you document any new/modified features?

Notes

docker/.env Outdated
Comment on lines 210 to 211
OLLAMA_BASE_URL=http://ollama:11434
OLLAMA_DEFAULT_MODEL=tinyllama:1.1b
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ui does not connect to llm directly, it should go through proxy, this fails to include auth and is wrong for cube

Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
Signed-off-by: WashingtonKK <washingtonkigan@gmail.com>
Comment on lines +215 to +216
OLLAMA_BASE_URL=http://cube-proxy:8900
OLLAMA_DEFAULT_MODEL=tinyllama:1.1b
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename these variables, we support both ollama and vllm, see make file. llm target for ui should just be proxy url, which you can name, also build this from existing variables

Comment on lines +215 to +216
OLLAMA_BASE_URL=http://cube-proxy:8900
OLLAMA_DEFAULT_MODEL=tinyllama:1.1b
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename these variables, we support both ollama and vllm, see make file. llm target for ui should just be proxy url, which you can name, also build this from existing variables

# capabilities:
# - gpu

open-webui:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

check vllm file as well

Comment on lines +294 to +295
OLLAMA_BASE_URL: ${OLLAMA_BASE_URL}
OLLAMA_DEFAULT_MODEL: ${OLLAMA_DEFAULT_MODEL}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use more appropriate variables

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants