ModuleNotFoundError: No module named 'python_on_whales' When Launching from WITHIN Ooba Docker Container

[Poisonsting]

Hi there, I'm running into an issue when I try to run this extension in an instance of Textgen Web UI that is itself inside docker.

text-generation-webui-text-generation-webui-1  | 19:53:51-422892 ERROR    Failed to load the extension "Memoir".                 
text-generation-webui-text-generation-webui-1  | Traceback (most recent call last):
text-generation-webui-text-generation-webui-1  |   File "/home/app/text-generation-webui/modules/extensions.py", line 37, in load_extensions
text-generation-webui-text-generation-webui-1  |     extension = importlib.import_module(f"extensions.{name}.script")
text-generation-webui-text-generation-webui-1  |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
text-generation-webui-text-generation-webui-1  |   File "/home/app/text-generation-webui/installer_files/env/lib/python3.11/importlib/__init__.py", line 126, in import_module
text-generation-webui-text-generation-webui-1  |     return _bootstrap._gcd_import(name[level:], package, level)
text-generation-webui-text-generation-webui-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
text-generation-webui-text-generation-webui-1  |   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
text-generation-webui-text-generation-webui-1  |   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
text-generation-webui-text-generation-webui-1  |   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
text-generation-webui-text-generation-webui-1  |   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
text-generation-webui-text-generation-webui-1  |   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
text-generation-webui-text-generation-webui-1  |   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
text-generation-webui-text-generation-webui-1  |   File "/home/app/text-generation-webui/extensions/Memoir/script.py", line 23, in <module>
text-generation-webui-text-generation-webui-1  |     from python_on_whales import DockerClient
text-generation-webui-text-generation-webui-1  | ModuleNotFoundError: No module named 'python_on_whales'

As you can see, I made sure to manually install all the reqs in my dockerfile:

# BUILDER
FROM ubuntu:22.04
WORKDIR /builder
ARG TORCH_CUDA_ARCH_LIST="${TORCH_CUDA_ARCH_LIST:-3.5;5.0;6.0;6.1;7.0;7.5;8.0;8.6+PTX}"
ARG BUILD_EXTENSIONS="${BUILD_EXTENSIONS:-}"
ARG APP_UID="${APP_UID:-6972}"
ARG APP_GID="${APP_GID:-6972}"

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked,rw \
    apt update && \
    apt install --no-install-recommends -y git vim build-essential python3-dev pip bash curl && \
    rm -rf /var/lib/apt/lists/*
WORKDIR /home/app/
RUN git clone https://github.com/oobabooga/text-generation-webui.git 
WORKDIR /home/app/text-generation-webui
RUN GPU_CHOICE=A USE_CUDA118=FALSE LAUNCH_AFTER_INSTALL=FALSE INSTALL_EXTENSIONS=TRUE ./start_linux.sh --verbose
COPY CMD_FLAGS.txt /home/app/text-generation-webui/
RUN pip install python_on_whales textblob validators qdrant_client sentence_transformers
EXPOSE ${CONTAINER_PORT:-7860} ${CONTAINER_API_PORT:-5000} ${CONTAINER_API_STREAM_PORT:-5005}
WORKDIR /home/app/text-generation-webui
# set umask to ensure group read / write at runtime
CMD umask 0002 && export HOME=/home/app/text-generation-webui && ./start_linux.sh --listen

Any ideas?

[Poisonsting]

I managed to get around the error above by making the following changes to my Dockerfile:

RUN . ./installer_files/conda/etc/profile.d/conda.sh && \
    conda activate ./installer_files/env && \
    pip install python_on_whales textblob validators qdrant_client sentence_transformers

But am now getting a new error:

text-generation-webui-text-generation-webui-1  | 20:14:28-162645 INFO     Loading the extension "Memoir"                         
text-generation-webui-text-generation-webui-1  | /home/app/text-generation-webui/installer_files/env/lib/python3.11/site-packages/python_on_whales/client_config.py:85: UserWarning: The docker client binary file was not found on your system. 
text-generation-webui-text-generation-webui-1  | Docker on whales will try to download it for you. 
text-generation-webui-text-generation-webui-1  | Don't worry, it won't be in the PATH and won't have anything to do with the package manager of your system. 
text-generation-webui-text-generation-webui-1  | Note: We are not installing the docker daemon, which is a lot heavier and harder to install. We're just downloading a single standalone binary file.
text-generation-webui-text-generation-webui-1  | If you want to trigger the download of the client binary file manually (for example if you want to do it in a Dockerfile), you can run the following command:
text-generation-webui-text-generation-webui-1  |  $ python-on-whales download-cli 
text-generation-webui-text-generation-webui-1  | 
text-generation-webui-text-generation-webui-1  |   warnings.warn(
100% 69.2M/69.2M [00:00<00:00, 118MiB/s] ui-1  | 
text-generation-webui-text-generation-webui-1  | : Error /home/app/text-generation-webui/extensions/Memoir/qdrant-docker-compose.yml: [Errno 13] Permission denied: '/home/app/text-generation-webui/.cache'

The permissions for the the folder in question are 775 for the user and group the Docker image is running as.

[brucepro]

You can try running one of the standalone versions of qdrant and commenting out the docker-compose load in startup of script.py. https://qdrant.tech/documentation/guides/installation/

[v1ckxy]

I added some checks to control docker usage in the extension and checking if the qdrant server is up and running.
To be honest I think It's better to have a qdrant server running standalone (vm or docker)

@Poisonsting you cannot run a docker composer file from within a docker file, matroskalike :D

[Poisonsting]

You totally can, actually.

https://devopscube.com/run-docker-in-docker/

[brucepro]

I recommend you just comment out all the startup() in script, then just run the qdrant locally or in another docker, or in the same docker for that matter.

[v1ckxy]

You totally can, actually.

https://devopscube.com/run-docker-in-docker/

Playing around with sockets and reducing Docker's security even further.

It's easier to just raise a standalone qdrant db server, as the setup is as simple as downloading a binary and running it.
I'm planning to push some scripts regarding this on a repo soon.

[Poisonsting]

My comment wasn’t to say that D-in-D was a good idea, merely that confidently stating it’s impossible is not true.

Given that I only ever launch Ooba in Docker on Nvidia hardware, and most folks that I speak with prefer to do so as well, it would be awesome to have a supported method to run this when launching that way.

[brucepro]

Will do a release the runs the exec of qdrant or neo4j soon without the need for docker. Just because I like docker, guess everyone would rather easy install script.

[kitcat121]

this is also happening to me. I'm not running Ubuntu but I'm running Garuda Linux. I'd like to think that I can do everything on Linux but I don't, I do not know how to work with docker

[brucepro]

I migrated most of the current code away from using python_on_whales and starting the docker up for the user. You can comment the startup() info out. Then make sure you have qdrant running on localhost:6333 or wherever you decide to run it. You can find more information on running qdrant here: https://github.com/qdrant/qdrant/releases