Extending the Docker Image (Galaxy. 24.01, 24.02) does not work.

[jayadevjoshi]

Issue:
I'm trying to create a Docker image with custom tools from the Tool Shed, but it's not working. It throws the following error during container startup:

Skipping Galaxy client build because git is not in use and the client build state cannot be compared against local changes. If you have made local modifications, then manual client builds will be required. See ./client/README.md for more information.
Activating virtualenv at /galaxy_venv
Error: <class 'gravity.state.GalaxyTUSDService'> init failed: To run tusd you need to set galaxy_infrastructure_url in the galaxy section of galaxy.yml
Executing: galaxy
Error: <class 'gravity.state.GalaxyTUSDService'> init failed: To run tusd you need to set galaxy_infrastructure_url in the galaxy section of galaxy.yml
cat: galaxy_install.pid: No such file or directory
calling galaxy_wait with timeout=600 ensure_admin=False
[00] Galaxy not up yet... HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /api/version (Caused
[01] Galaxy not up yet... HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /api/version (Caused
[02] Galaxy not up yet... HTTPConnectionPool(host='localhost', port=8080): Max retries exceeded with url: /api/version (Caused`

To resolve this, I created and added a custom gravity.yml to disable tusd temporary, which did fix the initial issue.

However, Galaxy gets stuck and does not move on to install the tools using the next command. To work around this, I created a custom install-tools script and modified the ./run.sh -d $install_log --pidfile galaxy_install.pid --http-timeout 3000 --daemon, added --daemon flag at the end to push the process in the background, followed by 'sleep 200':

./run.sh -d $install_log --pidfile galaxy_install.pid --http-timeout 3000 --daemon
sleep 200

....

shed-tools install -g "http://localhost:$PORT" -a fakekey -t "$1"

This appears to install the tools successfully (during the docker build) — the command runs and exits with no errors. However, when I later run the container, the tools do not appear in the Galaxy menu.

I notice that when you inspect the docker image using the command bellow it shows the /galaxy/database/shed_tools directory and installed tools inside this directory.

docker run --it galaxy_docker_image /bin/bash

However, when I run the galaxy instance using the command bellow:

docker run -i -t -p 8089:80 galaxy_docker_1:latest

it does not show the installed tools in the tool many also while you inspect the running container

docker exec -it container_id /bin/bash

It does not show the /galaxy/database/shed_tools directory, rather it was disappeared.

I suspect that some configuration settings are overriding the /galaxy/database directory, which might be causing this issue.

modified install-tools command: (changed port, added "--daemon" to ./run.sh ... )

#!/bin/bash

# Enable Test Tool Shed
echo "Enable installation from the Test Tool Shed."
export GALAXY_CONFIG_TOOL_SHEDS_CONFIG_FILE=$GALAXY_HOME/tool_sheds_conf.xml

. /tool_deps/_conda/etc/profile.d/conda.sh
conda activate base

if pgrep "supervisord" > /dev/null
then
    echo "System is up and running. Starting with the installation."
    export PORT=80
else
    # start Galaxy
    export PORT=4001
    service postgresql start
    install_log='galaxy_install.log'

    # wait for database to finish starting up
    STATUS=$(psql 2>&1)
    while [[ ${STATUS} =~ "starting up" ]]
    do
      echo "waiting for database: $STATUS"
      STATUS=$(psql 2>&1)
      sleep 1
    done

    echo "starting Galaxy"
    # Unset SUDO_* vars otherwise conda run chown based on that
    sudo -E -H -u galaxy -- bash -c "unset SUDO_UID; \
        unset SUDO_GID; \
        unset SUDO_COMMAND; \
        unset SUDO_USER; \
        ./run.sh -d $install_log --pidfile galaxy_install.pid --http-timeout 3000 --daemon"

    sleep 200

    galaxy_install_pid=`cat galaxy_install.pid`
    # galaxy-wait -g http://localhost:$PORT -v --timeout 600
fi

# Create the admin user if not already done
# Starting with 20.05 this user is only created at first startup of galaxy
# We need to create it here for Galaxy Flavors = installing from Dockerfile
if [[ ! -z $GALAXY_DEFAULT_ADMIN_USER ]]
    then
        (
        cd $GALAXY_ROOT_DIR
        . $GALAXY_VIRTUAL_ENV/bin/activate
        echo "Creating admin user $GALAXY_DEFAULT_ADMIN_USER with key $GALAXY_DEFAULT_ADMIN_KEY and password $GALAXY_DEFAULT_ADMIN_PASSWORD if not existing"
        python /usr/local/bin/create_galaxy_user.py --user "$GALAXY_DEFAULT_ADMIN_EMAIL" --password "$GALAXY_DEFAULT_ADMIN_PASSWORD" \
        -c "$GALAXY_CONFIG_FILE" --username "$GALAXY_DEFAULT_ADMIN_USER" --key "$GALAXY_DEFAULT_ADMIN_KEY"
        )
fi

shed-tools install -g "http://localhost:$PORT" -a fakekey -t "$1"

exit_code=$?

if [ $exit_code != 0 ] ; then
    if [ "$2" == "-v" ] ; then
        echo "Installation failed, Galaxy server log:"
        cat $install_log
    fi
    exit $exit_code
fi

if ! pgrep "supervisord" > /dev/null
then
    # stop everything
    sudo -E -H -u galaxy ./run.sh --stop --pidfile galaxy_install.pid
    rm $install_log
    service postgresql stop
fi

The Docker build completed successfully without any errors. According to the build log, the tool installation also proceeded without any issues. However, when I run the Galaxy instance, the installed tools are not visible, and the galaxy/database/shed_tools directory is missing—despite the shed-tools install command completing without errors.

Thank you!
Jay