Is it possible to append to the Ollama Dockerfile or run an OS level command on first run?

[stevejackson-daviesgroup]

Hi all!

I've been having a bit of a play with Ollama as part of an initial investigation into a RAG based "helper bot" for lack of a better term - for the most part I have been able to get everything working based on a few examples dotted around, and I can successfully ingest documents, ask questions, and get responses that are mostly what I would expect...not too bad for a laptop with an integrated NVidia GPU (though it is a tad slow, this is to be expected). However I have run into a problem that has, so far, been impossible to sort out - or at least impossible from the perspective of being automated.

Our corporate network uses ForcePoint as a MITM proxy for all network traffic - as a result it requires the installation of the ForcePoint root CA files. I have been able to inject the required files into the Docker image as part of the .AddOllama() chain - however in order for those files to be imported into the resulting Linux OS the command "update-ca-certificates" must be executed.

If I manually create a persistent Docker container (Docker desktop CLI), add the files, then run the command I can run my Aspire AppHost and it will then download the requested models correctly - without the root CA's installed it gets an error about the certificate not being trusted (again, this is understandable). The issue with this is that I don't want to have to do this manually each time, and if I forget to manually start the Ollama container Aspire explodes with an error as the container name is already in use (or if it does get past that, it resets the OS and the root CA's are no longer registered).

Is there a way to append to the Dockerfile that is downloaded from docker.io? I can see there is a WithDockerfile() method as part of the ContainerResourceBuilderExtensions - but I've not found a working example, and anything I try to put in said method results in an error that just says File Not Found (no path details etc, so not helpful at all).

If that is not possible are we able to force a bash command to be executed inside of the container? This would result in the container always failing to download/validate the model on first execution - but restarting the Ollama resource should resolve this (as far as I can tell).

I really do need to get this automated as the resulting project(s) that come out of this will need to be run on many different laptops, and it will need to work "out of the box". Sadly a dedicated Ollama external resource (Server/VM/Container) is currently out of the question.

This is the code that I am currently using to setup the Ollama container and request the required models.

IDistributedApplicationBuilder builder = DistributedApplication.CreateBuilder(args);

IResourceBuilder<OllamaResource> ollama = builder.AddOllama("ollama")
    .WithContainerFiles("/usr/local/share/ca-certificates", [
        new ContainerFile
        {
            Name = "Forcepoint Cloud CA.crt",
            Contents = GetFileContent("support-files\\Forcepoint Cloud CA.crt"),
            Mode = UnixFileMode.UserRead | UnixFileMode.GroupRead | UnixFileMode.OtherRead
        },
        new ContainerFile
        {
            Name = "Forcepoint Cloud OPS CA.crt",
            Contents = GetFileContent("support-files\\Forcepoint Cloud OPS CA.crt"),
            Mode = UnixFileMode.UserRead | UnixFileMode.GroupRead | UnixFileMode.OtherRead
        },
        new ContainerFile
        {
            Name = "Forcepoint Edge Firewalls Global.crt",
            Contents = GetFileContent("support-files\\Forcepoint Edge Firewalls Global.crt"),
            Mode = UnixFileMode.UserRead | UnixFileMode.GroupRead | UnixFileMode.OtherRead
        }
    ])
    .WithDataVolume()
    .WithGPUSupport();

IResourceBuilder<OllamaModelResource> chat = ollama.AddModel("chat", "mistral");
IResourceBuilder<OllamaModelResource> embedding = ollama.AddModel("embedding", "nomic-embed-text");

And this is the custom command that I placed into a Dockerfile - the file may not be right, I may need to add extra steps? But the file right now does indeed just have this single line.

RUN update-ca-certificates

I would be most appreciative of any guidance that can be provided - it's so frustrating being so close yet so far from my goal.

[aaronpowell]

The best approach would probably be to create your own docker image that uses Ollama as a base and then does the additional steps needed by your company policies, then do builder.AddOllama("ollama').WithImage(...).

Alternatively, you could provide args to the container start which overrides the entry point so that you run your own script (that you would side-load like you're doing with the crt files), and have that new entry point run the update-ca-certificates method then start Ollama as normal in the container.