Building from source on Windows, and building the container image

[DinoChiesa]

Hi

I wanted to try running the container image from Google Cloud Run. I saw your comment somewhere in the documentation about how containers free people from the big cloud infrastructure providers. But I've got a different set of criteria. Anyway I'd like to just run it in "Cloud Run". Specifically I am talking about the headless option.

Theoretically this could be pretty easy, as Cloud Run just runs an image. So I can do this:

gcloud run jobs create JOB_NAME `
    --image philosowaffle/peloton-to-garmin:console-stable `
    --tasks 1 `
    --max-retries 1 `
     ...other options....

(that works in Windows PowerShell, where the backtick is the newline continuation character)

But to do that, I need to mount a configuration file, and an output directory.

Cloud Run allows me to mount one or more read-only or read/write storage buckets (via gcsfuse). So that would get me close. But there are couple problems here:

• Cloud Run allows mounting of only directories as volumes. Mounting a single file isn't possible. So this would allow me to mount the output directory, but it would not allow me to provide the configuration file.
• The p2g docker image does not let me specify the paths I want to use for the output directory or the configuration file. It's always /app/output and /app/configuration.local.json as far as I can tell from reading the documentation. I looked in the code and it does appear that if there is an argument given to the program, it uses THAT as the directory to look in, for configuration . But this isn't documented. and also I couldn't get it to work properly.

therefore, a feature request:
allow the image to get its arguments from the environment. So I could specify P2G_CONFIG=/mnt/configdir and it would use the configuration file in that directory. Similarly I could specify P2G_OUTPUT=/mnt/outputdir and

I tried forking and modifying the C# code, and I'm reasonably ok at that. I was able to build it successfully. The next step was to build the Docker image. But I know much less about Docker. There is no documentation describing how to build the image. I tried on windows. I guessed this:

   docker build . -f .\docker\Dockerfile.console -t p2g-test-console

But that failed, because of errors like this:

 /build/src/Peloton/Peloton.csproj : error NU1301: Unable to load the service index for source https://api.nuget.org/v3/index.json. [/build/src/ConsoleClient/ConsoleClient.csproj]

So I tried

   docker build . --network=host -f .\docker\Dockerfile.console -t p2g-test-console

And that seemed to work.

Therefore, a documentation improvement request:
Document how to build an image from the various Dockerfiles.

[DinoChiesa]

Also, when building the docker image from Windows, I had to change a couple things:

• explicitly convert the entrypoint.sh file back to use unix line-endings (newline only). Because of this problem (tl/dr: by default git changes line endings when you pull on Windows)
• modify the Dockerfile.console to use

    RUN mkdir -m770 working
    RUN mkdir -m770 output
    RUN mkdir -m770 data
  

  instead of

    RUN mkdir -m770 {working,output,data}
  

  When I used the latter I got a single directory named {working,output,data} in the created image.

When I changed these things, it worked to run the image in Cloud Run, with something like this:

gcloud run jobs create $env:JOB_NAME `
        --image us-west1-docker.pkg.dev/my-gcp-project/docker-images/p2g-console-modified `
        --tasks 1 `
        --max-retries 1 `
        --region "$env:REGION" `
        --project "$env:PROJECT_ID" `
        --set-env-vars "P2G_CONFIG=/app/mounted-config" `
        --add-volume "name=volume_output,type=cloud-storage,bucket=peloton-to-garmin-output" `
        --add-volume-mount "volume=volume_output,mount-path=/app/output"  `
        --add-volume "name=volume_config,type=cloud-storage,bucket=peloton-to-garmin-config" `
        --add-volume-mount "volume=volume_config,mount-path=/app/mounted-config" `

And then I could set that as a scheduled task, to run every 15-20 minutes or whatever.

[DinoChiesa]

Using my modified console program, and the image built with the modified Dockerfile.console, I can set that image to run as a Cloud Run job, following these instructions:
https://gist.github.com/DinoChiesa/d42687fa2ffd450fd995abcbdbe851fb

Thank you very much for building this cool tool.

[geoffbon]

I'd also love to be able to specify the location of the config file; I'm running on Kubernetes, which makes it hard to inject a single file into an existing directory (from a config map).
An environment variable would be perfect! Right now I have to a have a real file on disk somewhere.

[philosowaffle]

Created #698 to change how config files are mounted to the container. Slotted for v5 as I think this will likely end up being a non-backwards compatible change.

[DinoChiesa]

Did you see my PR? I think it is backwards compatible. #680

[DinoChiesa]

Bailey, what about the other stuff I mentioned above?

• document how to build the docker containers
• update the Dockerfiles to allow them to work on Windows (See fix: modify Dockerfile.console so that it works properly on windows #679)