docker-cicd-python

A specific Python container for building Python with FastAPI. You may want to use an official container unless you have similar needs and tool stacks.

Toolchain

• GitHub CLI
• Python
• Poetry
• Dumb Init

Base container / toolchain info

This container is based off of: https://hub.docker.com/_/python and also adds poetry.

List of install resources used:

• https://github.com/cli/cli/blob/trunk/docs/install_linux.md#debian-ubuntu-linux-raspberry-pi-os-apt
• https://github.com/Yelp/dumb-init?tab=readme-ov-file#option-1-installing-from-your-distros-package-repositories-debian-ubuntu-etc

Updating

1. Edit ./VERSION to the new tag you want. Ideally matching the python version. If the only difference is another tool adding -<tool> works.
2. Make the changes to ./Dokcerfile starting with the base container version for python version changes. Now is a good time to see if the linux version can be bumped up.
3. If versions of other tools that are tested change, follow Generating the Test App to update the test app.
4. It's also a good time to check for any other dependcency upgrades.
5. Run ./build.sh to build the container
6. Run ./test.sh to test the container
7. Run ./run.sh to interactively run and inspect the container if needed.
8. Open a PR with the changes

Release

After testing and merging the PR, run the following:

1. git checkout main
2. git fetch --all
3. git pull --rebase upstream main
4. Run ./release.sh to create a release

Local Testing

To use the build scripts, use docker buildx with multiplatform.

Create a builder

docker buildx create \
  --name container \
  --driver=docker-container

Setup a local registry

./local_registry.sh

Run the scripts

To use port 5001, you will need to edit the insecure registires in your docker daemon or use a tool like ngrok to proxy port 5001. If using ngrok, the url will be the one generated by ngrok.

docker:
  insecure-registries:
    - host.docker.internal:5001

Use: localhost or host.docker.internal

Run docker info to see the config. Example output insecure registries section:

 Insecure Registries:
  host.docker.internal:5001
  127.0.0.0/8

Build

# ./build.sh <platforms> <localdomain:port>
# ./build.sh linux/arm64 host.docker.internal:5001 # to pass in a local domain, edit hostsfile
./build.sh linux/arm64 localhost:5001 # insecure registry

Run

# ./run.sh <localdomain:port>
# ./run.sh host.docker.internal:5001
./run.sh localhost:5001

Test

# ./build.sh <localdomain:port>
# ./test.sh host.docker.internal:5001
./test.sh localhost:5001

Generating the Test App

This tests FastAPI and NX, see https://betterprogramming.pub/poetry-python-nx-monorepo-5750d8627024 for a guide. For updates it may be cleaner to start over.

There is a script called generate-test-nx.sh that will create the test project.

Updates

Update the root nx.json. Make sure to update the version to match the output from the base Dockerfile and have a compatible version string. i.e. docker run -it python:3.12-slim-bookworm python --version will output 3.12.8 today.

The script will execute something like the following base steps based on https://betterprogramming.pub/poetry-python-nx-monorepo-5750d8627024. If there are changes update the script. Use the script vs. manual setup, this is just documentation.

npx create-nx-workspace test-nx --preset=npm --nxCloud=skip
cp ./Dockerfile.test-nx ./test-nx/Dockerfile
cp ./test-app/poetry.toml ./test-nx/poetry.toml

cd test-nx
npm install @nxlv/python --save-dev
cp ../nx.json ./nx.json

npx nx generate @nxlv/python:poetry-project proj1 \
--projectType application \
--description='My Project 1' \
--directory packages/proj1

npx nx run-many --target install -- --group=dev
npx nx run-many --target build
npx nx run-many --target test

NOTE: In your CI you would use affected instead of run-many. When testing a new project it's a good idea to test all once.