Merge branch 'main' into upgradable-bypass

Author: robert-cronin

.github/copilot-instructions.md

@@ -9,12 +9,13 @@ Copacetic (Copa) is a CLI tool that patches container image vulnerabilities usin
 ## Folder Structure
 - `pkg/patch/`: CLI commands and core patching logic
 - `pkg/buildkit/`: BuildKit integration and platform discovery
-- `pkg/pkgmgr/`: Package manager adapters (dpkg, rpm, apk)
+- `pkg/pkgmgr/`: OS package manager adapters (dpkg, rpm, apk)
+- `pkg/langmgr/`: Language (application/library) package managers (e.g. Python/pip)
 - `pkg/report/`: Vulnerability report parsing and scanner plugin interface
 - `pkg/imageloader/`: Container engine integration (Docker, Podman)
 - `pkg/types/`: Type definitions and configurations
 - `website/docs/`: Project documentation and user guides
-- `integration/`: Integration tests for multi-arch and single-arch scenarios
+- `integration/`: Integration tests for multi-arch and single-arch scenarios (OS + language flows)
 - `main.go`: Root CLI setup with Cobra framework
 
 ## Libraries and Frameworks
@@ -36,19 +37,63 @@ Copacetic (Copa) is a CLI tool that patches container image vulnerabilities usin
 ## Key Architecture Concepts
 - **Patching modes**: Targeted (with vulnerability reports) or comprehensive (all available updates)
 - **Multi-platform support**: Handles amd64, arm64, and other architectures with QEMU emulation
-- **Package managers**: Debian (apt/dpkg), RHEL (yum/rpm), Alpine (apk), Azure Linux (tdnf)
+- **OS package managers**: Debian (apt/dpkg), RHEL family (yum/rpm/dnf/microdnf/tdnf), Alpine (apk), Azure Linux / CBL-Mariner (tdnf)
+- **Language (application/library) patching**: Experimental support for upgrading vulnerable application dependencies (currently Python via pip) with semantic version / patch-level controls.
 - **BuildKit integration**: Uses LLB operations for image building and manipulation
 - **Scanner plugins**: Supports custom vulnerability scanners via `customParseScanReport` interface
+- **Selective package types**: User can choose to patch only OS, only library, or both via `--pkg-types`.
 
-## Supported Operating Systems & Package Managers
-- **Debian/Ubuntu**: Uses `dpkg` and `apt`
-- **RHEL/CentOS/Rocky/Alma/Oracle/Amazon**: Uses `rpm`, `yum`, and `dnf`
-- **Alpine**: Uses `apk`
-- **CBL-Mariner/Azure Linux**: Uses `rpm` and `tdnf`
+## Supported Targets
+### Operating Systems (OS package layer)
+- **Debian/Ubuntu**: `dpkg` + `apt`
+- **RHEL/CentOS/Rocky/Alma/Oracle/Amazon**: `rpm`, `yum`, `dnf`, `microdnf`, `tdnf` (as available)
+- **Alpine**: `apk`
+- **CBL-Mariner/Azure Linux**: `rpm` + `tdnf`
+
+### Language / Application Dependencies (Experimental)
+- **Python**: pip-based site-packages upgrades with version validation and patch-level selection.
+  - Controlled via `--pkg-types library` (or `os,library`) and `--library-patch-level`.
+  - Patch levels: `patch` (default), `minor`, `major`; influences chosen fixed version when multiple are available.
+  - Special per-package overrides supported (see `getSpecialPackagePatchLevels()` inside Python manager for curated exceptions).
 
 ## Key Functions
+
+### CLI / Orchestration
 - `Patch()`: Main entry point for patching operations
 - `patchSingleArchImage()` / `patchMultiPlatformImage()`: Core patching logic
 - `DiscoverPlatformsFromReference()` / `DiscoverPlatformsFromReport()`: Platform discovery
-- `InstallUpdates()`: Package manager interface for applying updates
 - `InitializeBuildkitConfig()`: Initializes BuildKit configuration for patching operations
+
+### OS Package Layer
+- `pkgmgr.GetPackageManager()` and concrete managers' `InstallUpdates()` methods
+- `GetUniqueLatestUpdates()` (OS) for deduplicating + selecting latest OS package versions
+
+### Language / Library Layer
+- `langmgr.GetLanguageManagers()` returns appropriate language managers based on manifest content
+- `pythonManager.InstallUpdates()` coordinates: selecting versions, performing pip upgrades, validating results
+- `langmgr.GetUniqueLatestUpdates()` (libraries) similar to OS but tolerant of empty sets & patch-level filtering
+- `FindOptimalFixedVersionWithPatchLevel()` (Trivy parsing path) chooses best fixed version under patch-level constraint
+
+### Report Parsing & Filtering
+- `report.TryParseScanReport()` / Trivy parser: builds unified UpdateManifest (OS + Lang)
+- Filtering by `--pkg-types` occurs early and again before build execution for safety.
+
+### Validation & VEX
+- `vex.TryOutputVexDocument()` generates optional VEX documents (only if report + updates applied)
+
+## Language Patching
+Language/library patching is gated behind `COPA_EXPERIMENTAL=1`.
+
+- `--pkg-types`: comma list of `os`, `library` (default `os`). Determines which sections of the UpdateManifest are acted upon.
+- `--library-patch-level`: one of `patch|minor|major` (default `patch`). Sets semantic version boundary for chosen upgrade version.
+- Behavior when report only has library vulns:
+  - If `--pkg-types` includes `library`, proceed (even if no OS updates). Empty OS set no longer triggers an error.
+  - If `--pkg-types os` only, library updates are ignored (manifest language section cleared early).
+
+## Library Patching Flow (Python)
+1. Trivy report parsed -> vulnerable Python packages aggregated with all candidate fixed versions.
+2. Patch level rule applied to select optimal fixed version per package (with per-package override map for exceptions).
+3. Upgrade executed in ephemeral tooling container (derives base Python image tag when possible; fallback tag `3-slim`).
+4. Post-upgrade validation via `pip freeze` subset matching: ensures requested versions actually installed.
+5. Failed installs or mismatches collected; errors either propagate or are logged based on `--ignore-errors`.
+6. Validated updates merged into final manifest fed to VEX generation.

.github/workflows/build.yml

@@ -33,16 +33,16 @@ permissions:
 jobs:
   unit-test:
     name: Unit Test
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-latest-16-cores
     timeout-minutes: 5
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.3.1
+        uses: step-security/harden-runner@f4a75cfd619ee5ce8d5b864b0d183aff3c69b55a # v2.3.1
         with:
           egress-policy: audit
       - name: Check out code
         uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
-      - uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
+      - uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
         with:
           go-version: "1.24"
           check-latest: true
@@ -56,26 +56,23 @@ jobs:
           CODECOV_OPTS: "-coverprofile=coverage.txt -covermode=atomic"
         run: make test
       - name: Upload coverage to Codecov
-        uses: codecov/codecov-action@fdcc8476540edceab3de004e990f80d881c6cc00 # v5.5.0
+        uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1
         env:
           CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
   build:
     name: Build
-    runs-on: ${{ matrix.os }}
+    runs-on: ubuntu-latest-16-cores
     timeout-minutes: 5
     permissions:
       packages: write
       contents: read
-    strategy:
-      matrix:
-        os: [ubuntu-latest]
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.3.1
+        uses: step-security/harden-runner@f4a75cfd619ee5ce8d5b864b0d183aff3c69b55a # v2.3.1
         with:
           egress-policy: audit
       - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
-      - uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
+      - uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
         with:
           go-version: "1.24"
           check-latest: true
@@ -104,7 +101,7 @@ jobs:
   test-patch-trivy:
     needs: build
     name: Test patch with trivy ${{ matrix.buildkit_mode }}
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-latest-16-cores
     timeout-minutes: 30
     strategy:
       fail-fast: false
@@ -118,7 +115,7 @@ jobs:
       - run: docker system prune -a -f --volumes
       - name: Check out code
         uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
-      - uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
+      - uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
         with:
           go-version: "1.24"
           check-latest: true
@@ -150,7 +147,7 @@ jobs:
   test-patch-no-report:
     needs: build
     name: Test patch no report ${{ matrix.buildkit_mode }}
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-latest-16-cores
     timeout-minutes: 30
     strategy:
       fail-fast: false
@@ -164,7 +161,7 @@ jobs:
       - run: docker system prune -a -f --volumes
       - name: Check out code
         uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
-      - uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
+      - uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
         with:
           go-version: "1.24"
           check-latest: true
@@ -202,11 +199,11 @@ jobs:
             if [[ -n "${COPA_BUILDKIT_ADDR}" && "${COPA_BUILDKIT_ADDR}" == docker://* ]]; then
                 export DOCKER_HOST="${COPA_BUILDKIT_ADDR#docker://}"
             fi
-      
+
             docker create --name test ghcr.io/project-copacetic/copacetic/test/openssl:test-rpm-patched /bin/sh
             tmp="$(mktemp)"
             docker cp test:/etc/pki/tls/openssl.cnf "${tmp}"
-            
+
             if ! grep -q foo "${tmp}"; then
                 echo "Error: openssl.cnf content replaced" >&2
                 rm "${tmp}"
@@ -256,11 +253,11 @@ jobs:
                 export DOCKER_HOST="${COPA_BUILDKIT_ADDR#docker://}"
                 trap '_cleanup' EXIT
             fi
-      
+
             docker create --name test ghcr.io/project-copacetic/copacetic/test/openssl:test-debian-patched /bin/sh
             tmp="$(mktemp)"
             docker cp test:/etc/ssl/openssl.cnf "${tmp}"
-            
+
             if ! grep -q foo "${tmp}"; then
                 echo "Error: openssl.cnf content replaced" >&2
                 rm "${tmp}"
@@ -274,16 +271,16 @@ jobs:
   test-plugin:
     needs: build
     name: Test plugin
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-latest-16-cores
     timeout-minutes: 5
     steps:
       - name: Harden Runner
-        uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.3.1
+        uses: step-security/harden-runner@f4a75cfd619ee5ce8d5b864b0d183aff3c69b55a # v2.3.1
         with:
           egress-policy: audit
       - name: Check out code
         uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
-      - uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
+      - uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
         with:
           go-version: "1.24"
           check-latest: true
@@ -321,12 +318,12 @@ jobs:
   test-push:
     needs: build
     name: Test push
-    runs-on: ubuntu-latest
+    runs-on: ubuntu-latest-16-cores
     timeout-minutes: 5
     steps:
       - name: Check out code
         uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
-      - uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
+      - uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
         with:
           go-version: "1.24"
           check-latest: true
@@ -339,7 +336,7 @@ jobs:
         with:
           install: true
       - name: Install oras CLI
-        uses: oras-project/setup-oras@8d34698a59f5ffe24821f0b48ab62a3de8b64b20 # v1.2.3
+        uses: oras-project/setup-oras@22ce207df3b08e061f537244349aac6ae1d214f6 # v1.2.4
       - name: Install required tools
         shell: bash
         run: .github/workflows/scripts/download-tooling.sh
@@ -364,12 +361,12 @@ jobs:
         with:
           paths: "test-results.xml"
 
-          
+
   test-patch-multiplatform:
     needs: build
     name: Test patch with multiplatform ${{ matrix.buildkit_mode }}
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
+    runs-on: ubuntu-latest-16-cores
+    timeout-minutes: 40
     strategy:
       fail-fast: false
       matrix:
@@ -385,7 +382,7 @@ jobs:
           install: true
           platforms: linux/amd64,linux/arm/v5,linux/arm64,linux/386,linux/mips64le
       - name: Install oras CLI
-        uses: oras-project/setup-oras@8d34698a59f5ffe24821f0b48ab62a3de8b64b20 # v1.2.3
+        uses: oras-project/setup-oras@22ce207df3b08e061f537244349aac6ae1d214f6 # v1.2.4
       - name: Set up local registry
         run: |
           docker run -d -p 5000:5000 --restart=always --name registry registry:2
@@ -409,7 +406,7 @@ jobs:
       - run: docker system prune -a -f --volumes
       - name: Check out code
         uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
-      - uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
+      - uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
         with:
           go-version: "1.24"
           check-latest: true
@@ -434,15 +431,15 @@ jobs:
         shell: bash
         run: |
           set -eu -o pipefail
-          
+
           if [[ "${{ matrix.buildkit_mode }}" == "docker" ]]; then
             # For docker mode, use the default docker daemon
             export COPA_BUILDKIT_ADDR="docker://"
           else
             # For other modes, source the corresponding script in the same shell session
             . .github/workflows/scripts/buildkitenvs/${{ matrix.buildkit_mode }}
           fi
-          
+
           gotestsum --format testname --junitfile test-results.xml -- ./integration/multiarch --addr="${COPA_BUILDKIT_ADDR}" --copa="$(pwd)/copa" -timeout 0
       - name: Test Summary
         uses: test-summary/action@v2
@@ -452,8 +449,8 @@ jobs:
   test-patch-multiplatform-plugin:
     needs: build
     name: Test multiplatform with plugin
-    runs-on: ubuntu-latest
-    timeout-minutes: 30
+    runs-on: ubuntu-latest-16-cores
+    timeout-minutes: 50
     steps:
       - name: Change docker daemon config
         run: |
@@ -466,7 +463,7 @@ jobs:
           install: true
           platforms: linux/amd64,linux/arm64
       - name: Install oras CLI
-        uses: oras-project/setup-oras@8d34698a59f5ffe24821f0b48ab62a3de8b64b20 # v1.2.3
+        uses: oras-project/setup-oras@22ce207df3b08e061f537244349aac6ae1d214f6 # v1.2.4
       - name: Set up local registry
         run: |
           docker run -d -p 5000:5000 --restart=always --name registry registry:2
@@ -485,7 +482,7 @@ jobs:
           docker push localhost:5000/nginx:latest
       - name: Check out code
         uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
-      - uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
+      - uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
         with:
           go-version: "1.24"
           check-latest: true

.github/workflows/check-deps.yml

@@ -18,13 +18,13 @@ jobs:
     timeout-minutes: 10
     steps:
       - name: Harden the runner (Audit all outbound calls)
-        uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.13.0
+        uses: step-security/harden-runner@f4a75cfd619ee5ce8d5b864b0d183aff3c69b55a # v2.13.1
         with:
           egress-policy: audit
 
       - name: Check out code into the Go module directory
         uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
-      - uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
+      - uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
         with:
           go-version: "1.24"
           check-latest: true

.github/workflows/codeql.yml

@@ -44,21 +44,21 @@ jobs:
 
     steps:
     - name: Harden Runner
-      uses: step-security/harden-runner@ec9f2d5744a09debf3a187a3f4f675c53b671911 # v2.3.1
+      uses: step-security/harden-runner@f4a75cfd619ee5ce8d5b864b0d183aff3c69b55a # v2.3.1
       with:
         egress-policy: audit
 
     - name: Checkout repository
       uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
 
-    - uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5.5.0
+    - uses: actions/setup-go@44694675825211faa026b3c33043df3e48a5fa00 # v6.0.0
       with:
         go-version: "1.24"
         check-latest: true
 
     # Initializes the CodeQL tools for scanning.
     - name: Initialize CodeQL
-      uses: github/codeql-action/init@3c3833e0f8c1c83d449a7478aa59c036a9165498 # v3.29.5
+      uses: github/codeql-action/init@192325c86100d080feab897ff886c34abd4c83a3 # v3.29.5
       with:
         languages: ${{ matrix.language }}
         # If you wish to specify custom queries, you can do so here or in a config file.
@@ -72,7 +72,7 @@ jobs:
     # Autobuild attempts to build any compiled languages  (C/C++, C#, Go, or Java).
     # If this step fails, then you should remove it and run the build manually (see below)
     - name: Autobuild
-      uses: github/codeql-action/autobuild@3c3833e0f8c1c83d449a7478aa59c036a9165498 # v3.29.5
+      uses: github/codeql-action/autobuild@192325c86100d080feab897ff886c34abd4c83a3 # v3.29.5
 
     # ℹ️ Command-line programs to run using the OS shell.
     # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
@@ -85,6 +85,6 @@ jobs:
     #     ./location_of_script_within_repo/buildscript.sh
 
     - name: Perform CodeQL Analysis
-      uses: github/codeql-action/analyze@3c3833e0f8c1c83d449a7478aa59c036a9165498 # v3.29.5
+      uses: github/codeql-action/analyze@192325c86100d080feab897ff886c34abd4c83a3 # v3.29.5
       with:
         category: "/language:${{matrix.language}}"