proxmox-exporter can't be parametrized with docker secrets

[graysievert]

I tried to populate PROXMOX_EXPORTER_PROXMOX_TOKEN and PROXMOX_EXPORTER_PROXMOX_TOKEN_ID using docker secrets but failed to do so.

I expected to use the trick I saw in Authentik: set environment variable with file:///run/secrets/SOMESECRET.
That way, when processed in go code variable is set with the contents of file /run/secrets/SOMESECRET. Unfortunately, that did not work.
I'm not a GO expert in any approximation, but hopefully something similar to this https://github.com/goauthentik/authentik/blob/a6225ad7a7880a5e1e63225e81084e27c35fe626/internal/config/struct.go could be implementd for exporter?

Here's my log

#####################
## This works
#####################
$ cat -> compose.yaml
services:
  exporter:
    image: ghcr.io/starttoaster/proxmox-exporter:latest

    ports:
      - 8080:8080
    volumes:
      - /etc/pki/ca-trust/source/anchors/localCA.crt:/etc/ssl/certs/localCA.crt:ro # local CA certificate
    environment:
      PROXMOX_EXPORTER_LOG_LEVEL: 'debug'
      PROXMOX_EXPORTER_PROXMOX_API_INSECURE: 'false'
      PROXMOX_EXPORTER_PROXMOX_ENDPOINTS: 'https://pve.lan:8006/'
      PROXMOX_EXPORTER_PROXMOX_TOKEN: '5f8ebe64-e70d-4840-aef4-7cac331dddd2'
      PROXMOX_EXPORTER_PROXMOX_TOKEN_ID: 'proxmox-exporter@pve!proxmox-exporter'
      PROXMOX_EXPORTER_SERVER_PORT: '8080'
      # PROXMOX_EXPORTER_PROXMOX_TOKEN: file:///run/secrets/PROXMOX_EXPORTER_PROXMOX_TOKEN
      # PROXMOX_EXPORTER_PROXMOX_TOKEN_ID: file:///run/secrets/PROXMOX_EXPORTER_PROXMOX_TOKEN_ID
    # secrets:
    #   - PROXMOX_EXPORTER_PROXMOX_TOKEN
    #   - PROXMOX_EXPORTER_PROXMOX_TOKEN_ID

# secrets:
#   PROXMOX_EXPORTER_PROXMOX_TOKEN:
#     file: ./token.txt
#   PROXMOX_EXPORTER_PROXMOX_TOKEN_ID:
#     file: ./id.txt


$ docker compose up
Attaching to exporter-1
exporter-1  | time=2024-08-23T15:56:56.044Z level=DEBUG msg="Creating Proxmox client" endpoint=https://pve.lan:8006/ hostname=pve.lan
exporter-1  | time=2024-08-23T15:56:56.059Z level=INFO msg="Starting server" port=8080
exporter-1  | time=2024-08-23T15:57:28.154Z level=DEBUG msg="finished requests for node data" node=pve
exporter-1  | time=2024-08-23T15:57:46.808Z level=DEBUG msg="proxmox request was found in cache for GetNodes"
exporter-1  | time=2024-08-23T15:57:46.808Z level=DEBUG msg="proxmox request was found in cache for GetNodeQemu" node=pve
exporter-1  | time=2024-08-23T15:57:46.808Z level=DEBUG msg="proxmox request was found in cache for GetNodeLxc" node=pve
exporter-1  | time=2024-08-23T15:57:46.808Z level=DEBUG msg="proxmox request was found in cache for GetNodeStorage" node=pve
exporter-1  | time=2024-08-23T15:57:46.809Z level=DEBUG msg="proxmox request was found in cache for GetNodeDisksList" node=pve

#####################
## This failed
#####################

$ cat -> token.txt
5f8ebe64-e70d-4840-aef4-7cac331dddd2

$ cat -> id.txt
proxmox-exporter@pve!proxmox-exporter

$ cat -> compose.yaml
services:
  exporter:
    image: ghcr.io/starttoaster/proxmox-exporter:latest

    ports:
      - 8080:8080
    volumes:
      - /etc/pki/ca-trust/source/anchors/localCA.crt:/etc/ssl/certs/localCA.crt:ro # local CA certificate
    environment:
      PROXMOX_EXPORTER_LOG_LEVEL: 'debug'
      PROXMOX_EXPORTER_PROXMOX_API_INSECURE: 'false'
      PROXMOX_EXPORTER_PROXMOX_ENDPOINTS: 'https://pve.lan:8006/'
      # PROXMOX_EXPORTER_PROXMOX_TOKEN: '5f8ebe64-e70d-4840-aef4-7cac331dddd2'
      # PROXMOX_EXPORTER_PROXMOX_TOKEN_ID: 'proxmox-exporter@pve!proxmox-exporter'
      PROXMOX_EXPORTER_SERVER_PORT: '8080'
      PROXMOX_EXPORTER_PROXMOX_TOKEN: file:///run/secrets/PROXMOX_EXPORTER_PROXMOX_TOKEN
      PROXMOX_EXPORTER_PROXMOX_TOKEN_ID: file:///run/secrets/PROXMOX_EXPORTER_PROXMOX_TOKEN_ID
    secrets:
      - PROXMOX_EXPORTER_PROXMOX_TOKEN
      - PROXMOX_EXPORTER_PROXMOX_TOKEN_ID

secrets:
  PROXMOX_EXPORTER_PROXMOX_TOKEN:
    file: ./token.txt
  PROXMOX_EXPORTER_PROXMOX_TOKEN_ID:
    file: ./id.txt

$ docker compose up
Attaching to exporter-1
exporter-1  | time=2024-08-23T16:01:00.184Z level=DEBUG msg="Creating Proxmox client" endpoint=https://pve.lan:8006/ hostname=pve.lan
exporter-1  | time=2024-08-23T16:01:03.199Z level=DEBUG msg="banning client" name=pve.lan duration=1m0s
exporter-1  | time=2024-08-23T16:01:03.199Z level=INFO msg="Starting server" port=8080
exporter-1  | time=2024-08-23T16:01:09.487Z level=ERROR msg="request to get node list was not successful. It's possible all clients are banned"

[Starttoaster]

I'm not all that familiar with Docker Secrets, as I mostly use k8s, where their Secrets can be mounted into the container as environment variables. From what I can tell, Docker Secrets can only be mounted as files, with an ability to specify the path to the file in an environment variable. I can see where this wouldn't support that, as the code isn't written to take in files containing those values.

Can you help me with understanding the value of Docker's version of "secrets"? They seem to just be a file on the host, versus an environment variable supplied to the container? And I guess the main security win for using them would be if you had a multi-tenant Docker host with multiple teams or otherwise unrelated people running Docker commands on that host from different linux users. And even at that point, if a person has the access to run docker commands, they usually have some form of root/sudo-ing privileges to interact with the docker daemon... Is that your use case? Seems fairly niche, and I'm not sure there's much value in Docker's version of "secrets" personally. Especially if it's not a multi-tenant docker host anyway. It just moves the secret from the docker-compose yaml file to different text file 🤔

[Starttoaster]

And Docker Secrets seem even less useful for the multi-tenant docker host situation... The container for proxmox-exporter doesn't have a shell and all the binary utilities that tend to come alongside a linux shell. But in other containers, like nginx:latest for example, if I have permission to run docker commands, I can just run docker exec -it <container_name> cat /run/secrets/SECRET_NAME to exfiltrate your secret, assuming I didn't have permission to just read the file from the host OS. So now I'm not really sure what Docker Secrets are really meant to solve for..

Maybe some kind of situation where the host OS is multi-tenant but only my linux user has permission to run docker CLI commands? But that doesn't sound like a reasonable situation that would ever really occur. Let me know what your thoughts are on this, I'm leaning to not supporting Docker's Secrets because they seem a bit like security theater anyway.

[graysievert]

My apologies for wording it poorly from the start.

My question is whether it is possible to add an option to read at least the token from an arbitrary file on the filesystem.

That will allow the utilization of secret mechanics in both Kubernetes and Docker Swarm.

I think the secret mechanics allow for the very effective decoupling of secret material from configuration which helps to avoid leaking sensitive data into the codebase.

You are completely right that with sufficient access permissions on the host, a secret is not a secret regardless of whether it is in a file on tempfs or in the env variable: docker inspect <container_name> | jq '.[-1].Config.Env'

But again, the purpose of secrets is to allow people to share the working configuration in a public repo and be sure that secret material was not published.

I'm aware of several ways how other projects usually achieve that:

1. Command line option. One can mount a secret to the container and then mention that mountpath in 'command:'
2. Check for environment variables with _FILE suffix and read data from the path contained in that variable right via the application. (e.g. Grafana)
   2a. Authentik way, which was already mentioned. Now I realize that they did perhaps a clever workaround to have support of secrets in mountpaths.
3. Check for environment variables with _FILE suffix and set the environment variable without that suffix with the contents of that file via bash black magic (https://github.com/docker-library/mysql/blob/a482468640c602ccd8a7c86a4c7422f18a307326/docker-entrypoint.sh#L24)

[Starttoaster]

I usually do this with j2-style templating, similar to Ansible's template module. But I'll usually make the file I put in git a j2 template, and render it and deploy it in CI using CI secrets added to the CI job's environment variables. There's a few tools for this besides Ansible, including j2cli, p2cli, and jinjanator.

Noting that this is already a solved problem, I actually do think your ask here makes sense to do anyway for the sake of flexibility. But the way I'll probably solve it is with a configuration file, where you'd be able to mount a whole file containing all of the config parameters for proxmox-exporter.

[bielids]

Somewhat related to this, for Kube I ended up adding something like that

└─➜ git --no-pager diff 9c44d22be9861fc75ca742831192ee85d3424c63 7c111d62750c3392b687c3fd5118b88e714bdfae                                                               [0]
diff --git a/chart/proxmox-exporter/templates/deployment.yaml b/chart/proxmox-exporter/templates/deployment.yaml
index 41d0484..2c5d44b 100644
---  a/chart/proxmox-exporter/templates/deployment.yaml
+++ b/chart/proxmox-exporter/templates/deployment.yaml
@@ -52,10 +52,23 @@ spec:
               value: '{{ .Values.config.api_insecure }}'
             - name: PROXMOX_EXPORTER_PROXMOX_ENDPOINTS
               value: '{{ .Values.config.endpoints }}'
+            {{- if .Values.config.existingSecret }}
+            - name: PROXMOX_EXPORTER_PROXMOX_TOKEN_ID
+              valueFrom:
+                secretKeyRef:
+                  name: {{ .Values.config.existingSecret }}
+                  key: PROXMOX_EXPORTER_PROXMOX_TOKEN_ID
             - name: PROXMOX_EXPORTER_PROXMOX_TOKEN
-              value: '{{ .Values.config.token }}'
+              valueFrom:
+                secretKeyRef:
+                  name: {{ .Values.config.existingSecret }}
+                  key: PROXMOX_EXPORTER_PROXMOX_TOKEN
+            {{- else }}
             - name: PROXMOX_EXPORTER_PROXMOX_TOKEN_ID
               value: '{{ .Values.config.tokenID }}'
+            - name: PROXMOX_EXPORTER_PROXMOX_TOKEN
+              value: '{{ .Values.config.token }}'
+            {{- end }}
             - name: PROXMOX_EXPORTER_SERVER_PORT
               value: '{{ .Values.config.port }}'
           resources:

to make it easier to work with my existing secret management system.