Step CA in Docker doesn't trust it's self

[tashian]

Discussed in #1904

^{Originally posted by hardillb June 29, 2024}
I'm running a Step CA in the provided docker container, I've got it set up with a ACME provisioner which I'm using with cert-manger to issue certificates for Ingress objects exposing services in my Kubernetes cluster.

I was trying to add a ODIC provisioner to issue SSH certificates, My SSO provider is a Keyclock install in my Kubernetes environment.

I had initially not setup up HTTPS for Keycloak and ran into this problem.

So I used cert-manager to add a Step CA issued certificate to the Keycloak ingress, this works just fine from my browser and other tools where I have imported the Step CA root certificate, but when I try to add the ODIC provider again, I get errors like this in the logs:

time="2024-06-29T09:44:38Z" level=error duration=79.061144ms duration-ns=79061144 error="error initializing provisioner keycloak: failed to connect to https://keycloak.k8s.loc/realms/FlowFuse/.well-known/openid-configuration: Get \"https://keycloak.k8s.loc/realms/FlowFuse/.well-known/openid-configuration\": tls: failed to verify certificate: x509: certificate signed by unknown authority" fields.time="2024-06-29T09:44:38Z" method=PUT name=ca path=/admin/provisioners/keycloak protocol=HTTP/2.0 referer= remote-address=192.168.1.111 request-id=fa8127d1-02c0-4f18-a6c4-55d91f37d46a response="{\"type\":\"internalServerError\",\"detail\":\"the server experienced an internal error\",\"message\":\"error initializing provisioner keycloak: failed to connect to https://keycloak.k8s.loc/realms/FlowFuse/.well-known/openid-configuration: Get \\\"https://keycloak.k8s.loc/realms/FlowFuse/.well-known/openid-configuration\\\": tls: failed to verify certificate: x509: certificate signed by unknown authority\"}" size=396 status=500 user-agent="Smallstep CLI/0.21.0 (linux/amd64)" user-id=

This implies that the step-ca instance doesn't trust certificates it issued.

How do I fix this?

[hardillb]

Just as a data point here. I worked out how to get a root shell in the docker container so (as mentioned in smallstep/cli#1216) tried to add the root_ca cert to the trusted certs. I did following

• symlinked /home/step/certs/root_ca.crt to /usr/local/share/ca-certificates/step.crt
• ran update-ca-certificates which updated the hashes in /etc/sssl/certs and added the cert to the end of /etc/ssl/certs/ca-certificates.crt

Now running curl in the container trusts certs issued by an Step CA ACME provisioner, but trying to add an OIDC provisioner that points to a configuration-endpoint that is protected by a Step CA issued cert still fails

$ step ca provisioner add keycloak --type OIDC --client-id step-ca --client-secret 0OLuF5LOrP3xxxxxxxxxxxxxxxx --configuration-endpoint https://keycloak.k8s.loc/realms/FlowFuse/.well-known/openid-configuration --listen-address :10000
✔ Please enter admin name/subject (e.g., name@example.com): step█
✔ Provisioner: admin (JWK) [kid: 0a-rqRAmTchbpudXPKJhCmnPDz0B2cxKehEf_VwYDl8]
Please enter the password to decrypt the provisioner key: 
error storing provisioner keycloak: error validating configuration for provisioner "keycloak": failed to connect to https://keycloak.k8s.loc/realms/FlowFuse/.well-known/openid-configuration: Get "https://keycloak.k8s.loc/realms/FlowFuse/.well-known/openid-configuration": tls: failed to verify certificate: x509: certificate signed by unknown authority

[hardillb]

Does the step-ca binary check certs against /etc/ssl/certs/ca-certificate.crt file on every request?

If not, does it reload it on a HUP signal?