How do I replace a non public facing K3s active cluster with corperate signed CA certs?

[davehouser1]

Here is a post with more details about this topic.

The problem is I need to use cooperate signed certs on our K3s cluster, they must be signed by our PKI. Cannot use lets encrypt, should not be using any other CA to sign the certs. Also, not sure its possible to get a intermediate signing certs for K3s to use.

Is there a way to replace certs manually for K3s with contents of a .pfx generated cert from our corporate CA?
Tried manually changing the certs, but I get this error

Jul 01 16:40:38 k3s01.our.domain  k3s[3388482]: time="2025-07-01T16:40:38Z" level=info msg="Reconciling bootstrap data between datastore and disk"
Jul 01 16:40:38 k3s01.our.domain  k3s[3388482]: time="2025-07-01T16:40:38Z" level=fatal msg="/var/lib/rancher/k3s/server/tls/client-ca.crt, /var/lib/rancher/k3s/server/tls/etcd/peer-ca.key, /var/lib/rancher/k3s/server/tls/etcd/server-ca.crt, /var/lib/rancher/k3s/server/tls/client-ca.key, /var/lib/rancher/k3s/server/tls/etcd/peer-ca.crt, /var/lib/rancher/k3s/server/tls/etcd/server-ca.key, /var/lib/rancher/k3s/server/tls/request-header-ca.crt, /var/lib/rancher/k3s/server/tls/request-header-ca.key, /var/lib/rancher/k3s/server/tls/server-ca.crt, /var/lib/rancher/k3s/server/tls/server-ca.key, /var/lib/rancher/k3s/server/tls/service.key newer than datastore and could cause a cluster outage. Remove the file(s) from disk and restart to be recreated from datastore."
Jul 01 16:40:38 k3s01.our.domain  systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
Jul 01 16:40:38 k3s01.our.domain  systemd[1]: k3s.service: Failed with result 'exit-code'.

The post has our system versioning if you need to know that.

Anyway to accomplish this?

[brandond]

Please see the docs at https://docs.k3s.io/cli/certificate#using-custom-ca-certificates

I need to use cooperate signed certs on our K3s cluster, they must be signed by our PKI

Note that this is a terrible idea. Golang and kubernetes do not support CRL checking, so if you ever need to revoke a compromised certificate you'll need to replace your entire internal PKI structure. If you use a standalone PKI for the Kubernetes cluster this is easy, all you need to do is rotate the certs and restart a bunch of things. If you use your corp PKI for the cluster then you're signing up to rekey everything in your entire enterprise.

Use a corp PKI for your ingress controller if you must, so that you have valid certs things that will be exposed externally. Do not use your corp PKI for the cluster CA.

[davehouser1]

Thanks for the details.

Use a corp PKI for your ingress controller if you must

Yes I need to replace the certs for the exposed services (port 6443 and 10250) that is what our security report gave back. Is that what you mean by ingress controller? or are you talking about some other type of network mesh ontop of K3s like Istio?

I have already read the page you shared. Which section are you referring to specifically? (Using Custom CA Certificates?). If so, its not clear to me how I load my certs into K3s. It mentions using that script:

An example script to pre-create the appropriate certificates and keys is available in the K3s repo at contrib/util/generate-custom-ca-certs.sh. This script should be run prior to starting K3s for the first time, and will create a full set of leaf CA certificates signed by common Root and Intermediate CA certificates. If you have an existing Root or Intermediate CA, this script can be used (or used as a starting point) to create the correct CA certificates to provision a K3s cluster with PKI rooted in an existing authority.

This script seems to require me to provide a private and public key for some K3s CA to use. Unfortunately I cant do that. I read through the script the script still tries to create its own CA for signing. Is there no way around this?

I think I am missing something here. Is there an easy way to just replace the certs of the exposed services? This cluster only has one master node right now. We may expand later, I can generate certs for those nodes too if needed and integrate. I am just not understand the process of how to accomplish this.

[brandond]

Yes I need to replace the certs for the exposed services (port 6443 and 10250) that is what our security report gave back.

This would be the cluster certs then, not the ingress certs. Your corp security folks need to be educated as to how Kubernetes manages certificates.

This script seems to require me to provide a private and public key for some K3s CA to use. Unfortunately I cant do that.

Yes, that is what is necessary. If you can't do that, then you can't use your corp CA.

I read through the script the script still tries to create its own CA for signing. Is there no way around this?

No, there is no way around this. Kubernetes needs its own internal CA certs that it can use to sign things on demand. This is how Kubernetes works.

Is there an easy way to just replace the certs of the exposed services.

No. Kubernetes needs a bunch of certificates, which are (generally) issued by different CAs to limit the blast radius should something need to be revoked or rekeyed.

[davehouser1]

Ok I was able to generate a intermediate CA public, private, and root chain files from our CA. I tried the following

• I move the new intermediate CA public, private, and root chain to the host. I also copied over the current service.key as well

$ find /tmp/k3s-ca-rotate/
/tmp/k3s-ca-rotate/
/tmp/k3s-ca-rotate/server
/tmp/k3s-ca-rotate/server/tls
/tmp/k3s-ca-rotate/server/tls/intermediate-ca.key
/tmp/k3s-ca-rotate/server/tls/service.key
/tmp/k3s-ca-rotate/server/tls/root-ca.pem
/tmp/k3s-ca-rotate/server/tls/intermediate-ca.pem

• Current certs

$ sudo k3s certificate check --output table --debug --data-dir /var/lib/rancher/k3s
DEBU[0000] Asset dir /var/lib/rancher/k3s/data/def189ea4d6d3f91781467cdee8a14c124d826f1b3b370103bd7a6e866f1144f
DEBU[0000] Running /var/lib/rancher/k3s/data/def189ea4d6d3f91781467cdee8a14c124d826f1b3b370103bd7a6e866f1144f/bin/k3s-certificate [k3s certificate check --output table --debug --data-dir /var/lib/rancher/k3s]
INFO[0000] Server detected, checking agent and server certificates

CERTIFICATE                      SUBJECT                                             STATUS  EXPIRES
-----------                      -------                                             ------  -------
client-scheduler.crt             CN=system:kube-scheduler                            OK      2026-03-20T15:16:02Z
client-scheduler.crt             CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-kube-proxy.crt            CN=system:kube-proxy                                OK      2026-07-02T18:31:39Z
client-kube-proxy.crt            CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-kubelet.crt               CN=system:node:k3s01.our.domain,O=system:nodes  OK      2026-07-02T18:31:39Z
client-kubelet.crt               CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
serving-kubelet.crt              CN=k3s01.our.domain                             OK      2026-07-02T18:31:39Z
serving-kubelet.crt              CN=k3s-server-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-kube-apiserver.crt        CN=system:apiserver,O=system:masters                OK      2026-03-20T15:16:02Z
client-kube-apiserver.crt        CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
serving-kube-apiserver.crt       CN=kube-apiserver                                   OK      2026-03-20T15:16:02Z
serving-kube-apiserver.crt       CN=k3s-server-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-admin.crt                 CN=system:admin,O=system:masters                    OK      2026-03-20T15:16:02Z
client-admin.crt                 CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-controller.crt            CN=system:kube-controller-manager                   OK      2026-03-20T15:16:02Z
client-controller.crt            CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
client.crt                       CN=etcd-client                                      OK      2026-03-20T15:16:02Z
client.crt                       CN=etcd-server-ca@1742483762                        OK      2035-03-18T15:16:02Z
server-client.crt                CN=etcd-server                                      OK      2026-03-20T15:16:02Z
server-client.crt                CN=etcd-server-ca@1742483762                        OK      2035-03-18T15:16:02Z
peer-server-client.crt           CN=etcd-peer                                        OK      2026-03-20T15:16:02Z
peer-server-client.crt           CN=etcd-peer-ca@1742483762                          OK      2035-03-18T15:16:02Z
client-auth-proxy.crt            CN=system:auth-proxy                                OK      2026-03-20T15:16:02Z
client-auth-proxy.crt            CN=k3s-request-header-ca@1742483762                 OK      2035-03-18T15:16:02Z
client-k3s-cloud-controller.crt  CN=k3s-cloud-controller-manager                     OK      2026-03-20T15:16:02Z
client-k3s-cloud-controller.crt  CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-supervisor.crt            CN=system:k3s-supervisor,O=system:masters           OK      2026-03-20T15:16:02Z
client-supervisor.crt            CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-k3s-controller.crt        CN=system:k3s-controller                            OK      2026-07-02T18:31:39Z
client-k3s-controller.crt        CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z

• I tried using the generate-custom-ca-certs.sh script

$ sudo DATA_DIR=/tmp/k3s-ca-rotate/server ./generate-custom-ca-certs.sh
Using /usr/bin/openssl: OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
Generating Kubernetes service account issuer RSA key
Generating root certificate authority RSA key and certificate
Generating intermediate certificate authority RSA key and certificate
Using configuration from .ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-intermediate-ca@1751482776'
Certificate is to be certified until Aug 19 18:59:40 2035 GMT (3700 days)

Write out database with 1 new entries
Data Base Updated
Generating k3s-client-ca leaf certificate authority EC key and certificate
Using configuration from .ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-client-ca@1751482776'
Certificate is to be certified until Aug 19 18:59:40 2035 GMT (3700 days)

Write out database with 1 new entries
Data Base Updated
Generating k3s-server-ca leaf certificate authority EC key and certificate
Using configuration from .ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-server-ca@1751482776'
Certificate is to be certified until Aug 19 18:59:40 2035 GMT (3700 days)

Write out database with 1 new entries
Data Base Updated
Generating k3s-request-header-ca leaf certificate authority EC key and certificate
Using configuration from .ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-request-header-ca@1751482776'
Certificate is to be certified until Aug 19 18:59:40 2035 GMT (3700 days)

Write out database with 1 new entries
Data Base Updated
Generating k3s-etcd-peer-ca leaf certificate authority EC key and certificate
Using configuration from .ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-etcd-peer-ca@1751482776'
Certificate is to be certified until Aug 19 18:59:40 2035 GMT (3700 days)

Write out database with 1 new entries
Data Base Updated
Generating k3s-etcd-server-ca leaf certificate authority EC key and certificate
Using configuration from .ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-etcd-server-ca@1751482776'
Certificate is to be certified until Aug 19 18:59:40 2035 GMT (3700 days)

Write out database with 1 new entries
Data Base Updated

CA certificate generation complete. Required files are now present in: /tmp/k3s-ca-rotate/server/server/tls
For security purposes, you should make a secure copy of the following files and remove them from cluster members:
         /tmp/k3s-ca-rotate/server/server/tls/intermediate-ca.crt
         /tmp/k3s-ca-rotate/server/server/tls/intermediate-ca.key
         /tmp/k3s-ca-rotate/server/server/tls/intermediate-ca.pem
         /tmp/k3s-ca-rotate/server/server/tls/
         /tmp/k3s-ca-rotate/server/server/tls/root-ca.key
         /tmp/k3s-ca-rotate/server/server/tls/root-ca.pem

To update certificates on an existing cluster, you may now run:
    k3s certificate rotate-ca --path=/tmp/k3s-ca-rotate/server/server
$ sudo find /tmp/k3s-ca-rotate/server/server/tls
/tmp/k3s-ca-rotate/server/server/tls
/tmp/k3s-ca-rotate/server/server/tls/intermediate-ca.crt
/tmp/k3s-ca-rotate/server/server/tls/root-ca.key
/tmp/k3s-ca-rotate/server/server/tls/request-header-ca.crt
/tmp/k3s-ca-rotate/server/server/tls/server-ca.crt
/tmp/k3s-ca-rotate/server/server/tls/client-ca.key
/tmp/k3s-ca-rotate/server/server/tls/etcd
/tmp/k3s-ca-rotate/server/server/tls/etcd/peer-ca.key
/tmp/k3s-ca-rotate/server/server/tls/etcd/peer-ca.pem
/tmp/k3s-ca-rotate/server/server/tls/etcd/server-ca.crt
/tmp/k3s-ca-rotate/server/server/tls/etcd/peer-ca.crt
/tmp/k3s-ca-rotate/server/server/tls/etcd/server-ca.pem
/tmp/k3s-ca-rotate/server/server/tls/etcd/server-ca.key
/tmp/k3s-ca-rotate/server/server/tls/intermediate-ca.key
/tmp/k3s-ca-rotate/server/server/tls/client-ca.crt
/tmp/k3s-ca-rotate/server/server/tls/request-header-ca.pem
/tmp/k3s-ca-rotate/server/server/tls/service.key
/tmp/k3s-ca-rotate/server/server/tls/root-ca.pem
/tmp/k3s-ca-rotate/server/server/tls/server-ca.pem
/tmp/k3s-ca-rotate/server/server/tls/server-ca.key
/tmp/k3s-ca-rotate/server/server/tls/request-header-ca.key
/tmp/k3s-ca-rotate/server/server/tls/root-ca.crt
/tmp/k3s-ca-rotate/server/server/tls/intermediate-ca.pem
/tmp/k3s-ca-rotate/server/server/tls/client-ca.pem

At this point I am confused on what I should be doing next. First of all the certs were generated without prompting me for the Intermediate CA private key password, how is that possible? Second of all, when I try to run the following, it looks like the certs were rotated

$ sudo k3s certificate rotate-ca --path=/tmp/k3s-ca-rotate/server/server --force
certificates saved to datastore

However when I restart k3s, nothing trusts the certs and the journalctl and kubectl commands show no trust.
I also tried replacing the /etc/rancher/k3s/k3s.yaml but it did not help. I even tried pointing to the root-ca.crt root chain, still does not work.

At this point I started over, I rotated, then I downloaded this script and ran it.

sudo ./rotate-default-ca-certs.sh
Using /usr/bin/openssl: OpenSSL 3.0.2 15 Mar 2022 (Library: OpenSSL 3.0.2 15 Mar 2022)
Generating k3s-client-root root and cross-signed certificate authority key and certificates
Using configuration from ./.ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-client-root@1751483501'
Certificate is to be certified until Jun 27 19:11:41 2045 GMT (7300 days)

Write out database with 1 new entries
Data Base Updated
Generating k3s-client-ca intermediate certificate authority key and certificates
Using configuration from ./.ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-client-ca@1751483501'
Certificate is to be certified until Jun 27 19:11:41 2045 GMT (7300 days)

Write out database with 1 new entries
Data Base Updated
Generating k3s-server-root root and cross-signed certificate authority key and certificates
Using configuration from ./.ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-server-root@1751483501'
Certificate is to be certified until Jun 27 19:11:41 2045 GMT (7300 days)

Write out database with 1 new entries
Data Base Updated
Generating k3s-server-ca intermediate certificate authority key and certificates
Using configuration from ./.ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-server-ca@1751483501'
Certificate is to be certified until Jun 27 19:11:41 2045 GMT (7300 days)

Write out database with 1 new entries
Data Base Updated
Generating k3s-request-header-root root and cross-signed certificate authority key and certificates
Using configuration from ./.ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-request-header-root@1751483501'
Certificate is to be certified until Jun 27 19:11:41 2045 GMT (7300 days)

Write out database with 1 new entries
Data Base Updated
Generating k3s-request-header-ca intermediate certificate authority key and certificates
Using configuration from ./.ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-request-header-ca@1751483501'
Certificate is to be certified until Jun 27 19:11:41 2045 GMT (7300 days)

Write out database with 1 new entries
Data Base Updated
Generating k3s-etcd-peer-root root and cross-signed certificate authority key and certificates
Using configuration from ./.ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-etcd-peer-root@1751483501'
Certificate is to be certified until Jun 27 19:11:41 2045 GMT (7300 days)

Write out database with 1 new entries
Data Base Updated
Generating k3s-etcd-peer-ca intermediate certificate authority key and certificates
Using configuration from ./.ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-etcd-peer-ca@1751483501'
Certificate is to be certified until Jun 27 19:11:41 2045 GMT (7300 days)

Write out database with 1 new entries
Data Base Updated
Generating k3s-etcd-server-root root and cross-signed certificate authority key and certificates
Using configuration from ./.ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-etcd-server-root@1751483501'
Certificate is to be certified until Jun 27 19:11:41 2045 GMT (7300 days)

Write out database with 1 new entries
Data Base Updated
Generating k3s-etcd-server-ca intermediate certificate authority key and certificates
Using configuration from ./.ca/config
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'k3s-etcd-server-ca@1751483501'
Certificate is to be certified until Jun 27 19:11:41 2045 GMT (7300 days)

Write out database with 1 new entries
Data Base Updated

Cross-signed CA certs and keys now available in /var/lib/rancher/k3s/server/rotate-ca
Updated server token:    K10594a8f6f1b3c54a5c801e3dbdc31bcf59072f5908b5b38607c29704de48bbb5c::server:b9c3a65d9cebeeaa65b215f77abac923
Updated agent token:     K10594a8f6f1b3c54a5c801e3dbdc31bcf59072f5908b5b38607c29704de48bbb5c::server:b9c3a65d9cebeeaa65b215f77abac923

To update certificates, you may now run:
    k3s certificate rotate-ca --path=/var/lib/rancher/k3s/server/rotate-ca

$ sudo k3s certificate rotate-ca --path=/var/lib/rancher/k3s/server/rotate-ca
certificates saved to datastore

I restarted k3s, and it works fine, and when I check certs it looks like there are many more certs. Which means something happened.

$ sudo k3s certificate check --output table --debug --data-dir /var/lib/rancher/k3s
DEBU[0000] Asset dir /var/lib/rancher/k3s/data/def189ea4d6d3f91781467cdee8a14c124d826f1b3b370103bd7a6e866f1144f
DEBU[0000] Running /var/lib/rancher/k3s/data/def189ea4d6d3f91781467cdee8a14c124d826f1b3b370103bd7a6e866f1144f/bin/k3s-certificate [k3s certificate check --output table --debug --data-dir /var/lib/rancher/k3s]
INFO[0000] Server detected, checking agent and server certificates

CERTIFICATE                      SUBJECT                                             STATUS  EXPIRES
-----------                      -------                                             ------  -------
client-k3s-controller.crt        CN=system:k3s-controller                            OK      2026-07-02T19:12:31Z
client-k3s-controller.crt        CN=k3s-client-ca@1751483501                         OK      2045-06-27T19:11:41Z
client-k3s-controller.crt        CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-k3s-controller.crt        CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-k3s-controller.crt        CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-controller.crt            CN=system:kube-controller-manager                   OK      2026-07-02T19:12:30Z
client-controller.crt            CN=k3s-client-ca@1751483501                         OK      2045-06-27T19:11:41Z
client-controller.crt            CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-controller.crt            CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-controller.crt            CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-scheduler.crt             CN=system:kube-scheduler                            OK      2026-07-02T19:12:30Z
client-scheduler.crt             CN=k3s-client-ca@1751483501                         OK      2045-06-27T19:11:41Z
client-scheduler.crt             CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-scheduler.crt             CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-scheduler.crt             CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-kubelet.crt               CN=system:node:v00lin-ljf01.cso.lab,O=system:nodes  OK      2026-07-02T19:12:31Z
client-kubelet.crt               CN=k3s-client-ca@1751483501                         OK      2045-06-27T19:11:41Z
client-kubelet.crt               CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-kubelet.crt               CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-kubelet.crt               CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
serving-kubelet.crt              CN=v00lin-ljf01.cso.lab                             OK      2026-07-02T19:12:31Z
serving-kubelet.crt              CN=k3s-server-ca@1751483501                         OK      2045-06-27T19:11:41Z
serving-kubelet.crt              CN=k3s-server-root@1751483501                       OK      2045-06-27T19:11:41Z
serving-kubelet.crt              CN=k3s-server-root@1751483501                       OK      2045-06-27T19:11:41Z
serving-kubelet.crt              CN=k3s-server-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-k3s-cloud-controller.crt  CN=k3s-cloud-controller-manager                     OK      2026-07-02T19:12:30Z
client-k3s-cloud-controller.crt  CN=k3s-client-ca@1751483501                         OK      2045-06-27T19:11:41Z
client-k3s-cloud-controller.crt  CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-k3s-cloud-controller.crt  CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-k3s-cloud-controller.crt  CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
client.crt                       CN=etcd-client                                      OK      2026-07-02T19:12:30Z
client.crt                       CN=k3s-etcd-server-ca@1751483501                    OK      2045-06-27T19:11:41Z
client.crt                       CN=k3s-etcd-server-root@1751483501                  OK      2045-06-27T19:11:41Z
client.crt                       CN=k3s-etcd-server-root@1751483501                  OK      2045-06-27T19:11:41Z
client.crt                       CN=etcd-server-ca@1742483762                        OK      2035-03-18T15:16:02Z
server-client.crt                CN=etcd-server                                      OK      2026-07-02T19:12:30Z
server-client.crt                CN=k3s-etcd-server-ca@1751483501                    OK      2045-06-27T19:11:41Z
server-client.crt                CN=k3s-etcd-server-root@1751483501                  OK      2045-06-27T19:11:41Z
server-client.crt                CN=k3s-etcd-server-root@1751483501                  OK      2045-06-27T19:11:41Z
server-client.crt                CN=etcd-server-ca@1742483762                        OK      2035-03-18T15:16:02Z
peer-server-client.crt           CN=etcd-peer                                        OK      2026-07-02T19:12:30Z
peer-server-client.crt           CN=k3s-etcd-peer-ca@1751483501                      OK      2045-06-27T19:11:41Z
peer-server-client.crt           CN=k3s-etcd-peer-root@1751483501                    OK      2045-06-27T19:11:41Z
peer-server-client.crt           CN=k3s-etcd-peer-root@1751483501                    OK      2045-06-27T19:11:41Z
peer-server-client.crt           CN=etcd-peer-ca@1742483762                          OK      2035-03-18T15:16:02Z
client-supervisor.crt            CN=system:k3s-supervisor,O=system:masters           OK      2026-07-02T19:12:30Z
client-supervisor.crt            CN=k3s-client-ca@1751483501                         OK      2045-06-27T19:11:41Z
client-supervisor.crt            CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-supervisor.crt            CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-supervisor.crt            CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-kube-proxy.crt            CN=system:kube-proxy                                OK      2026-07-02T19:12:31Z
client-kube-proxy.crt            CN=k3s-client-ca@1751483501                         OK      2045-06-27T19:11:41Z
client-kube-proxy.crt            CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-kube-proxy.crt            CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-kube-proxy.crt            CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-kube-apiserver.crt        CN=system:apiserver,O=system:masters                OK      2026-07-02T19:12:30Z
client-kube-apiserver.crt        CN=k3s-client-ca@1751483501                         OK      2045-06-27T19:11:41Z
client-kube-apiserver.crt        CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-kube-apiserver.crt        CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-kube-apiserver.crt        CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
serving-kube-apiserver.crt       CN=kube-apiserver                                   OK      2026-07-02T19:12:30Z
serving-kube-apiserver.crt       CN=k3s-server-ca@1751483501                         OK      2045-06-27T19:11:41Z
serving-kube-apiserver.crt       CN=k3s-server-root@1751483501                       OK      2045-06-27T19:11:41Z
serving-kube-apiserver.crt       CN=k3s-server-root@1751483501                       OK      2045-06-27T19:11:41Z
serving-kube-apiserver.crt       CN=k3s-server-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-admin.crt                 CN=system:admin,O=system:masters                    OK      2026-07-02T19:12:30Z
client-admin.crt                 CN=k3s-client-ca@1751483501                         OK      2045-06-27T19:11:41Z
client-admin.crt                 CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-admin.crt                 CN=k3s-client-root@1751483501                       OK      2045-06-27T19:11:41Z
client-admin.crt                 CN=k3s-client-ca@1742483762                         OK      2035-03-18T15:16:02Z
client-auth-proxy.crt            CN=system:auth-proxy                                OK      2026-07-02T19:12:30Z
client-auth-proxy.crt            CN=k3s-request-header-ca@1751483501                 OK      2045-06-27T19:11:41Z
client-auth-proxy.crt            CN=k3s-request-header-root@1751483501               OK      2045-06-27T19:11:41Z
client-auth-proxy.crt            CN=k3s-request-header-root@1751483501               OK      2045-06-27T19:11:41Z
client-auth-proxy.crt            CN=k3s-request-header-ca@1742483762                 OK      2035-03-18T15:16:02Z

However, when I check the actual services I see that the Issuer field is signed by something that is not our CA or Intermediate CA cert

$ ./check-k3s-certs-used.sh
[+] Check port 127.0.0.1:6443
        Issuer: CN = k3s-server-ca@1742483762
        Validity
--
        Subject: O = k3s, CN = k3s
        Subject Public Key Info:
--
            X509v3 Subject Alternative Name:
                DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc, DNS:kubernetes.default.svc.cluster.local, DNS:localhost, DNS:k3s01.our.domain, IP Address:10.200.2.238, IP Address:10.43.0.1, IP Address:127.0.0.1, IP Address:0:0:0:0:0:0:0:1
[+] Check port 127.0.0.1:10250
        Issuer: CN = k3s-server-ca@1751483501
        Validity
--
        Subject: CN = k3s01.our.domain
        Subject Public Key Info:
--
            X509v3 Subject Alternative Name:
                DNS:k3s01.our.domain, DNS:localhost, IP Address:127.0.0.1, IP Address:10.200.2.238

I would think that that Issuer would be our Root CA, this is what the Intermediate cert I loaded looks like

$ openssl x509 -in /tmp/k3s-ca-rotate/server/tls/intermediate-ca.pem -text -noout
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 4112 (0x1010)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, ST = MO, L = Sometown, O = OurCompany, OU = someOU, CN = ourca.our.domain, emailAddress = admin@our.domain
        Validity
            Not Before: Jul  2 15:12:34 2025 GMT
            Not After : Jun 30 15:12:34 2035 GMT
        Subject: C = US, ST = MO, L = Sometown, O = OurCompany, OU = someOU, CN = k3s01.our.domain
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (4096 bit)

Am I missing a step or doing something wrong here? Why was I not prompted for the private key password when using the generate script? Why are the new scripts look like they are signed by some CA that is not ours?
Sorry for all the questions, I appreciate your help through this.

[brandond]

I would not recommend trying to rotate an existing cluster from the default autogenerated CA to an existing corp CA, especially if you are struggling to understand all the moving pieces. Build a new cluster that uses your corp certs from the start, as covered in the docs.

[davehouser1]

Thanks for your help with this. I was able to uninstall K3s, and redeploy with our signed certs. I have confirmed that the new certs were used to sign the k3s certs when it deployed with the generate-custom-ca-certs.sh script. You were right. Rotation works too. You were right, its easier to uninstall and redeploy with the correct certs.