Using cert-manager to manage K8s certificates (Part 1)

The cert-manager operator for RedHat OpenShift conveniently provides support to automate PKI certificate management. The operator is RedHat's delivery of the upstream Cert Manager project, which adds a few CRDs to manage certificates and certificate issuers (CAs) in a Kubernetes-based environment.
Hence obtaining, renewing and using certificates is significantly simplified and certificate-related objects can be treated as Kubernetes manifests. This allows us integrate certificate management into our CI/CD process and manage these according to well-known GitOps principles.

In this post we'll explore two widely-used scenarios for our cert-manager operator in an RedHat OpenShift v4.13 environment. While cert-manager operator provides multiple certificate issuer integrations, we will focus on obtaining certificates using the following 2 issuer-types here:

• Scenario 1: Using a Self-Signed issuer
• Scenario 2: Using the Let's Encrypt issuer via ACME protocol (described in part-2 in this series)

Note:
Cert-manager supports two ways to request certificates - namely via CertificateRequest and Certificate objects. We want to demonstrate the fully automated approach for issuing certificates, so we will be using the Certificate object only.

Scenario 1: Using a self-signed Cluster CA

To ensure using SSL/TLS-encrypted traffic from the beginning on in the development lifecycle without any additional cost, you can easily setup your K8s cluster to use a signing certificate authority (CA), and thus quickly obtain SSL/TLS server certificates signed by this CA on demand.

Setting up the cert-manager to provide a self-signing CA can be achieved quite easily by performing the following steps:

1. Create the self-signed ClusterIssuer
2. Create the CA certificate and have it signed by the self-signed ClusterIssuer
3. Create the cluster CA ClusterIssuer

1. Create the self-signed ClusterIssuer

The self-signed ClusterIssuer can be created by applying the following yaml:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned-issuer
  namespace: cert-manager-operator
spec:
  selfSigned: {}

Wait for the operator to create the ClusterIssuer and set the status to True:isReady.
Run oc -n cert-manager-operator describe clusterissuers.cert-manager.io selfsigned-issuer to verify its status before proceeding.

2. Create the CA certificate and have it signed by the self-signed ClusterIssuer

Once the self-signed ClusterIssuer is ready we can proceed by creating the CA certificate. This can be achieved simply by creating a Certificate object referring to the self-signed issuer created before.
For example:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: selfsigned-ca
  namespace: cert-manager
spec:
    commonName: apps.lab.cloud.example.com
    duration: 87600h0m0s
    isCA: true
    issuerRef:
    group: cert-manager.io
    kind: ClusterIssuer
    name: selfsigned-issuer
    privateKey:
    algorithm: ECDSA
    size: 256
    secretName: ca-root-secret
    subject:
    countries:
    - AT
    localities:
    - Vienna
    organizations:
    - alpium-it
    provinces:
    - Vienna

• As this is a demo CA certificate, we've chosen a rather long lifetime of 87600h0m0s (10 years).
• Note that we also need to set isCA: true for a CA certificate!

The name of the K8s secret created for the CA certificate will be ca-root-secret in the cert-manager namespace.
Run

to verify that that the certificate has been issued.

Once that's completed, you can view the actual CA certificate by running

3. Create the cluster CA ClusterIssuer

Now that we have got the CA certificate's private- & public-keys created (as a K8s secret), we can proceed and create the CA ClusterIssuer, which will be the actual signing authority of all new server certificates which users will request to secure their services.
Things might become slightly confusing now, so let me explain:
The cluster CA ClusterIssuer called cluster-ca is another issuer, but this time referring to the CA certificate to be used as a signer (which we created in the previous step).
It can be instantiated using the following example. Note that this time we specify a ca-secret in the spec section instead of making it self-signed!

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: cluster-ca
spec:
  ca:
    secretName: ca-root-secret

Again, check the creation of the CA ClusterIssuer to become ready and that both the signing CA abd the key pair are verified by running oc describe ClusterIssuer cluster-ca.

Notes:

• cert-manager provides namespaced Issuers and cluster-wide ClusterIssuer CRDs. For simplicity, we are using ClusterIssuers only in our example.
• Do not forget that to allow seamless communication from any client to any service using a self-signed certificate, it is best-practice to import that CA's signer-certificate into the client's trust-store.

Using our newly created cluster CA

Using cert-manager certificates to easily secure applications

Once we have finished setting up our cluster CA, we can start using it to sign new SSL/TLS certificates. However, we must keep in mind that the representation of a signed certificate in a Kubernetes environments will be a K8s secret.
Cert-manager's Certificate CRD simply is the resource used trigger a CertificateRequest, signing etc. - and finally creates a new secret which can then be used by your application - or by the Ingress itself.

Using an Ingress/Route certificate

Openshift provides the Route resource which lets you access applications running in the cluster from the outside - just like K8s Ingress. A Route however does not allow to refer to an existing external secret, which might change over time underneath (e.g. when cert-manager does a certificate renewal). To be able to use cert-manager-issued certificates to encrypt the traffic between the client and the application running on OpenShift we need to create an Ingress object. Openshift by default watches Ingress objects and automatically creates the dependent Route object for you.

Long story short - we just need to add a few annotations to the Ingress object which allows us to trigger the certificate creation process, resulting in a TLS-encrypted Route object.

To provide a TLS Route to an application (here the very simple hello-openshift application) we have to create a manifest for an Ingress which we annotate with the ClusterIssuer information as show below:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hello-openshift
  namespace: infra
  annotations:
    cert-manager.io/cluster-issuer: cluster-ca
    cert-manager.io/duration: 8760h
    cert-manager.io/renew-before: 360h
    haproxy.router.openshift.io/timeout: 15m  # custom annot. will be copied to the Route!
spec:
  ingressClassName: openshift-default
  tls:
    - hosts:
      - hello-openshift.apps.lab.cloud.example.com
      secretName: hello-crt
  rules:
    - host: hello-openshift.apps.lab.cloud.example.com
      http:
        paths:
        - backend:
            service:
              name: hello-openshift
              port:
                number: 8080
          path: /
          pathType: Prefix

Once applied, this Ingress specification will:

• trigger the creation of a certificate secret named hello-crt for host provided via Ingress.spec.tls.hostname. This certificate will be signed by the ClusterIssuer provided via the cert-manager.io/cluster-issuer annotation.
  Note that we are using additional, optional cert-manager annotations:
  • cert-manager.io/duration to specify the validity of 1 year (365*24 hours)
  • cert-manager.io/renew-before to have this certificate be renewed 15 days (15 * 24 hours) before expiration.
• create a Route which uses the hostname from Ingress.spec.tls.hostname and the certificate's public and private key-pair obtained from the hello-crt secret.

Using cert-manager certificates in your application

If your application takes care of encrypting traffic itself (ie. using a secure pass-though route), you can get a TLS certificate (stored in a secret, of course) by applying a Certificate resource like the following:

kind: Certificate
apiVersion: cert-manager.io/v1
metadata:
  name: hello-crt-app
  namespace: infra
spec:
  commonName: hello-openshift.apps.lab.cloud.example.com  # optional when SAN (dnsNames) are used!
  # isCA: false
  issuerRef:
    group: cert-manager.io
    kind: ClusterIssuer
    name: cluster-ca
  privateKey:
    algorithm: ECDSA
    size: 256
  secretName: hello-crt-app
  duration: 8760h0m0s
  renewBefore: 360h0m0s
  dnsNames:
    - "hello-openshift.apps.lab.cloud.example.com"
    # add other DNS names to this list

This certificate specification looks pretty much like the first certificate we created for the cluster CA - however, this time we must not specify the isCA: true attribute!

Once the manifest is applied, we'll get the secret hello-crt-app containing a tls.key and tls.crt data field created which we can then mount into our application pods, or use them in any other way we can use certificates in OpenShift/Kubernetes.

Note: If the certificate is volume-mounted to a pod and gets renewed while the pod is running, the pod usually needs to be restarted to pick up the renewed certificate!

zurück