Securing Kubernetes Cluster With Cert-Manager And Self-Signed Certificates

2024-05-06 1133 words 6 minutes

Contents

Kubernetes is an incredible tool for deploying, scaling, and managing containerized applications. One crucial aspect of kubernetes security is ensuring that communication between different entities is secure. By default, kubernetes management network is secure and pod network is handled by 3rd party plugin which mostly support encryption.

Today we will focus on properly securing outside-in web communication to our cluster with Cert-Manager and self-signed certificates. We assume that you have access to working kubernetes cluster with ingress controller.

Setting up the certificate chain

Self-signed certificates are digital certificates issued by the same entity that will be using them. In other words, you’re essentially issuing a certificate to yourself. This approach is often preferred when you don’t have access to a public CA or when you want to keep your traffic private.

Usually you don’t have just one cluster on premise but a combination of cluster, vms, servers etc. To make management of certificates easier, we will build a certificate chain and use dedicated intermediate certificate to sign certificates in the cluster.

/securing-kubernetes-cluster-with-cert-manager-and-self-signed-certificates/certificate_chain.png — Fig 1. Certificate chain

To start install ansible on your computer.

1
2
3


sudo apt-add-repository ppa:ansible/ansible
sudo apt update
sudo apt install ansible

Use following playbook to create root and intermediate certificates.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90


# file: create-certificates.yaml
---

- name: Ensure ca server has needed certificates
  hosts: localhost
  connection: local
  tasks:

    - name: Ensure ssl directories are created
      ansible.builtin.file:
        path: "{{ item.path }}"
        state: directory
        mode: "{{ item.mode }}"
      loop:
        - { path: "./private", mode: "0700" }
        - { path: "./certs", mode: "0755"}
        - { path: "./csr", mode: "0755"}


    - name: Ensure root ca private key is present
      community.crypto.openssl_privatekey:
        path: ./private/root-ca.key
        size: 4096


    - name: Ensure root ca csr is present
      community.crypto.openssl_csr:
        path: ./csr/root-ca.csr
        privatekey_path: ./private/root-ca.key
        common_name: Ansible CA
        basic_constraints:
          - "CA:TRUE"
        basic_constraints_critical: true
        key_usage:
          - keyCertSign
        key_usage_critical: true


    - name: Ensure root ca self signed OpenSSL certificate is present
      community.crypto.x509_certificate:
        path: ./certs/root-ca.crt
        privatekey_path: ./private/root-ca.key
        csr_path: ./csr/root-ca.csr
        provider: selfsigned
        selfsigned_not_after: "+36500d"


    - name: Ensure kubernetes intermediate OpenSSL private key is present
      community.crypto.openssl_privatekey:
        path: ./private/intermediate-kubernetes.key
        size: 4096


    - name: Ensure kubernetes intermediate csr is present
      community.crypto.openssl_csr:
        path: ./csr/intermediate-kubernetes.csr
        privatekey_path: ./private/intermediate-kubernetes.key
        common_name: Kubernetes CA
        basic_constraints:
          - "CA:TRUE"
        basic_constraints_critical: true
        key_usage:
          - keyCertSign
        key_usage_critical: true


    - name: Ensure kubernetes intermediate self signed OpenSSL certificate is present
      community.crypto.x509_certificate:
        path: ./certs/intermediate-kubernetes.crt
        privatekey_path: ./private/intermediate-kubernetes.key
        csr_path: ./csr/intermediate-kubernetes.csr
        provider: ownca
        ownca_path: ./certs/root-ca.crt
        ownca_privatekey_path: ./private/root-ca.key
        ownca_not_after: "+36500d"

    - name: Set certificate order for cert-manager
      ansible.builtin.set_fact:
        intermediate_kubernetes_chain_files:
            - ./certs/intermediate-kubernetes.crt
            - ./certs/root-ca.crt

    - name: Read certificate content in order
      command: awk 1 {{ intermediate_kubernetes_chain_files | join(" ") }}
      register: intermediate_kubernetes_chain_contents

    - name: Ensure kubernetes intermediate chain file is present
      copy:
        dest: ./certs/intermediate-kubernetes-chain.crt
        content: "{{ intermediate_kubernetes_chain_contents.stdout_lines | unique |join('\n') }}"

Securing and managing your certificates is outside of the scope of this post but make sure you follow best practice for managing sensitive files.

1

ansible-playbook create-certificates.yaml

After executing the playbook you will have access to intermediate certificate file ./certs/intermediate-kubernetes-chain.crt and corresponding private key ./private/intermediate-kubernetes.key.

Setting Up Cert-Manager with Self-Signed Certificates

Cert-Manager is a tool for managing TLS/SSL certificates in Kubernetes. Install Cert-Manager using following command. Make sure that the version is compatible with your current kubernetes version.

1

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.5/cert-manager.yaml

Once installed, create a secret object to hold information about the certificate and the private key from previous step. Both certificate and the key values need to be base64 encoded. You can encode the values with:

1

base64 -w 0 <file>

Secret has to be in the same namespace as Cert-Manager, by default that is cert-manager. Secret will be used by cluster issuer object to sign certificates in your cluster. Find manifest file below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20


# cluster-issuer-and-secret.yaml
---
apiVersion: v1
kind: Secret
metadata:
  name: nginx-ca-secret
  namespace: cert-manager
type: Opaque
data:
    tls.crt: ### enter base 64 encoded certificate from step before
    tls.key:  ### enter base 64 encoded key from step before

---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: nginx-clusterissuer
spec:
  ca:
    secretName: nginx-ca-secret

Create kubernetes objects and check that cluster issuer is in ready state.

1
2


kubectl apply -f cluster-issuer-and-secret.yaml
kubectl get clusterissuer

Example Ingress with Self-Signed Certificates

Now lets see it all in action. We will issue new certificate with cluster issuer and use it with our test application and ingress.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70


# file: test-application.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: test-deployment
  name: test-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test-pod
  template:
    metadata:
      labels:
        app: test-pod
    spec:
      containers:
      - image: nginx
        name: nginx

---
apiVersion: v1
kind: Service
metadata:
  name: test-deployment-svc
spec:
  ports:
  - port: 8080
    targetPort: 80
    protocol: TCP
  selector:
    app: test-pod

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
    name: nginx-cert
spec:
    secretName: nginx-tls-secret
    issuerRef:
        name: nginx-clusterissuer
        kind: ClusterIssuer
    dnsNames:
        - test.mycluster.local

---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - test.mycluster.local
    secretName: nginx-tls-secret
  rules:
  - host: test.mycluster.local
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: test-deployment-svc
            port:
              number: 8080

Apply the file.

1

kubectl apply -f test-application.yaml

Test that certificate is issued and is in ready state with kubectl get certificate. By default self signed certificates are not trusted by browsers. Open your browser and add your root certificate ./certs/root-ca.crt to trusted certificates on your local machine.

Ensure that the domain in your ingress object resolves to your ingress controller. For testing purposes you can edit your local /etc/hosts/ file.

1
2


# file: /etc/hosts
<ingress ip>  test.mycluster.local

Test that you have secure access to the application on https://test.mycluster.local and that you are not getting any warnings in your browser.

Conclusion

In this post, we explored how to use Cert-Manager with self-signed certificates to secure our Kubernetes cluster. We issued root certificate and dedicated intermediate certificate for kubernetes cluster. After that we created and configured cluster issuer object for issuing certificates in the cluster and tested everything with test application.

Happy engineering!