Kubernetes service with SSL (Let’s Encrypt) on GCP and Cloud DNS configuration

G B Martins gbmartins@gbmartins.com 2020-04-20

Requisites

Tools

  • Helm 3+

  • kubectl

  • gcloud sdk

  • Certbot

  • Cloud DNS running with your zone configuration

Environment

Summary

In this tutorial we will learn how to:

  • Create a basic kubernetes cluster

  • Install new kubernetes resources on a cluster

  • Create basic pod, service and load balancer for tests purposes

  • Config Cloud DNS through the certbot

Create a Kubernetes Cluster

That is a basic kubernetes cluster configuration for this tutorial. If there is already a cluster with an application namespace configured you can skip this.

Create a kubernetes container
# Change the PROJECT_ID according to your environment.
export PROJECT_ID=project_id
# Choose a cluster id
export CLUSTER_ID=ssl-google-dns

gcloud beta container --project ${PROJECT_ID} clusters create ${CLUSTER_ID} \
    --zone "us-central1-c" \
    --no-enable-basic-auth \
    --cluster-version "1.15.11-gke.5" \
    --machine-type "n1-standard-1" \
    --image-type "COS" \
    --disk-type "pd-standard" \
    --disk-size "100" --metadata disable-legacy-endpoints=true \
    --num-nodes "3" \
    --no-enable-master-authorized-networks \
    --addons HorizontalPodAutoscaling,HttpLoadBalancing \
    --no-enable-autoupgrade \
    --no-enable-autorepair

The next step is creating a namespace for your application.

Create a kubernetes container
# Choose the namespace
export NAMESPACE=app

kubectl create namespace ${NAMESPACE}

Install Cert-Manager

In order to heve your certs working in your cluster, you need to install cert-managers.

cert-manager is a native Kubernetes certificate management controller. It can help with issuing certificates from a variety of sources, such as Let’s Encrypt, HashiCorp Vault, Venafi, a simple signing key pair, or self signed.

— cert-manager docs
https://cert-manager.io/docs/

Follow the steps.footnote[From https://cert-manager.io/docs/installation/kubernetes/] below in order to install Cert-Manager in your cluster

All steps here aim kubernetes version 1.15+.
Install the CustomResourceDefinition.
kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v0.14.1/cert-manager.crds.yaml
Create the namespace for cert-manager.
kubectl create namespace cert-manager
Add the Jetstack Helm repository and update cache.
helm repo add jetstack https://charts.jetstack.io
helm repo update
Use Helm 3+.
Install the cert-manager Helm chart
helm install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --version v0.14.1

Create Service Account and Secrets

We need a GCP Service Account to have access and permissions granted for handling DNS entries.

Create DNS Admin Service Account
export GCP_PROJECT=<project_id>

gcloud iam service-accounts create dns-admin \
    --display-name=dns-admin \
    --project=${GCP_PROJECT}

gcloud iam service-accounts keys create ./gcp-dns-admin.json \
    --iam-account=dns-admin@${GCP_PROJECT}.iam.gserviceaccount.com \
    --project=${GCP_PROJECT}

gcloud projects add-iam-policy-binding ${GCP_PROJECT} \
    --member=serviceAccount:dns-admin@${GCP_PROJECT}.iam.gserviceaccount.com \
    --role=roles/dns.admin

After those commands a new file is created: gcp-dns-admin.json. The next step is upload this file as a secret on cert-manager namespace.

Create cloud-dns-key secret
kubectl create secret --namespace cert-manager generic cloud-dns-key \
  --from-file=key.json=./gcp-dns-admin.json

Install Cluster Issuer and Certificate

Issuers, and ClusterIssuers, are Kubernetes resources that represent certificate authorities (CAs) that are able to generate signed certificates by honoring certificate signing requests.

— cert-manager docs
https://cert-manager.io/docs/concepts/issuer/

Cluster Issuer

cluster-issuer.yaml
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
  namespace: cert-manager
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: name@example.com

    privateKeySecretRef:
      name: letsencrypt-prod

    solvers:
    - selector: {}
      dns01:
        clouddns:
          project: project-id
          serviceAccountSecretRef:
            name: cloud-dns-key
            key: key.json

Copy the previous content in your computer with the name cluster-issues.yaml and run the following command:

Apply cluster-issuer.yaml
kubectl apply -f cluster-issuer.yaml

There are two fields that need to change here:

  • email: change field email to match to an administrator email. This is used by Let’s Encrypt to send important notices.

  • project: change field project. This is the project id where the Cloud DNS is available. If the project is the same of current Kubernetes cluster, repeat the project id here.

Certificate

cluster-issuer.yaml
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: app-certs
  namespace: app
spec:
  secretName: app-certs-tls
  issuerRef:
    kind: ClusterIssuer
    name: letsencrypt-prod

  # you can config several domains here
  dnsNames:
  - example.com
  - subdomain.example.com

Copy the previous content in your computer with the name certificate.yaml and run the following command:

Apply certificate.yaml
kubectl apply -f certificate.yaml
Once again, change the field dnsNames to match to your domain or domains.

Deployment for tests

This section will teach you how to configure a dummy service in order to test our configuration. If you have Kubernetes' deployment and service installed you can skip that.

To save our time, there’s a ready container waiting for you to be deployed. The next steps can be skipped, they are here just for further information and, of course, if you can create by yourself your own deployment and services based on them.

Go to Deploy Test Application to use ready container.

Test Application

Create Application

Creating an application
export WORK_DIRECTORY=~/tmp-deploy
mkdir -p ${WORK_DIRECTORY}
cd ${WORK_DIRECTORY}

cat <<EOF >./server.js
var http = require('http');

var handleRequest = function(request, response) {
  console.log('Received request for URL: ' + request.url);
  response.writeHead(200);
  response.end('Hello World!');
};
var www = http.createServer(handleRequest);
www.listen(8080);
EOF

Create container

Before you starting this section, you need a Docker Hub Id. Create one following this page.

Creating a container
export WORK_DIRECTORY=~/tmp-deploy
export DOCKER_ID=gbmartins
cd ${WORK_DIRECTORY}

cat <<EOF >./Dockerfile
FROM node:6.14.2
EXPOSE 8080
COPY server.js .
CMD [ "node", "server.js" ];
EOF

docker build -t ${DOCKER_ID}/node-server:1.0 .
docker push ${DOCKER_ID}/node-server:1.0

Deploy Test Application

Deployment

Save the following deployment file with the name pod.yaml.

pod.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: hello-node
  name: hello-node
  namespace: app
spec:
  template:
    metadata:
      labels:
        app: hello-node
    spec:
      containers:
      - image: gbmartins/node-server:1.0
        imagePullPolicy: IfNotPresent
        name: node-server
        resources: {}
      dnsPolicy: ClusterFirst
      restartPolicy: Always

Change namespace and container.image according to your environment and needs.

Run this command to create the deployment.

Apply pod.yaml
kubectl apply -f pod.yaml

Service

Save the following service file with the name service.yaml.

service.yaml
apiVersion: v1
kind: Service
metadata:
  name: hello-node
spec:
  selector:
    app: hello-node
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
  type: NodePort

If you modified some fields on previous steps, you need to change all fields here to match with new values.

Run this command to create the service

Apply service.yaml
kubectl apply -f service.yaml

Ingress

Now we will create an Ingress networking service.

An API object that manages external access to the services in a cluster, typically HTTP.

Ingress may provide load balancing, SSL termination and name-based virtual hosting.

— Kubernetes Ingress
https://kubernetes.io/docs/concepts/services-networking/ingress/
ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    certmanager.k8s.io/cluster-issuer: letsencrypt-prod
    certmanager.k8s.io/acme-challenge-type: dns01
    ingress.kubernetes.io/ssl-redirect: 'true'
  labels:
    name: ingress
  name: ingress
  namespace: app
spec:
  rules:
    - host: example.com
      http:
        paths:
          - backend:
              serviceName: hello-node
              servicePort: 8080
  tls:
  - hosts:
    - example.com
    secretName: app-certs-tls

You need to adapt the file according to your configurations and if you modified another fields on previous steps, you need to change the same fields here to match with new values.

Run this command to create the ingress service.

Apply ingress.yaml
kubectl apply -f ingress.yaml

DNS Configuration

Create a DNS01 challenge

Install certbot on your system.

Run the following commands. The file gcp-dns-admin.json is the same one created on Create Service Account and Secrets.

Config DNS01 with certbot
export ENDPOINT=example.com
sudo certbot certonly \
  --dns-google \
  --dns-google-credentials ./gcp-dns-admin.json \
  -d ${ENDPOINT}

Config DNS

External IP

At this point, the Ingress service should have created the load balancer and an external IP. Check this out running the command below.

Check external IP
export NAMESPACE=app
kubectl --namespace ${NAMESPACE} get ing ingress

The result of this command looks like this:

NAME      HOSTS         ADDRESS           PORTS     AGE
ingress   example.com   123.123.123.133   80, 443   56s

If you cannot see something like that, wait a couple minutes and try again.

Create a DNS Entry

In order to create a new entry on Cloud DNS, run the commands below.

Create DNS entry
export DNS_PROJECT_ID=project_id
export ZONE=example-zone
export NAME=example.com.
export IP=<from previous step>

gcloud beta dns --project=${DNS_PROJECT_ID} record-sets transaction start --zone=${ZONE}
gcloud beta dns --project=${DNS_PROJECT_ID} record-sets transaction add ${IP} \
    --name=${NAME} \
    --ttl=300 \
    --type=A \
    --zone=${ZONE}
gcloud beta dns --project=${DNS_PROJECT_ID} record-sets transaction execute --zone=${ZONE}

Test it!

Open your URL in your browser!!

Tags: Google DNS, certbot, kubernetes, k8s, gcloud, kubectl, helm, Let's Encrypt