Let's Encrypt Certificates - DNS01 Challenge

Pre-Requisites

To make a DNS challenge work, you’ll need one big pre-requisite:

  • A Publicly registered domain with a domain registrar. Specifically, one on this list (if you want it done automatically). If not, you can register for a free cloudflare account and forward your DNS nameservers to cloudflare.

TL;DR

In this article, we will:

  • Set up staging certificates with let’s encrypt
  • Convert the staging certificates to trusted production certificates

What is a Let’s Encrypt DNS Challenge?

A DNS Challenge is where you tell Let’s Encrypt, “I would like a certificate that covers my entire domain”. And Let’s Encrypt says, “Here’s a big text file to stick in your DNS registrar for us to read and validate your domain”. Once you do that, and once they read it, they give you a certificate that covers your entire domain (referred to as a wildcard certificate) for 3 months.

A DNS challenge is really the best option if you have the right pre-requisites. You don’t have to expose any services at all for it to work, which is especially important if you are stuck without a public IPv4 address. You also only need to request one certificate, and you can reuse that certificate for all of your services.

Registering your Cert Provider

Where we left off, we had just installed cert-manager. Now we have to register a key from our DNS provider. For this guide we will be using cloudflare, following this guide.

  • Log in to cloudflare and go to User Profile > API Tokens > API Tokens
  • Create a token with the following permissions:
    • Zone - DNS - EDIT
    • Zone - Zone - Read
    • Include All Zones

  • Take the key that gets generated, and go to Rancher→Secrets→Create

  • Create an opaque secret for the cert-manager namespace. Fill out the token with a name of cloudflare-api-token-secret, a key of api-token, and the token you copied from cloudflare

Registering a Staging Issuer

Rancher does not have fillable forms for cert-manager based instructions. Instead, we will fall back to the default way of applying settings: Kubernetes Manifests!

Different providers have different structures for Kubernetes Manifests. You can read the documentation for cert-manager manifests on their documentation.

  • At the top of the rancher interface (regardless of what you have open in the sidebar) there is an Import YAML option. Open that now, and paste the following, changing:
    • The acme email to your email
    • cloudflare email to your cloudflare email
    • the namespace to your namespace (if you didn’t name it homelab).
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    email: myemail@mydomain.com.au
    privateKeySecretRef:
      name: letsencrypt-staging
    solvers:
      - dns01:
          cloudflare:
            apiTokenSecretRef:
              key: api-token
              name: cloudflare-api-token-secret
            email: myemail@mydomain.com.au

Once you have done that, you can clear your namespace filters, and find the status of our manifest in more resources→cert-manager→ClusterIssuers. You should see a registered status for your issuer:

Generating a Staging Certificate

  • Now that we have an issuer configured with an api token, we need to create a certificate. Import the following YAML, changing:
    • mydomain to your domain
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: mydomain-staging
  namespace: homelab
spec:
  secretName: mydomain-staging
  issuerRef:
    name: letsencrypt-staging
    kind: ClusterIssuer
  dnsNames:
  - '*.mydomain.com.au'

Alright! If all goes well, that’s enough to generate your certificate. You should get a nice active under more resources→cert manager→certificates after a while (give it up to 10 minutes, but typically within 30 seconds).

if you do not get active for your certificate, you can investigate what went wrong by looking up certificaterequests on the side panel.

  • Head back to ingresses and edit config on the helloworld ingress. You should now have a -staging certificate available.

Change to that and hit Save. Check your site, and you should be getting a new certificate from R3! It will still be invalid, which is fine: we start with staging to make sure our config is right and we don’t get ratelimited by Let’s Encrypt.

Registering a Production Issuer

Now that we have configured a functional workflow, we can move from staging to production.

  • Head back to issuers on the sidebar and clone the staging issuer:

  • Set the name and privateKeySecretRef name to letsencrypt-production. Set the server URL to https://acme-v02.api.letsencrypt.org/directory

  • Do the same thing for certificates

  • Once the production certificate becomes active, we can go back to Ingresses and swap out for the production certificate. Then we can test and see if we are getting a wildcard certificate! You can see me do this below:

Hooray! We can now use this certificate for any subdomain of the same domain. If you wish, you can also go delete the staging certificate from certficates, the staging issuer from ClusterIssuers, and the staging secret from secrets.

We can also set up additional issuers (or use a single issuer with selectors) to certify other domains. Cert-manager will automatically monitor the certificates and renew them, nothing to be done on your end. Fantastic!

Changing Rancher’s certificate to Let’s Encrypt

We can also create a valid certificate for our rancher installation. This is especially important if our rancher install is using the same domain as our let’s encrypt wildcard. Chrome will install stricter SSL enforcement (known as HSTS) which will break rancher’s self signed certificate after visiting a site using our let’s encrypt certificate.

you can temporarily get around this problem by using an incognito or guest profile in chrome

  • clear the namespace filters and navigate to More Resources→K3S→Helm Charts. Edit YAML on rancher

  • Add the following to the valuesContent section:
ingress:
  tls:
    source: secret
  extraAnnotations:
    nginx.ingress.kubernetes.io/whitelist-source-range: "127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"

The extraAnnotations section is optional, but useful if you want to restrict who can access your rancher web interface. You may even want to lock down the interface to a smaller subnet.

  • Under cert manager→certificates, delete the existing tls-rancher-ingress certificate

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: tls-rancher-ingress
  namespace: cattle-system
spec:
  secretName: tls-rancher-ingress
  commonName: kub01.mydomain.com.au
  dnsNames:
  - kub01.mydomain.com.au
  issuerRef:
    name: letsencrypt-production
    kind: ClusterIssuer

  • Finally under storage→secrets, delete the existing tls-rancher-ingress secret to regenerate a lets encrypt version:

  • If all goes well, after 30 seconds to 5 minutes, you can access your rancher interface with a let’s encrypt certificate!

As a side benefit, you have also added additional security measures for accessing your rancher web interface via the IP whitelist!