Backing Up Rancher

Pre-Requisites

  • A device capable of being an NFS (or S3) server. Any linux device will do (including a raspberry pi), but NFS servers require some command-line fu to get operational. We will use a synology nas as an example, which makes the nfs server creation process incredibly easy.

TL;DR

In this article, we will:

  • Set up a NFS server to target our backups
  • Redeploy longhorn to automatically generate backups on PVC creation
  • Set up rancher to generate configuration backups to longhorn
  • Navigate the longhorn management console to verify backups are operational

Introduction

Many people confuse high availability with backups, as often as they confuse RAID with backups. High Availability is not about backups: it’s about uptime and scalability. You still need backups and retention. In a single node cluster, you especially need backups. Luckily longhorn makes this process fairly painless (as long as you have a NFS or S3 target). Let’s set that up now.

Configuring the NFS Server

There’s many ways to set up an NFS server, including on a Raspberry Pi. For this guide however we will use a synology NAS. If you do not have a synology NAS you will have to find another method of creating a NFS server.

  • Log into your synology nas and check file services→nfs. Make sur eyou have nfs and nfs4 enabled. Use your own domain of course.

  • created a new shared drive. I’m going to call it longhorn-backups.

  • Work through the shared folder creation. Under NFS Permissions, create a NFS Rule and set the hostname to your rancher host’s hostname (or IP). Make a note of the Mount Path as well: we will need that.

Relaunching Longhorn

Let’s relaunch longhorn to automatically back up to the NFS server.

  • Under apps and marketplace, find the longhorn installed app (you will have to uncheck the show only user namespaces filter). Choose edit/upgrade.

  • Under Longhorn Storage Class Settings, set the Enable Storage Class Recurring Job and set to:
[{"name":"backup", "task":"backup", "cron":"0 0 * * *", "retain":20,"labels": {"interval":"daily"}}]

This will tell any newly created volumes to automatically back up daily to the NFS target and retain 20 days of backups.

Press update

Testing Backups in the Longhorn Manager

On the sidebar you can find the longhorn category. Launch longhorn now.

The longhorn manager lets you manually create persistent volumes (if you wish, as well as view the status of your backups, snapshots, and volumes.

under settings, find backup target. Set the backup target to nfs://:/volume1/longhorn-backups (don’t forget that extra colon). Save at the bottom.

under volume, let’s do a test backup (new volumes will have a schedule created automatically, existing volumes will not). select the pvc bound to helloworld (there should only be one) and press create backup.

If all goes well, you should now have a backup on your NAS! Furthermore, any future volumes created (via PVC) will automatically have a daily backup schedule!

Backups made by longhorn are considered atomic, which means that you can back up running containers without fear of data corruption. Longhorn also has a snapshotting feature that you can enable if you like.

Recurring Rancher Configuration Backups

We are still not backing up our rancher configuration, which is important if we want to migrate rancher.

  • Under Apps and Marketplace, find the Rancher Backups application

  • Press install and next. Under Default Storage Location choose an exsiting storage class with longhorn. Install.

  • On the left sidebar, you should now have an option called rancher backups. Select that now and create a backup.

  • Set the name to rancher-backups and schedule to 0 0 * * *

that number notation is known as a cron schedule. You can read up more about that here.

  • Head back to the longhorn manager, and you should see that there is now a new volume, and the volume automatically backs up every day! This volume will house the configuration backups that Rancher does.

Moving from Storage to Networking

As of now, we have our storage needs sorted. We can request a volume whenever we create a deployment and that volume gets created in the longhorn engine. Furthermore any volumes created will have backup and retention settings to our NAS! By default we have set a daily schedule, but 1 hour or even 15 minute increments are perfectly doable.

With storage out of the way, we have a problem. We have been publishing our services on ports in the 30000 range. That’s not a normal port range. How do we get our services on a normal port like 80 or 443? How do we encrypt those services. Find out in Ingresses