Skip to main content

Automated backups

It is crucial to set up automated backups for the OSIE deployment.
Luckily, there is Velero that covers backups and restores in Kubernetes.

Database backups are configured separately

When MongoDB is run by the Percona operator, as in the Kubernetes installation, the operator takes its own scheduled database backups with point-in-time recovery to S3 — that is the backup block of the psmdb.ha.yml manifest. Velero complements it by capturing the rest of the namespace, so a restore brings back the whole installation and not only the data.

Back up to S3 with Velero​

This example installs and configures Velero to back up the data and upload it to an S3 bucket every 1 hour.
Preferably that S3 bucket should be outside your current data center, so that you can ensure a disaster recovery as well.
Velero will back up pretty much all the Kubernetes resources from your namespace including:

  • Persistent Volumes (data is copied using File System Backup with Restic)
  • Pod, Deployment, StatefulSet, Secret, ConfigMap
  • Other kubernetes CRDs such as: Certificate, etc.

This means Velero will copy not only your data (databases files, etc) but also the entire state of the kubernetes applications.
Therefore, when you will do a restore from that backup, all the Pods, ConfigMaps, Ingress, etc will come up as well.

Install Velero​

  1. Install the Velero CLI on your local machine. Follow the install documentation.

  2. Add the Velero helm repository.

helm repo add vmware-tanzu https://vmware-tanzu.github.io/helm-charts
  1. Create a velero.yml configuration file. See the all settings here.
deployNodeAgent: true
nodeAgent:
privileged: true
snapshotsEnabled: false
credentials:
secretContents:
cloud: |
[default]
aws_access_key_id = <your_access_key_id>
aws_secret_access_key = <your_secret_access_key>
region = eu-central-1
initContainers:
- name: velero-plugin-for-aws
image: velero/velero-plugin-for-aws:v1.7.0
imagePullPolicy: IfNotPresent
volumeMounts:
- mountPath: /target
name: plugins
configuration:
defaultVolumesToFsBackup: true
backupStorageLocation:
- name: osie-backups
provider: aws
bucket: velero-backups-osie
prefix: osie-k8s
accessMode: ReadWrite
config:
region: eu-central-1
schedules:
hourly-backup:
disabled: false
schedule: "0 * * * *"
useOwnerReferencesInBackup: false
template:
ttl: "240h"
storageLocation: osie-backups
includedNamespaces:
- default # Here place the K8S namespaces you want to back up

The above file configures the following

  • a backup storage location called "osie-backups" that represents an S3 bucket. You can define multiple storage locations if you'd like to have redundant backups.
  • a backup schedule that runs every 1 hour and has 240h (10 days) data retention.
  1. Install and apply the Velero configuration
helm install velero vmware-tanzu/velero \
--namespace velero \
--create-namespace \
-f velero.yml

Verify the operation​

velero get schedule
NAME                 STATUS    CREATED                          SCHEDULE     BACKUP TTL   LAST BACKUP   SELECTOR   PAUSED
hourly-backup Enabled 2023-09-18 10:09:45 +0300 EEST 0 * * * * 240h0m0s 1h ago <none> false

After a while, backups will also show up.

velero get backup
NAME STATUS ERRORS WARNINGS CREATED EXPIRES STORAGE LOCATION SELECTOR
hourly-backup-20231107000017 Completed 0 29 2023-11-07 02:00:17 +0200 EET 10d osie-backups <none>
hourly-backup-20231107010059 Completed 0 29 2023-11-07 03:00:59 +0200 EET 10d osie-backups <none>
hourly-backup-20231107020036 Completed 0 29 2023-11-07 04:00:36 +0200 EET 10d osie-backups <none>

S3 compatible buckets​

You may use other providers than AWS that also offer S3-compatible buckets, such as Scaleway or even your own S3-compatible endpoints.
In that case, the configuration can be adapted as follows.

configuration:
...
backupStorageLocation:
- name: osie-backups
...
config:
...
s3Url: https://s3.fr-par.scw.cloud

The region has also to be adapted to the one from that cloud provider.
You can visit the Backup Storage Location docs for a complete list of available parameters.

Read more​

Please visit the Velero Documentation page for a more thorough understanding of the platform.