Automated backups
It is crucial to set up automated backups for the OSIE deployment.
Luckily, there is Velero that covers backups and restores in Kubernetes.
When MongoDB is run by the Percona operator, as in the Kubernetes installation, the operator takes its own
scheduled database backups with point-in-time recovery to S3 — that is the backup block of the psmdb.ha.yml
manifest. Velero complements it by capturing the rest of the namespace, so a restore brings back the whole installation
and not only the data.
Back up to S3 with Velero
This example installs and configures Velero to back up the data and upload it to an S3 bucket every 1 hour.
Preferably that S3 bucket should be outside your current data center, so that you can ensure a disaster recovery as well.
Velero will back up pretty much all the Kubernetes resources from your namespace including:
- Persistent Volumes (data is copied using File System Backup with Restic)
- Pod, Deployment, StatefulSet, Secret, ConfigMap
- Other kubernetes CRDs such as: Certificate, etc.
This means Velero will copy not only your data (databases files, etc) but also the entire state of the kubernetes applications.
Therefore, when you will do a restore from that backup, all the Pods, ConfigMaps, Ingress, etc will come up as well.
Install Velero
-
Install the Velero CLI on your local machine. Follow the install documentation.
-
Add the Velero helm repository.
helm repo add vmware-tanzu https://vmware-tanzu.github.io/helm-charts
- Create a
velero.ymlconfiguration file. See the all settings here.
deployNodeAgent: true
nodeAgent:
privileged: true
snapshotsEnabled: false
credentials:
secretContents:
cloud: |
[default]
aws_access_key_id = <your_access_key_id>
aws_secret_access_key = <your_secret_access_key>
region = eu-central-1
initContainers:
- name: velero-plugin-for-aws
image: velero/velero-plugin-for-aws:v1.7.0
imagePullPolicy: IfNotPresent
volumeMounts:
- mountPath: /target
name: plugins
configuration:
defaultVolumesToFsBackup: true
backupStorageLocation:
- name: osie-backups
provider: aws
bucket: velero-backups-osie
prefix: osie-k8s
accessMode: ReadWrite
config:
region: eu-central-1
schedules:
hourly-backup:
disabled: false
schedule: "0 * * * *"
useOwnerReferencesInBackup: false
template:
ttl: "240h"
storageLocation: osie-backups
includedNamespaces:
- default # Here place the K8S namespaces you want to back up
The above file configures the following
- a backup storage location called "osie-backups" that represents an S3 bucket. You can define multiple storage locations if you'd like to have redundant backups.
- a backup schedule that runs every 1 hour and has 240h (10 days) data retention.
- Install and apply the Velero configuration
helm install velero vmware-tanzu/velero \
--namespace velero \
--create-namespace \
-f velero.yml
Verify the operation
velero get schedule
NAME STATUS CREATED SCHEDULE BACKUP TTL LAST BACKUP SELECTOR PAUSED
hourly-backup Enabled 2023-09-18 10:09:45 +0300 EEST 0 * * * * 240h0m0s 1h ago <none> false
After a while, backups will also show up.
velero get backup
NAME STATUS ERRORS WARNINGS CREATED EXPIRES STORAGE LOCATION SELECTOR
hourly-backup-20231107000017 Completed 0 29 2023-11-07 02:00:17 +0200 EET 10d osie-backups <none>
hourly-backup-20231107010059 Completed 0 29 2023-11-07 03:00:59 +0200 EET 10d osie-backups <none>
hourly-backup-20231107020036 Completed 0 29 2023-11-07 04:00:36 +0200 EET 10d osie-backups <none>
S3 compatible buckets
You may use other providers than AWS that also offer S3-compatible buckets, such as Scaleway or even your own S3-compatible endpoints.
In that case, the configuration can be adapted as follows.
configuration:
...
backupStorageLocation:
- name: osie-backups
...
config:
...
s3Url: https://s3.fr-par.scw.cloud
The region has also to be adapted to the one from that cloud provider.
You can visit the Backup Storage Location docs for a complete list of available parameters.
Read more
Please visit the Velero Documentation page for a more thorough understanding of the platform.