From bdff60178ad288fa4e8e1089c87192e2edf2fd9e Mon Sep 17 00:00:00 2001 From: Lyndon-Li Date: Tue, 20 Aug 2024 14:24:32 +0800 Subject: [PATCH] add doc for backup repo config Signed-off-by: Lyndon-Li --- changelogs/unreleased/8131-Lyndon-Li | 1 + .../main/backup-repository-configuration.md | 50 +++++++++++++++++++ .../docs/main/csi-snapshot-data-movement.md | 3 +- site/content/docs/main/file-system-backup.md | 3 +- 4 files changed, 55 insertions(+), 2 deletions(-) create mode 100644 changelogs/unreleased/8131-Lyndon-Li create mode 100644 site/content/docs/main/backup-repository-configuration.md diff --git a/changelogs/unreleased/8131-Lyndon-Li b/changelogs/unreleased/8131-Lyndon-Li new file mode 100644 index 0000000000..4616c3ba00 --- /dev/null +++ b/changelogs/unreleased/8131-Lyndon-Li @@ -0,0 +1 @@ +Fix issue #7620, add doc for backup repo config \ No newline at end of file diff --git a/site/content/docs/main/backup-repository-configuration.md b/site/content/docs/main/backup-repository-configuration.md new file mode 100644 index 0000000000..0e1d6343cd --- /dev/null +++ b/site/content/docs/main/backup-repository-configuration.md @@ -0,0 +1,50 @@ +--- +title: "Backup Repository Configuration" +layout: docs +--- + +Velero uses selectable backup repositories for various backup/restore methods, i.e., [file-system backup][1], [CSI snapshot data movement][2], etc. To achieve the best performance, backup repositories may need to be configured according to the running environments. + +Velero uses a BackupRepository CR to represent the instance of the backup repository. Now, a new field `repositoryConfig` is added to support various configurations to the underlying backup repository. + +Velero also allows you to specify configurations before the BackupRepository CR is created through a configMap. The configurations in the configMap will be copied to the BackupRepository CR when it is created at the due time. +The configMap should be in the same namespace where Velero is installed. If multiple Velero instances are installed in different namespaces, there should be one configMap in each namespace which applies to Velero instance in that namespace only. The name of the configMap should be specified in the Velero server parameter `--backup-repository-config`. + +Conclusively, you have two ways to add/change/delete configurations of a backup repository: +- If the BackupRepository CR for the backup repository is already there, you should modify the `repositoryConfig` field. The new changes will be applied to the backup repository at the due time, it doesn't require Velero server to restart. +- Otherwise, you can create the backup repository configMap as a template for the BackupRepository CRs that are going to be created. + +The backup repository configMap is repository type specified, so for one repository type, you only need to create one set of configurations, they will be applied to all BackupRepository CRs of the same type. Whereas, the changes of `repositoryConfig` field apply to the specific BackupRepository CR only, you may need to change every BackupRepository CR of the same type. + +Below is an example of the BackupRepository configMap with the configurations: +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: + namespace: velero +data: + : | + { + "cacheLimitMB": 2048 + } + : | + { + "cacheLimitMB": 1024 + } +``` + +To create the configMap, you need to save something like the above sample to a file and then run below commands: +```shell +kubectl apply -f +``` + +When and how the configurations are used is decided by the backup repository itself. Though you can specify any configuration to the configMap or `repositoryConfig`, the configuration may/may not be used by the backup repository, or the configuration may be used at an arbitrary time. + +Below is the supported configurations by Velero and the specific backup repository. +***Kopia repository:*** +`cacheLimitMB`: specifies the size limit(in MB) for the local data cache. The more data is cached locally, the less data may be downloaded from the backup storage, so the better performance may be achieved. Practically, you can specify any size that is smaller than the free space so that the disk space won't run out. This parameter is for repository connection, that is, you could change it before connecting to the repository. E.g., before a backup/restore/maintenance. + + +[1]: file-system-backup.md +[2]: csi-snapshot-data-movement.md \ No newline at end of file diff --git a/site/content/docs/main/csi-snapshot-data-movement.md b/site/content/docs/main/csi-snapshot-data-movement.md index 60d72aecf9..621997c53f 100644 --- a/site/content/docs/main/csi-snapshot-data-movement.md +++ b/site/content/docs/main/csi-snapshot-data-movement.md @@ -481,7 +481,7 @@ For Velero built-in data mover, Velero uses [BestEffort as the QoS][13] for node If you want to constraint the CPU/memory usage, you need to [customize the resource limits][11]. The CPU/memory consumption is always related to the scale of data to be backed up/restored, refer to [Performance Guidance][12] for more details, so it is highly recommended that you perform your own testing to find the best resource limits for your data. During the restore, the repository may also cache data/metadata so as to reduce the network footprint and speed up the restore. The repository uses its own policy to store and clean up the cache. -For Kopia repository, the cache is stored in the node-agent pod's root file system and the cleanup is triggered for the data/metadata that are older than 10 minutes (not configurable at present). So you should prepare enough disk space, otherwise, the node-agent pod may be evicted due to running out of the ephemeral storage. +For Kopia repository, the cache is stored in the node-agent pod's root file system. Velero allows you to configure a limit of the cache size so that the data mover pod won't be evicted due to running out of the ephemeral storage. For more details, check [Backup Repository Configuration][16]. ### Node Selection @@ -509,3 +509,4 @@ For the restore, this is not supported because sometimes the data movement resto [13]: https://kubernetes.io/docs/concepts/workloads/pods/pod-qos/ [14]: node-agent-concurrency.md [15]: data-movement-backup-node-selection.md +[16]: backup-repository-configuration.md diff --git a/site/content/docs/main/file-system-backup.md b/site/content/docs/main/file-system-backup.md index 5dac5e3c53..a2d81461ca 100644 --- a/site/content/docs/main/file-system-backup.md +++ b/site/content/docs/main/file-system-backup.md @@ -642,7 +642,7 @@ Velero node-agent uses [BestEffort as the QoS][14] for node-agent pods (so no CP If you want to constraint the CPU/memory usage, you need to [customize the resource limits][15]. The CPU/memory consumption is always related to the scale of data to be backed up/restored, refer to [Performance Guidance][16] for more details, so it is highly recommended that you perform your own testing to find the best resource limits for your data. During the restore, the repository may also cache data/metadata so as to reduce the network footprint and speed up the restore. The repository uses its own policy to store and clean up the cache. -For Kopia repository, the cache is stored in the node-agent pod's root file system and the cleanup is triggered for the data/metadata that are older than 10 minutes (not configurable at present). So you should prepare enough disk space, otherwise, the node-agent pod may be evicted due to running out of the ephemeral storage. +For Kopia repository, the cache is stored in the node-agent pod's root file system. Velero allows you to configure a limit of the cache size so that the node-agent pod won't be evicted due to running out of the ephemeral storage. For more details, check [Backup Repository Configuration][18]. ## Restic Deprecation @@ -695,3 +695,4 @@ In the output of `velero backup describe` command for a backup with fs-backup: [15]: customize-installation.md#customize-resource-requests-and-limits [16]: performance-guidance.md [17]: https://github.com/vmware-tanzu/velero/blob/main/GOVERNANCE.md#deprecation-policy +[18]: backup-repository-configuration.md