> ## Documentation Index > Fetch the complete documentation index at: https://docs.siderolabs.com/llms.txt > Use this file to discover all available pages before exploring further. # Create Etcd Backups > Configure S3 backup storage, create manual backups, and enable automatic etcd backups for Omni-managed clusters. Etcd is the key-value store that holds all Kubernetes cluster state. If etcd becomes corrupted or unavailable, your cluster will stop functioning. Omni supports creating etcd backups so that you can restore a cluster to a known good state if something goes wrong. Omni supports two backup storage backends: `local` and `s3`. The backend is selected during Omni initialization. This guide covers how to configure S3 backup storage, create a manual backup, and enable automatic backups. To restore a cluster from a backup, see [Restore Etcd of a Cluster Managed by Cluster Templates](./restore-etcd-of-a-cluster-managed-by-cluster-templates). ## Check backup status Before configuring backups, check the current state of the backup subsystem: ```bash theme={null} omnictl get etcdbackupoverallstatus ``` On a freshly created Omni instance, the output will look similar to this: ```bash theme={null} NAMESPACE TYPE ID VERSION CONFIGURATION NAME CONFIGURATION ERROR LAST BACKUP STATUS LAST BACKUP ERROR LAST BACKUP TIME CONFIGURATION ATTEMPT ephemeral EtcdBackupOverallStatus etcdbackup-overall-status 1 s3 not initialized ``` The key fields to pay attention to are: | Field | Description | | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------ | | `CONFIGURATION NAME` | The backup backend currently configured (`s3` or `local`). | | `CONFIGURATION ERROR` | Any error with the current backup configuration. `not initialized` means the backend has been selected but not yet configured. | | `LAST BACKUP STATUS` | The result of the most recent backup attempt. | | `LAST BACKUP ERROR` | Any error from the most recent backup attempt. | | `LAST BACKUP TIME` | When the most recent backup completed. | | `CONFIGURATION ATTEMPT` | The number of times Omni has attempted to apply the backup configuration. | In the output above, `CONFIGURATION NAME` is `s3` and `CONFIGURATION ERROR` shows `not initialized`, which means the S3 backend has been selected but the credentials have not yet been configured. The remaining fields are empty because no backups have been created yet. ## Configure S3 backup storage Omni stores etcd backups in an S3-compatible bucket. Before creating any backups, you need to provide Omni with the credentials and configuration it needs to access your bucket. This section covers compatibility considerations for non-AWS providers and how to apply the S3 configuration. ### S3-compatible providers Omni uses the AWS S3 API for storing etcd backups. Non-AWS providers that advertise S3 compatibility may work when configured with a custom `endpoint`, but Omni does not maintain a compatibility list for third-party providers. If you use a non-AWS S3 provider: * Test backup and restore workflows before relying on them in production. * Some providers may require additional configuration, such as a `region` value or provider-specific credentials. * Ensure the endpoint and credentials are valid and accessible from Omni. During configuration, Omni validates access by attempting to list objects in the bucket. If validation fails, the configuration is not applied. ### Apply the S3 configuration To configure S3 as the backup backend, create an `EtcdBackupS3Configs.omni.sidero.dev` resource. The following example configures Omni to use a MinIO S3 instance, edit this example to fit your S3 instance: ```yaml theme={null} cat < etcd-backup-s3-conf.yaml metadata: namespace: default type: EtcdBackupS3Configs.omni.sidero.dev id: etcd-backup-s3-conf spec: bucket: mybucket region: us-east-1 endpoint: http://127.0.0.1:9000 accesskeyid: access secretaccesskey: secret123 sessiontoken: "" EOF ``` The fields in the `spec` are: | Field | Required | Description | | ----------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | `bucket` | Yes | The name of the S3 bucket to store backups in. | | `region` | No | The S3 bucket region. Defaults to the AWS default region if not set. | | `endpoint` | No | The S3 endpoint URL. Defaults to the AWS S3 endpoint if not set. Required for non-AWS providers. | | `accesskeyid` | No | The access key ID for the S3 bucket. If not set, Omni assumes it is running on an EC2 instance with an IAM role that has access to the bucket. | | `secretaccesskey` | No | The secret access key for the S3 bucket. | | `sessiontoken` | No | A session token for temporary credentials, if required. | Apply the S3 configuration: ```bash theme={null} omnictl apply -f etcd-backup-s3-conf.yaml ``` Omni validates the credentials during resource creation by attempting to list objects in the bucket. If validation fails, the resource will not be created and an error will be returned. To confirm the configuration was applied successfully, check the backup status again: ```bash theme={null} omnictl get etcdbackupoverallstatus ``` The `CONFIGURATION ERROR` field should now be empty: ```bash theme={null} NAMESPACE TYPE ID VERSION CONFIGURATION NAME CONFIGURATION ERROR LAST BACKUP STATUS LAST BACKUP ERROR LAST BACKUP TIME CONFIGURATION ATTEMPT ephemeral EtcdBackupOverallStatus etcdbackup-overall-status 2 s3 ``` ## Create a manual backup To trigger a backup immediately, create and apply an `EtcdManualBackups.omni.sidero.dev` resource. The `` value must be no more than one minute in the future or past. The `nanos` field must always be `0`. 1. Run the following command, replacing `` with the name of your cluster: ```bash theme={null} cat < etcd-manual-backup.yaml metadata: namespace: ephemeral type: EtcdManualBackups.omni.sidero.dev id: spec: backupat: seconds: $(date +%s) nanos: 0 EOF ``` 2. Apply the resource: ```bash theme={null} omnictl apply -f etcd-manual-backup.yaml ``` 3. After a few seconds, check the per-cluster backup status: ```bash theme={null} omnictl get etcdbackupstatus -o yaml ``` The output will look similar to this: ```yaml theme={null} metadata: namespace: ephemeral type: EtcdBackupStatuses.omni.sidero.dev id: version: 1 owner: EtcdBackupController phase: running spec: status: 1 error: "" lastbackuptime: seconds: 1702166400 nanos: 985220192 lastbackupattempt: seconds: 1702166400 nanos: 985220192 ``` 4. To check the overall backup subsystem status: ```bash theme={null} omnictl get etcdbackupoverallstatus -o yaml ``` ## Enable automatic backups Omni supports scheduled automatic backups. You can configure the backup interval using either cluster templates or by editing the cluster resource directly. To enable automatic etcd backups using a cluster template: 1. Add a `features.backupConfiguration` block to your cluster template: ```yaml theme={null} cat < etcd-backup.yaml kind: Cluster name: talos-default kubernetes: version: v1.28.2 talos: version: v1.5.5 # add the backupConfiguration block to your cluster templates features: backupConfiguration: interval: 1h --- kind: ControlPlane machines: - # control plane machine uuid --- kind: Workers machines: - # worker machine uuid - # worker machine uuid EOF ``` Replace ``, ``, and `` with the UUIDs of your control plane and worker machines. For more information on configuring cluster features with cluster templates, see the [Cluster Template reference documentation](../reference/cluster-templates). 2. Before applying, check whether automatic backups are already enabled on the cluster: ```bash theme={null} omnictl cluster template -f etcd-backup.yaml diff ``` If automatic backups are **not** yet enabled, the output will show the backup configuration being added: ```bash theme={null} --- Clusters.omni.sidero.dev(default/talos-default) +++ Clusters.omni.sidero.dev(default/talos-default) @@ -19,4 +19,7 @@ features: enableworkloadproxy: false diskencryption: false - backupconfiguration: null + backupconfiguration: + interval: + seconds: 3600 + nanos: 0 ``` If automatic backups are **already enabled**, the diff will show an existing `backupconfiguration` block with a non-null value. In that case, you can update the `interval` value in your template and re-run the diff to confirm the change before applying, **or** skip to step 4 to verify the existing backup schedule is working as expected. 3. Apply the template: ```bash theme={null} omnictl cluster template -f etcd-backup.yaml sync ``` If no backups exist for the cluster yet, Omni will create one immediately rather than waiting for the first scheduled interval. 4. Verify that the backup was created by running the following command, replacing `` with the name of your cluster: ```bash theme={null} omnictl get etcdbackup --selector omni.sidero.dev/cluster= ``` This command will output something similar to this: ```bash theme={null} NAMESPACE TYPE ID VERSION CREATED AT external EtcdBackup talos-default-1702166400 undefined {"nanos":0,"seconds":1702166400} ``` If your cluster is not managed by cluster templates, you can enable automatic backups by editing the cluster resource directly. To enable automatic etcd backups by editing the cluster resource directly: 1. Retrieve the current cluster resource, replace the `` placeholder with the name of your cluster: ```bash theme={null} omnictl get cluster -o yaml > cluster-backup.yaml ``` 2. In the `cluster-backup.yaml` file, add the `backupconfiguration` block while preserving all existing fields: ```yaml theme={null} # cluster-backup.yaml metadata: namespace: default type: Clusters.omni.sidero.dev id: talos-default finalizers: - KubernetesUpgradeStatusController - TalosUpgradeStatusController - SecretsController - ClusterController spec: installimage: "" kubernetesversion: 1.28.2 talosversion: 1.5.5 features: enableworkloadproxy: false diskencryption: false # add the backupconfiguration block backupconfiguration: interval: seconds: 3600 nanos: 0 ``` 3. Apply the updated file: ```bash theme={null} omnictl apply -f cluster-backup.yaml ``` ## Next steps To restore a cluster from an etcd backup, see [Restore Etcd of a Cluster Managed by Cluster Templates](./restore-etcd-of-a-cluster-managed-by-cluster-templates).