[Yandex Cloud documentation](../../index.md) > [Yandex Managed Service for Kubernetes](../index.md) > [Tutorials](index.md) > Backups > Backing up to Object Storage

# Backing up a Managed Service for Kubernetes cluster to Object Storage


The Yandex Cloud infrastructure provides secure storage and replication for data in [Managed Service for Kubernetes clusters](../concepts/index.md#kubernetes-cluster). However, you can back up data from [Managed Service for Kubernetes cluster node groups](../concepts/index.md#node-group) at any time and store them in [Yandex Object Storage](../../storage/index.md) or other types of storage.

You can create backups of Managed Service for Kubernetes cluster node group data using [Velero](https://velero.io/). It supports Yandex Cloud [disks](../../compute/concepts/disk.md) through the Kubernetes CSI driver and helps create [disk](../../compute/concepts/snapshot.md) and [volume snapshots](../concepts/volume.md).

If installed manually, Velero allows you to use [nfs](https://kubernetes.io/docs/concepts/storage/volumes/#nfs), [emptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir), [local](https://kubernetes.io/docs/concepts/storage/volumes/#local), or any other volume type without built-in support for snapshots. To use one of these volume types, install Velero with the [restic plugin](https://velero.io/docs/v1.8/restic/). Velero installed from [Cloud Marketplace](https://yandex.cloud/en/marketplace/products/yc/velero-yc-csi) does not include the `restic` plugin.

In this article, you will learn how to create a backup of a Managed Service for Kubernetes cluster node group using Velero, save it in Object Storage, and restore it in a different cluster’s node group:
1. [Create a backup of your Managed Service for Kubernetes node group](#backup).
1. [Restore a node group of another Managed Service for Kubernetes cluster from a backup](#restore).

If you no longer need the resources you created, [delete them](#clear-out).


## Required paid resources {#paid-resources}

The support cost for this solution includes:

* Fee for each Managed Service for Kubernetes cluster: using the master and outgoing traffic (see [Managed Service for Kubernetes pricing](../pricing.md)).
* VM fee (two node groups and a VM for managing a cluster with no public access): using computing resources, OS, and storage (see [Compute Cloud pricing](../../compute/pricing.md)).
* Fee for public IP addresses for the cluster nodes (see [Virtual Private Cloud pricing](../../vpc/pricing.md#prices-public-ip)).
* Fee for an Object Storage bucket: data storage and operations with data (see [Object Storage pricing](../../storage/pricing.md)).


## Getting started {#before-you-begin}

### Set up your infrastructure {#deploy-infrastructure}

{% list tabs group=instructions %}

- Manually {#manual}

  1. [Create security groups](../operations/connect/security-groups.md) for the Managed Service for Kubernetes cluster and its node groups.

        You must configure these security groups for each cluster and node group.

        {% note warning %}
        
        The configuration of security groups determines performance and availability of the cluster and the services and applications running in it.
        
        {% endnote %}

  1. [Create two Managed Service for Kubernetes clusters](../operations/kubernetes-cluster/kubernetes-cluster-create.md). When creating, specify the preconfigured security groups.

     One Managed Service for Kubernetes cluster is for creating a node group backup and the other one, for restoring data from the backup.

     If you intend to use your clusters within the Yandex Cloud network, there is no need to allocate a public IP address to them. To allow connections from outside the network, assign a public IP to the clusters.

  1. [Create a node group](../operations/node-group/node-group-create.md) in each cluster. When creating node groups, allocate each group a public IP address and specify the security groups prepared earlier.
  1. [Create a bucket in Object Storage](../../storage/operations/buckets/create.md).
  1. [Create a service account](../../iam/operations/sa/create.md) with the `compute.admin` [role](../../iam/concepts/access-control/roles.md) for the [folder](../../resource-manager/concepts/resources-hierarchy.md#folder) to work with Velero.
  1. Grant the [service account](../../iam/concepts/users/service-accounts.md) **READ and WRITE** permissions to a [bucket](../../storage/concepts/bucket.md) in Object Storage. To do this, [configure the bucket ACL](../../storage/operations/buckets/edit-acl.md).
  1. [Create a static access key](../../iam/operations/authentication/manage-access-keys.md#create-access-key) for the service account and save its ID and value. This is the only time you can copy this key as it will not be shown again.

- Terraform {#tf}

  1. If you do not have Terraform yet, [install it](../../tutorials/infrastructure-management/terraform-quickstart.md#install-terraform).
  1. [Get the authentication credentials](../../tutorials/infrastructure-management/terraform-quickstart.md#get-credentials). You can add them to environment variables or specify them later in the provider configuration file.
  1. [Configure and initialize a provider](../../tutorials/infrastructure-management/terraform-quickstart.md#configure-provider). There is no need to create a provider configuration file manually, you can [download it](https://github.com/yandex-cloud-examples/yc-terraform-provider-settings/blob/main/provider.tf).
  1. Place the configuration file in a separate working directory and [specify the parameter values](../../tutorials/infrastructure-management/terraform-quickstart.md#configure-provider). If you did not add the authentication credentials to environment variables, specify them in the configuration file.

  1. Download the [velero-backup.tf](https://github.com/yandex-cloud-examples/yc-mk8s-velero-backup/blob/main/velero-backup.tf) configuration file to the same working directory.

     This file describes:
     * [Network](../../vpc/concepts/network.md#network).
     * [Subnet](../../vpc/concepts/network.md#subnet).
     * Two Managed Service for Kubernetes clusters and their node groups.
     * [Service account](../../iam/concepts/users/service-accounts.md) required to use Managed Service for Kubernetes clusters and node groups.
     * [Security groups](../../vpc/concepts/security-groups.md) which contain [rules](../operations/connect/security-groups.md) required for the Managed Service for Kubernetes cluster and its node groups.

        Both clusters will use these security groups.

        {% note warning %}
        
        The configuration of security groups determines performance and availability of the cluster and the services and applications running in it.
        
        {% endnote %}

     * Service account with the `compute.admin` [role](../../iam/concepts/access-control/roles.md) to work with Velero.
     * Static access key for the service account used to work with Velero.
     * [Bucket](../../storage/concepts/bucket.md) in Object Storage.
  1. In the `velero-backup.tf` file, specify the following:
     * `folder_id`: [ID of the folder](../../resource-manager/operations/folder/get-id.md) to create resources in.
     * `k8s_version`: [Kubernetes version](../concepts/release-channels-and-updates.md) 1.22 or higher.
     * `zone_a_v4_cidr_blocks`: CIDR of the subnet to host the Managed Service for Kubernetes clusters.
     * `sa_name_k8s`: Name of the Managed Service for Kubernetes cluster service account.
     * `sa_name_velero`: Name of the service account to work with Velero.
     * `storage_sa_id`: ID of the [service account](../../iam/concepts/users/service-accounts.md) with the `storage.admin` role. It will be used to create a bucket in Object Storage with `READ and WRITE` permissions in the [ACL](../../storage/concepts/acl.md) for the `sa_name_velero` service account.
     * `bucket_name`: Bucket name in Object Storage.
  1. Run the `terraform init` command in the directory with the configuration file. This command initializes the provider specified in the configuration files and enables you to use its resources and data sources.
  1. Validate your Terraform configuration files using this command:

     ```bash
     terraform validate
     ```

     Terraform will display any configuration errors detected in your files.
  1. Create the required infrastructure:

     1. Run this command to view the planned changes:
     
        ```bash
        terraform plan
        ```
     
        If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
     
     1. If everything looks correct, apply the changes:
        1. Run this command:
     
           ```bash
           terraform apply
           ```
     
        1. Confirm updating the resources.
        1. Wait for the operation to complete.

     All the required resources will be created in the specified folder. You can check resource availability and their settings in the [management console](https://console.yandex.cloud).

  1. Get and save the ID and value of the static access key for the service account used to work with Velero.
     * Key ID:

       ```bash
       terraform output -raw access_key
       ```

     * Key value:

       ```bash
       terraform output -raw secret_key
       ```

{% endlist %}

### Configure additional settings {#additional-settings}

1. Select a [Velero client](https://github.com/vmware-tanzu/velero/releases) for your platform based on the [compatibility table](https://github.com/vmware-tanzu/velero#velero-compatibility-matrix).
1. Download the Velero client, extract the archive contents, and install it. For more information about installation, see [this Velero guide](https://velero.io/docs/main/basic-install/#install-the-cli).
1. Make sure the Velero client is installed by running this command:

   ```bash
   velero version
   ```

   Result:

   ```text
   Client:
           Version: v1.10.3
           Git commit: 18ee078dffd9345df610e0ca9f61b31124e93f50
   ```

## Backups {#backup}

To back up Managed Service for Kubernetes node group data:
1. [Install kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl) and [configure it to work with the first Managed Service for Kubernetes cluster](../operations/connect/index.md#kubectl-connect).

   If your cluster has no public IP address assigned and `kubectl` is configured via the cluster's private IP address, run `kubectl` commands on a Yandex Cloud VM residing in the same network as the cluster.

1. Install the Velero application:
   
   {% list tabs group=instructions %}
   
   
   - Yandex Cloud Marketplace {#marketplace}
   
       Install Velero following [this guide](../operations/applications/velero-yc-csi.md). In the **Object Storage bucket name** field, specify the bucket you [created earlier](#before-you-begin).
   
       {% note warning %}
   
       If the name of the namespace where Velero is installed is not `velero`, use the additional `--namespace <Velero_namespace>` parameter for all the commands that follow.
   
       {% endnote %}
   
       {% note info %}
   
       Velero does not include the `restic` plugin for creating snapshots of [nfs](https://kubernetes.io/docs/concepts/storage/volumes/#nfs), [emptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir), [local](https://kubernetes.io/docs/concepts/storage/volumes/#local), and any other volume types without native snapshot support.
   
       {% endnote %}
   
   
   - Manually {#manual}
   
       1. If you do not have the Yandex Cloud CLI yet, [install and initialize it](../../cli/quickstart.md#install).
   
           The folder used by default is the one specified when [creating](../../cli/operations/profile/profile-create.md) the CLI profile. To change the default folder, use the `yc config set folder-id <folder_ID>` command. You can also specify a different folder for any command using `--folder-name` or `--folder-id`. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.
   
       1. [Create a service account](../../iam/operations/sa/create.md) for Velero to run.
       1. [Assign](../../iam/concepts/access-control/roles.md) it the `storage.editor` role to access [Yandex Object Storage](../../storage/index.md).
       1. [Create a static access key](../../iam/operations/authentication/manage-access-keys.md#create-access-key) for the service account in JSON format and save it to the `sa-key.json` file:
   
           ```bash
           yc iam access-key create \
              --service-account-name=<service_account_name> \
              --format=json > sa-key.json
           ```
   
       1. Create a file named `credentials` with the previously obtained static key data:
   
           ```ini
           [default]
           aws_access_key_id=<key_ID>
           aws_secret_access_key=<key_secret_part>
           ```
   
       1. Install the Velero server in the Managed Service for Kubernetes cluster:
   
           ```bash
           kubectl label volumesnapshotclasses.snapshot.storage.k8s.io yc-csi-snapclass \
           velero.io/csi-volumesnapshot-class="true" && \
           velero install \
             --backup-location-config s3Url=https://storage.yandexcloud.net,region=ru-central1 \
             --bucket <bucket_name> \
             --plugins velero/velero-plugin-for-aws:v1.5.2 \
             --provider aws \
             --secret-file <path_to_credentials_file> \
             --features=EnableCSI \
             --use-volume-snapshots=true \
             --snapshot-location-config region=ru-central1 \
             --uploader-type=restic
           ```
   
           Where:
           * `--backup-location-config`: Backup storage parameters, i.e., the URL of the Object Storage storage and region.
           * `--bucket`: Name of the [previously created](#before-you-begin) Object Storage bucket for backup storage.
           * `--plugins`: Plugin images for AWS API compatibility.
           * `--provider`: Name of the object storage provider.
           * `--secret-file`: Full path to the file with static access key data.
           * `--features`: List of enabled features.
           * `--snapshot-location-config`: Availability zone to host disk snapshots.
           * `--uploader-type=restic`: Enables the `restic` plugin for creating snapshots of [nfs](https://kubernetes.io/docs/concepts/storage/volumes/#nfs), [emptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir), [local](https://kubernetes.io/docs/concepts/storage/volumes/#local), and any other volume types without native snapshot support. This is an optional parameter.
   
           Result:
   
           ```text
           CustomResourceDefinition/backups.velero.io: attempting to create resource
           CustomResourceDefinition/backups.velero.io: already exists, proceeding
           CustomResourceDefinition/backups.velero.io: created
           ...
           Velero is installed! ⛵ Use 'kubectl logs deployment/velero -n velero' to view the status.
           ```
   
       1. Make sure the Velero pod state has changed to `Running`:
   
           ```bash
           kubectl get pods --namespace velero
           ```
   
   {% endlist %}
1. Make sure you have enough disk snapshot and disk size quotas to create a backup. To do this, you can use the [service for viewing quotas](../../quota-manager/operations/read-quotas.md).
1. Back up data from the Managed Service for Kubernetes cluster node group:

   ```bash
   velero backup create my-backup
   ```

   Result:

   ```text
   Backup request "my-backup" submitted successfully.
   Run `velero backup describe my-backup` or `velero backup logs my-backup` for more details.
   ```

1. Wait for the backup to complete. You will see `Completed` under `STATUS`.

   ```bash
   velero backup get
   ```

   Result:

   ```text
   NAME       STATUS     ERRORS  WARNINGS  CREATED                        EXPIRES  STORAGE LOCATION  SELECTOR
   my-backup  Completed  0       0         2020-10-19 17:13:25 +0300 MSK  29d      default           <none>
   ```

## Restoring data from a backup {#restore}

To restore data from a Managed Service for Kubernetes cluster node group:
1. [Configure kubectl](../operations/connect/index.md#kubectl-connect) to work with the second Managed Service for Kubernetes cluster.

   If your cluster has no public IP address assigned and `kubectl` is configured via the cluster's private IP address, run `kubectl` commands on a Yandex Cloud VM residing in the same network as the cluster.

1. Install the Velero application:
   
   {% list tabs group=instructions %}
   
   
   - Yandex Cloud Marketplace {#marketplace}
   
       Install Velero following [this guide](../operations/applications/velero-yc-csi.md). In the **Object Storage bucket name** field, specify the bucket you [created earlier](#before-you-begin).
   
       {% note warning %}
   
       If the name of the namespace where Velero is installed is not `velero`, use the additional `--namespace <Velero_namespace>` parameter for all the commands that follow.
   
       {% endnote %}
   
       {% note info %}
   
       Velero does not include the `restic` plugin for creating snapshots of [nfs](https://kubernetes.io/docs/concepts/storage/volumes/#nfs), [emptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir), [local](https://kubernetes.io/docs/concepts/storage/volumes/#local), and any other volume types without native snapshot support.
   
       {% endnote %}
   
   
   - Manually {#manual}
   
       1. If you do not have the Yandex Cloud CLI yet, [install and initialize it](../../cli/quickstart.md#install).
   
           The folder used by default is the one specified when [creating](../../cli/operations/profile/profile-create.md) the CLI profile. To change the default folder, use the `yc config set folder-id <folder_ID>` command. You can also specify a different folder for any command using `--folder-name` or `--folder-id`. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.
   
       1. [Create a service account](../../iam/operations/sa/create.md) for Velero to run.
       1. [Assign](../../iam/concepts/access-control/roles.md) it the `storage.editor` role to access [Yandex Object Storage](../../storage/index.md).
       1. [Create a static access key](../../iam/operations/authentication/manage-access-keys.md#create-access-key) for the service account in JSON format and save it to the `sa-key.json` file:
   
           ```bash
           yc iam access-key create \
              --service-account-name=<service_account_name> \
              --format=json > sa-key.json
           ```
   
       1. Create a file named `credentials` with the previously obtained static key data:
   
           ```ini
           [default]
           aws_access_key_id=<key_ID>
           aws_secret_access_key=<key_secret_part>
           ```
   
       1. Install the Velero server in the Managed Service for Kubernetes cluster:
   
           ```bash
           kubectl label volumesnapshotclasses.snapshot.storage.k8s.io yc-csi-snapclass \
           velero.io/csi-volumesnapshot-class="true" && \
           velero install \
             --backup-location-config s3Url=https://storage.yandexcloud.net,region=ru-central1 \
             --bucket <bucket_name> \
             --plugins velero/velero-plugin-for-aws:v1.5.2 \
             --provider aws \
             --secret-file <path_to_credentials_file> \
             --features=EnableCSI \
             --use-volume-snapshots=true \
             --snapshot-location-config region=ru-central1 \
             --uploader-type=restic
           ```
   
           Where:
           * `--backup-location-config`: Backup storage parameters, i.e., the URL of the Object Storage storage and region.
           * `--bucket`: Name of the [previously created](#before-you-begin) Object Storage bucket for backup storage.
           * `--plugins`: Plugin images for AWS API compatibility.
           * `--provider`: Name of the object storage provider.
           * `--secret-file`: Full path to the file with static access key data.
           * `--features`: List of enabled features.
           * `--snapshot-location-config`: Availability zone to host disk snapshots.
           * `--uploader-type=restic`: Enables the `restic` plugin for creating snapshots of [nfs](https://kubernetes.io/docs/concepts/storage/volumes/#nfs), [emptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir), [local](https://kubernetes.io/docs/concepts/storage/volumes/#local), and any other volume types without native snapshot support. This is an optional parameter.
   
           Result:
   
           ```text
           CustomResourceDefinition/backups.velero.io: attempting to create resource
           CustomResourceDefinition/backups.velero.io: already exists, proceeding
           CustomResourceDefinition/backups.velero.io: created
           ...
           Velero is installed! ⛵ Use 'kubectl logs deployment/velero -n velero' to view the status.
           ```
   
       1. Make sure the Velero pod state has changed to `Running`:
   
           ```bash
           kubectl get pods --namespace velero
           ```
   
   {% endlist %}
1. Make sure the data backup is there in the new Managed Service for Kubernetes cluster:

   ```bash
   velero backup get
   ```

   Result:

   ```text
   NAME       STATUS     ERRORS  WARNINGS  CREATED                        EXPIRES  STORAGE LOCATION  SELECTOR
   my-backup  Completed  0       0         2020-10-19 17:13:25 +0300 MSK  29d      default           <none>
   ```

1. Restore data from the backup:

   ```bash
   velero restore create my-restore \
     --exclude-namespaces velero \
     --from-backup my-backup
   ```

   Where:

   * `--exclude-namespaces`: List of namespaces to exclude from the recovery process.
   * `--from-backup`: Name of the backup for recovery.

   Result:

   ```text
   Restore request "my-restore" submitted successfully.
   Run `velero restore describe my-restore` or `velero restore logs my-restore` for more details.
   ```

1. Wait for the backup restoration to complete. You will see `Completed` under `STATUS`.

   ```bash
   velero get restore
   ```

   Result:

   ```text
   NAME        BACKUP     STATUS     STARTED                        COMPLETED                      ERRORS  WARNINGS  CREATED                        SELECTOR
   my-restore  my-backup  Completed  2020-10-20 14:04:55 +0300 MSK  2020-10-20 14:05:22 +0300 MSK  0       23        2020-10-20 14:04:55 +0300 MSK  <none>
   ```

## Delete the resources you created {#clear-out}

If you no longer need the resources you created, delete them:

{% list tabs group=instructions %}

- Manually {#manual}

  * [Delete the Managed Service for Kubernetes clusters](../operations/kubernetes-cluster/kubernetes-cluster-delete.md).
  * If you reserved [public static IP addresses](../../vpc/concepts/address.md#public-addresses) for the Managed Service for Kubernetes clusters, [delete them](../../vpc/operations/address-delete.md).
  * [Delete the Object Storage bucket](../../storage/operations/buckets/delete.md).
  * [Delete the service account](../../iam/operations/sa/delete.md) used to work with Velero.

- Terraform {#tf}

  1. In the terminal window, go to the directory containing the infrastructure plan.
  
      {% note warning %}
  
      Make sure the directory has no Terraform manifests with the resources you want to keep. Terraform deletes all resources that were created using the manifests in the current directory.
  
      {% endnote %}
  
  1. Delete resources:
  
      1. Run this command:
  
          ```bash
          terraform destroy
          ```
  
      1. Confirm deleting the resources and wait for the operation to complete.
  
      All the resources described in the Terraform manifests will be deleted.

{% endlist %}