[Yandex Cloud documentation](../../index.md) > [Yandex Managed Service for Kubernetes](../index.md) > [Tutorials](index.md) > Creating a Kubernetes cluster using the Yandex Cloud provider for the Kubernetes Cluster API

# Creating a self-managed Kubernetes cluster using the Yandex Cloud provider for the Kubernetes Cluster API

# Creating a self-managed Kubernetes cluster using the Yandex Cloud provider for the Kubernetes Cluster API

[Cluster-api-provider-yandex](https://github.com/yandex-cloud/cluster-api-provider-yandex) is a provider for deploying a self-managed Kubernetes cluster in Yandex Cloud infrastructure using the [Kubernetes Cluster API](https://cluster-api.sigs.k8s.io/).

The cluster is deployed based on Yandex Compute Cloud [virtual machines](../../compute/concepts/vm.md) and a [Yandex Application Load Balancer](../../application-load-balancer/concepts/application-load-balancer.md).

**Advantages of using the Yandex Cloud provider for creating clusters:** {#advantages}

* Integration with the [Yandex Cloud API](../../api-design-guide/concepts/general.md).
* Declarative approach to cluster creation and management.
* Describing a cluster as a custom resource, i.e., [CustomResourceDefinition](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/).
* Numerous parameters for configuring cluster compute resources.
* [Custom OS images](#prepare-os-image) for the master and worker nodes.
* Custom control plane.
* Alternative to [Terraform](https://www.terraform.io/) for CI pipelines.

**Provider compatibility with the Kubernetes Cluster API** {#compatibility}

| Provider version | Cluster API version |
| :---: | :---: |
| v1alpha1 | v1beta1 (v1.x) |

To deploy a Kubernetes cluster in Yandex Cloud using the Cluster API:

1. [Get your cloud ready](#before-you-begin).
1. [Set up your environment](#prepare-environment).
1. [Prepare an OS image for cluster nodes](#prepare-os-image).
1. [Get a Docker image with the Yandex Cloud provider](#get-docker-image).
1. [Install the Yandex Cloud provider and the Kubernetes Cluster API provider](#install-providers).
1. [Generate cluster manifests](#prepare-manifests).
1. [Deploy a cluster](#create-cluster).
1. [Connect to the cluster](#connect-to-cluster).
1. [Install a CCM](#install-ccm).
1. [Install a CNI](#install-cni).
1. [Check the connection between the management cluster and the new cluster](#check-connection).

If you no longer need the resources you created, [delete them](#clear-out).

## Get your cloud ready {#before-you-begin}

Sign up for Yandex Cloud and create a [billing account](../../billing/concepts/billing-account.md):
1. Navigate to the [management console](https://console.yandex.cloud) and log in to Yandex Cloud or create a new account.
1. On the **[Yandex Cloud Billing](https://center.yandex.cloud/billing/accounts)** page, make sure you have a billing account linked and it has the `ACTIVE` or `TRIAL_ACTIVE` [status](../../billing/concepts/billing-account-statuses.md). If you do not have a billing account, [create one](../../billing/quickstart/index.md) and [link](../../billing/operations/pin-cloud.md) a cloud to it.

If you have an active billing account, you can create or select a [folder](../../resource-manager/concepts/resources-hierarchy.md#folder) for your infrastructure on the [cloud page](https://console.yandex.cloud/cloud).

[Learn more about clouds and folders here](../../resource-manager/concepts/resources-hierarchy.md).

### Required paid resources {#paid-resources}

The infrastructure support costs include:
* Fee for computing resources and disks of VMs used for Kubernetes cluster deployment, auxiliary VM, and Managed Service for Kubernetes management cluster nodes (see [Compute Cloud pricing](../../compute/pricing.md)).
* Fee for using L7 load balancer's computing resources (see [Yandex Application Load Balancer pricing](../../application-load-balancer/pricing.md)).
* Fee for using Managed Service for Kubernetes management cluster master and outgoing traffic (see [Yandex Managed Service for Kubernetes pricing](../pricing.md)).
* Fee for [public IP addresses](../../vpc/concepts/address.md#public-addresses) for auxiliary VMs and Managed Service for Kubernetes management cluster (see [Yandex Virtual Private Cloud pricing](../../vpc/pricing.md)).
* Fee for a [NAT gateway](../../vpc/concepts/gateways.md) (see [Yandex Virtual Private Cloud pricing](../../vpc/pricing.md#nat-gateways)).

#### Optional costs {#optional-expenses}

* If intending to use a custom image for the new Kubernetes cluster nodes:
  * Fee for storing the image in the bucket and data operations (see [Yandex Object Storage pricing](../../storage/pricing.md)).
  * Fee for storing the image in Compute Cloud (see [Yandex Compute Cloud pricing](../../compute/pricing.md)).
* If intending to use a custom Docker image to deploy the Yandex Cloud provider in the management cluster, fee for storing the Docker image in the registry and outgoing traffic (see [Yandex Container Registry pricing](../../container-registry/pricing.md)).

### Set up your infrastructure {#infra}

1. Set up a Yandex Cloud [service account](../../iam/concepts/users/service-accounts.md):
    1. [Create](../../iam/operations/sa/create.md) a service account you will use to create resources for the cluster.
    1. [Assign](../../iam/operations/sa/assign-role-for-sa.md) the [compute.editor](../../compute/security/index.md#compute-editor) and [alb.editor](../../application-load-balancer/security/index.md#alb-editor) roles for the folder to the service account.
    1. [Create](../../iam/operations/authentication/manage-authorized-keys.md#create-authorized-key) an authorized key for the service account in JSON format.
1. If your folder does not have a Virtual Private Cloud [network](../../vpc/concepts/network.md#network) yet, [create](../../vpc/operations/network-create.md) one. [Create](../../vpc/operations/subnet-create.md) a subnet as well.

    {% note warning %}
    
    Do not change or delete Virtual Private Cloud resources used by the Managed Service for Kubernetes cluster. This may cause cluster errors or make it impossible to delete later.
    
    {% endnote %}

1. The new cluster infrastructure will automatically be assigned the default [security group](../../vpc/concepts/security-groups.md) which is created along with the network. [Add](../../vpc/operations/security-group-add-rule.md) the following rules for _incoming_ traffic to this group:

    Protocol | Port range | Source type | Source | Description
    --- | --- | --- | --- | ---
    `TCP` | `0-65535` | `Security group` | `Balancer` | `Health checks by an L7 load balancer`
    `Any` | `8443` | `CIDR` | `0.0.0.0/0` | `Access to the Kubernetes API`

1. The created cluster will be accessible within the cloud network via an [internal IP address](../../vpc/concepts/address.md#internal-addresses). To enable remote access to the cluster:
    1. [Create](../../compute/operations/vm-create/create-linux-vm.md) an auxiliary VM with a public IP address and the default security group in the same network where your cluster will be deployed.
    1. Install [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl) on the auxiliary VM.
1. Create a Managed Service for Kubernetes _management_ [cluster](../operations/kubernetes-cluster/kubernetes-cluster-create.md) with a public IP address and a [node group](../operations/node-group/node-group-create.md). You will need this cluster to deploy the new cluster using the Cluster API and to manage the cluster infrastructure.

    {% note tip %}

    You can also deploy the management cluster locally, e.g., using [`kind`](https://kind.sigs.k8s.io/).

    {% endnote %}

1. To enable the new cluster to access the internet and pull Docker images, [configure](../../vpc/operations/create-nat-gateway.md) a NAT gateway for the subnet that will host the new cluster.

## Set up your environment {#prepare-environment}

Set up the environment locally.

1. Install the following tools:
    * [Go](https://go.dev/doc/install) 1.22.0 or higher.
    * [Docker](https://www.docker.com/) 17.03 or higher.
    * [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl) 1.11.3 or higher.
    * [clusterctl](https://cluster-api.sigs.k8s.io/user/quick-start#install-clusterctl) 1.5.0 or higher.

1. [Configure](../operations/connect/index.md#kubectl-connect) `kubectl` access to the Managed Service for Kubernetes management cluster.

    If you deployed the management cluster locally with `kind`, follow [this guide](https://kind.sigs.k8s.io/docs/user/quick-start/#interacting-with-your-cluster) to configure cluster access.

1. Clone the [cluster-api-provider-yandex](https://github.com/yandex-cloud/cluster-api-provider-yandex) repository and navigate to the project directory.

    ```bash
    git clone https://github.com/yandex-cloud/cluster-api-provider-yandex.git
    cd cluster-api-provider-yandex
    ```
    
## Set up an OS image for cluster nodes {#prepare-os-image}

The OS [image](../../compute/concepts/image.md) to deploy on the nodes of the new cluster must be configured to support the Kubernetes Cluster API and compatible with Compute Cloud.

You can use a ready-made test image or build a custom one:

{% list tabs %}

- Ready-made image

  To use a test Ubuntu 24.04 image configured for Kubernetes 1.31.4, specify the `fd8a3kknu25826s8hbq3` image ID in the `YANDEX_CONTROL_PLANE_MACHINE_IMAGE_ID` variable when [generating the cluster manifest](#prepare-manifests).

  {% note warning %}

  This image is for demonstration purposes only, do not use it in production.

  {% endnote %}

- Custom image

  1. [Build](https://image-builder.sigs.k8s.io/capi/capi) a custom OS image using [Image Builder](https://github.com/kubernetes-sigs/image-builder).

      See also: [Setting up a disk image for Compute Cloud](../../compute/operations/image-create/custom-image.md).
  1. [Upload](../../compute/operations/image-create/upload.md) the image to Compute Cloud and save its ID.

{% endlist %}

## Get a Docker image with the Yandex Cloud provider {#get-docker-image}

You can use a ready-made Docker image with the Yandex Cloud provider from a public [Yandex Container Registry](../../container-registry/concepts/registry.md) or build your own image from the source code.

{% list tabs %}

- Ready-made image

  1. [Authenticate](../../container-registry/operations/authentication.md#cred-helper) in your Container Registry using the [Docker credential helper](https://docs.docker.com/engine/reference/commandline/login/#credential-helpers).
  1. Add the path to the Docker image with the Yandex Cloud provider in the public registry to the `IMG` environment variable:

      ```bash
      export IMG=cr.yandex/crpsjg1coh47p81vh2lc/capy/cluster-api-provider-yandex:latest
      ```

- Image built from source code

  1. [Create](../../container-registry/operations/registry/registry-create.md) a Container Registry and save its ID.
  1. [Authenticate](../../container-registry/operations/authentication.md#cred-helper) in your Container Registry using the [Docker credential helper](https://docs.docker.com/engine/reference/commandline/login/#credential-helpers).
  1. Add the path for storing the new Docker image in the registry to the `IMG` environment variable:

      ```bash
      export IMG=cr.yandex/<registry_ID>/cluster-api-provider-yandex:<tag>
      ```

  1. If you are building your Docker image on a non-[AMD64](https://ru.wikipedia.org/wiki/X86-64) computer, edit the `docker-build` section in the [Makefile](https://github.com/yandex-cloud/cluster-api-provider-yandex/blob/master/Makefile):

      ```text
      docker build --platform linux/amd64 -t ${IMG} .
      ```

  1. Run the Docker daemon.
  1. Build a Docker image and push it to the registry:

      ```bash
      make docker-build docker-push
      ```

{% endlist %}

## Install the Yandex Cloud provider and the Kubernetes Cluster API provider {#install-providers}

1. Initialize the management cluster:

    ```bash
    clusterctl init
    ```

    The management cluster will include the core components of the Kubernetes Cluster API and [cert-manager](https://cert-manager.io/).

1. Create a [CustomResourceDefinitions](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/) resource (CRD) for the new cluster:

    ```bash
    make install
    ```

1. Get the list of installed CRDs:

    ```bash
    kubectl get crd | grep cluster.x-k8s.io
    ```

    To get the manifest of a specific CRD, run the following command:

    ```bash
    kubectl get crd <CRD_name> \
      --output yaml
    ```

1. Create a namespace for the Yandex Cloud provider:

    ```bash
    kubectl create namespace capy-system
    ```

1. Create a secret with the authorized key for the Yandex Cloud service account:

    ```bash
    kubectl create secret generic yc-sa-key \
      --from-file=key=<path_to_file_with_authorized_key> \
      --namespace capy-system
    ```

1. Install the Yandex Cloud provider:

    ```bash
    make deploy
    ```

## Generate cluster manifests {#prepare-manifests}

1. Get the IDs of the Yandex Cloud resources required to deploy the cluster:
    * [OS image](../../compute/operations/image-control/image-control-get-info.md)
    * [Folder](../../resource-manager/operations/folder/get-id.md)
    * [Availability zone](../../overview/concepts/geo-scope.md)
    * [Network](../../vpc/operations/network-get-info.md)
    * [Subnet](../../vpc/operations/subnet-get-info.md) in the selected availability zone

1. Add the IDs to these environment variables:

    ```bash
    export YANDEX_CONTROL_PLANE_MACHINE_IMAGE_ID=<image_ID>
    export YANDEX_FOLDER_ID=<folder_ID>
    export YANDEX_NETWORK_ID=<network_ID>
    export YANDEX_SUBNET_ID=<subnet_ID>
    export YANDEX_ZONE_ID=<availability_zone_ID>
    ```

    If you [did not build a custom OS image](#prepare-os-image), set the `YANDEX_CONTROL_PLANE_MACHINE_IMAGE_ID` variable to `fd8a3kknu25826s8hbq3`. This is the ID of a test Ubuntu 24.04 image compatible with Kubernetes 1.31.4.

1. Generate cluster manifests:

    ```bash
    clusterctl generate cluster <name_of_new_cluster> \
      --from templates/cluster-template.yaml > /tmp/capy-cluster.yaml
    ```

    The `capy-cluster.yaml` manifest will describe the following:
    * Application Load Balancer with a dynamic internal IP address. You [assign it a static IP address](#configure-endpoint), if required.

        {% note warning %}

        * Once the cluster is created, you will not be able to assign a static IP address to the L7 load balancer.
        * Do not modify or delete the network load balancer and its child resources created using Managed Service for Kubernetes via the Yandex Cloud interfaces (the management console, Terraform, CLI, or API). This may cause incorrect operation of the cluster.

        {% endnote %}

    * Three control plane nodes for the cluster.

1. Optionally, to deploy worker nodes right away, add their description to the manifest.

    ```bash
    clusterctl generate cluster <name_of_new_cluster> \
        --worker-machine-count <number_of_worker_nodes> \
        --from templates/cluster-template.yaml > /tmp/capy-cluster.yaml
    ```

### Optionally, configure the API server endpoint {#configure-endpoint}

Configure the L7 load balancer in the `capy-cluster.yaml` manifest:

```yaml
  loadBalancer:
    listener:
      address: <static_IP_address_from_subnet_range>
      subnet:
        id: <subnet_ID>
```

## Deploy the cluster {#create-cluster}

Run this command:

```bash
kubectl apply -f /tmp/capy-cluster.yaml
```

You can monitor the cluster creation progress in the Yandex Cloud [management console](https://console.yandex.cloud) and the `capy-controller-manager` pod logs:

```bash
kubectl logs <capy-controller-manager_pod_name> \
  --namespace capy-system \
  --follow
```

## Connect to the cluster {#connect-to-cluster}

The credentials for accessing the new cluster will be stored in the `<name_of_new_cluster>-kubeconfig` secret in the management cluster.

1. Get the data from the secret:

    ```bash
    kubectl get secret <name_of_new_cluster>-kubeconfig \
      --output yaml | yq -r '.data.value' | base64 \
      --decode > capy-cluster-config
    ```

1. [Send](../../compute/operations/vm-connect/scp-sftp.md) the `kubectl` configuration file to the auxiliary VM:

    ```bash
    scp <path_to_capy-cluster-config_file_on_local_computer> \
    <username>@<VM_public_IP_address>:/home/<username>/.kube/config
    ```

1. [Connect](../../compute/operations/vm-connect/ssh.md) to the auxiliary VM over SSH.
1. Make sure the new cluster is accessible:

    ```bash
    kubectl cluster-info
    ```

## Install a CCM in the new cluster {#install-ccm}

For connection between the cluster resources and Yandex Cloud resources, install a [cloud controller manager](https://kubernetes.io/docs/concepts/architecture/cloud-controller/) in the new cluster, e.g., the [Kubernetes Cloud Controller Manager for Yandex Cloud](https://github.com/deckhouse/yandex-cloud-controller-manager/tree/master).

{% note info %}

If you want to use the Kubernetes Cloud Controller Manager for Yandex Cloud, add the current version of the Docker image and the `YANDEX_CLUSTER_NAME` environment variable with the new cluster's name to the [yandex-cloud-controller-manager.yaml](https://github.com/deckhouse/yandex-cloud-controller-manager/blob/master/manifests/yandex-cloud-controller-manager.yaml) manifest example for `DaemonSet`.

{% endnote %}

## Install a CNI in the new cluster {#install-cni}

To provide networking for pods in the new cluster, install a [container network interface](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/), such as [Cilium](https://github.com/cilium/cilium) or [Calico](https://github.com/projectcalico/calico).

For more information, see these docs:
* [Cilium Quick Installation](https://docs.cilium.io/en/stable/gettingstarted/k8s-install-default/).
* [Quickstart for Calico on Kubernetes](https://docs.tigera.io/calico/latest/getting-started/kubernetes/quickstart).

## Check the connection between the management cluster and the new cluster {#check-connection}

1. Connect to the auxiliary VM and make sure all pods with the required system components have been deployed in the cluster:

    ```bash
    kubectl get pods --all-namespaces
    ```

    Here is an example of the command output:

    ```bash
    NAMESPACE     NAME                                                       READY   STATUS    RESTARTS   AGE
    kube-system   calico-kube-controllers-695bcfd99c-rcc42                   1/1     Running   0          3h55m
    kube-system   calico-node-9qhxj                                          1/1     Running   0          3h55m
    kube-system   coredns-7c65d6cfc9-52tvn                                   1/1     Running   0          4h50m
    kube-system   coredns-7c65d6cfc9-dpgvg                                   1/1     Running   0          4h50m
    kube-system   etcd-capy-cluster-control-plane-p646q                      1/1     Running   0          4h50m
    kube-system   kube-apiserver-capy-cluster-control-plane-p646q            1/1     Running   0          4h50m
    kube-system   kube-controller-manager-capy-cluster-control-plane-p646q   1/1     Running   0          4h50m
    kube-system   kube-proxy-wb7jr                                           1/1     Running   0          4h50m
    kube-system   kube-scheduler-capy-cluster-control-plane-p646q            1/1     Running   0          4h50m
    kube-system   yandex-cloud-controller-manager-nwhwv                      1/1     Running   0          26s
    ```

1. Use your local computer to check the connection between the management cluster and the new cluster:

    ```bash
    clusterctl describe cluster <name_of_new_cluster>
    ```

    Result:

    ```bash
    NAME                                                             READY  SEVERITY  REASON  SINCE  MESSAGE                                  
    Cluster/capy-cluster                                             True                     10s
    ├─ClusterInfrastructure - YandexCluster/capy-cluster                                   
    └─ControlPlane - KubeadmControlPlane/capy-cluster-control-plane  True                     10s                                                                                       
      └─3 Machines...                                                True                     3m9s   See capy-cluster-control-plane-cf72l, capy-cluster-control-plane-g9jw7, ...
    ```

## Delete the resources you created {#clear-out}

Some resources are not free of charge. Delete the resources you no longer need to avoid paying for them:

1. Delete the Kubernetes cluster created using the Cluster API:

    ```bash
    kubectl delete -f /tmp/capy-cluster.yaml
    ```

1. Delete the CRD from the Managed Service for Kubernetes management cluster:

    ```bash
    make uninstall
    ```

1. Delete the Yandex Cloud provider controller from the management cluster:

    ```bash
    make undeploy
    ```

1. Delete the auxiliary Yandex Cloud resources if you created them:
    * [Node group](../operations/node-group/node-group-delete.md) of the Managed Service for Kubernetes management cluster
    * Managed Service for Kubernetes [management cluster](../operations/kubernetes-cluster/kubernetes-cluster-delete.md)
    * [Auxiliary VM](../../compute/operations/vm-control/vm-delete.md)
    * [NAT gateway](../../vpc/operations/delete-nat-gateway.md)
    * [OS image](../../compute/operations/image-control/delete.md) in Compute Cloud
    * [OS image](../../storage/operations/objects/delete.md) in Object Storage
    * [Bucket](../../storage/operations/buckets/delete.md)
    * [Docker image](../../container-registry/operations/docker-image/docker-image-delete.md)
    * [Registry](../../container-registry/operations/registry/registry-delete.md)