[Yandex Cloud documentation](../../index.md) > [Security in Yandex Cloud](../index.md) > [Cloud infrastructure security standard, version 1.4.2](index.md) > Security Kubernetes

# Kubernetes security requirements

## 7. Kubernetes security {#kubernetes-security}


[Yandex Managed Service for Kubernetes](../../managed-kubernetes/index.md) provides an environment for managing containerized applications in the Yandex Cloud infrastructure. Deploy, scale, and manage applications in containers using Kubernetes.

The user is responsible for all actions made inside the Kubernetes node. The user is responsible for the security of the nodes and their proper setup in accordance with PCI DSS requirements and other security standards.

Yandex Cloud is responsible for the Kubernetes API security.

The user is responsible for correctly choosing security settings in Managed Service for Kubernetes, including selecting the [channel](../../managed-kubernetes/concepts/release-channels-and-updates.md) and the update schedule.

### Overview {#general}

#### 7.1 The use of sensitive data is limited {#not-use-critical-data}

To comply with PCI DSS or other security standards when using Managed Service for Kubernetes, do not:

* Use sensitive data in names and descriptions of clusters, node groups, namespaces, services, and pods.
* Use sensitive data in [Kubernetes](../../managed-kubernetes/concepts/index.md#node-labels) node labels and [Yandex Cloud](../../resource-manager/concepts/labels.md) service resource labels.
* Use sensitive data in pod manifests.
* Use sensitive data in etcd in clear text.
* Write sensitive data to Managed Service for Kubernetes logs.

| Requirement ID | Severity |
| --- | --- |
| K8S1 | High |

{% list tabs group=instructions %}

- Manual check {#manual}

  * Make sure that names and descriptions of clusters, node groups, namespaces, services, and pods contain no sensitive data.
  * Check configuration files for critical data

- Performing a check in the management console {#console}

  1. In the [management console](https://console.yandex.cloud), select the [folder](../../resource-manager/concepts/resources-hierarchy.md#folder) where your Managed Service for Kubernetes instance is located.
  1. In the list of services, select **Managed Service for Kubernetes**.
  1. Make sure that [Kubernetes node labels](../../managed-kubernetes/concepts/index.md#node-labels) and [Yandex Cloud service resource labels](../../resource-manager/concepts/labels.md) contain no sensitive data.

{% endlist %}

**Guides and solutions to use:**

Manually edit the names or contents of manifests and other configuration files if they use sensitive data.

#### 7.2 Resources are isolated from each other {#maximum-isolation}

Wherever possible, ensure maximum isolation between resources:

* Use a separate organization for each "big" project.
* Use a separate cloud for each development team.
* Use a separate Kubernetes cluster located in a separate folder for each service.
* Use a separate namespace for each microservice.
* Your clouds must have no shared resources. Cloud members must have access only to their clouds.

Less strict isolation models are also possible, e.g., where:

* Projects are deployed in different clouds.
* Development teams use independent folders.
* Services have separate Kubernetes clusters.
* Microservices use different namespaces.

| Requirement ID | Severity |
| --- | --- |
| K8S2 | High |

{% list tabs group=instructions %}

- Manual check {#manual}

  Wherever possible, ensure maximum isolation between resources.

{% endlist %}

#### 7.3 There is no access to the Kubernetes API and node groups from untrusted networks {#api-security}

We do not recommend granting access to the Kubernetes API and node groups from non-trusted networks, e.g., from the internet. Use firewall protection where needed (for example, [security groups](../../vpc/concepts/security-groups.md)).

| Requirement ID | Severity |
| --- | --- |
| K8S3 | Medium |

**Guides and solutions to use:**

* [Guide on creating a cluster with no internet access](../../managed-kubernetes/tutorials/k8s-cluster-with-no-internet.md).
* [Security group setup guide](../../managed-kubernetes/operations/connect/security-groups.md).
* Use network policy configuration tools via the [Calico](../../managed-kubernetes/concepts/network-policy.md#calico) (basic) or [Cilium CNI](../../managed-kubernetes/concepts/network-policy.md#cilium) (advanced) plugins in Yandex Cloud. By default, apply the `default deny` rules for incoming and outgoing traffic with only the relevant traffic allowed.
* For online endpoints, allocate an independent Kubernetes cluster or independent node groups (using such mechanisms as [Taints and Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/#:~:text=Node%20affinity%20is%20a%20property,onto%20nodes%20with%20matching%20taints) + [Node affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/)). By doing this, you establish a DMZ so that if your nodes are compromised online, your attack surface is small.
* To enable incoming network access to your workloads via HTTP/HTTPS, use the [Ingress](https://kubernetes.io/docs/concepts/services-networking/ingress/) resource. There are at least two controller options you can use in Yandex Cloud:

  * [Yandex Cloud Gwin Controller](../../managed-kubernetes/alb-ref/gwin-index.md).
  * [Application Load Balancer of an Ingress controller](../../application-load-balancer/tools/k8s-ingress-controller/index.md).

#### 7.4 Authentication and access management are configured in Managed Service for Kubernetes {#kubernetes-auth}

For the Kubernetes cluster to run, you need two service accounts: [the service account of the cluster and the service account of the node group](../../managed-kubernetes/security/index.md#sa-annotation). The access of IAM accounts to Managed Service for Kubernetes resources is managed at the following levels:

* Managed Service for Kubernetes service roles (access to the Yandex Cloud API). These enable you to control clusters and node groups (for example, create a cluster, create/edit/delete a node group, and so on).
* Service roles to access the Kubernetes API. These let you control cluster resources via the Kubernetes API (for example, perform standard actions with Kubernetes: create, delete, view namespaces, work with pods, deployments, create roles, and so on). Only the basic global roles at the cluster level are available: `k8s.cluster-api.cluster-admin`, `k8s.cluster-api.editor`, and `k8s.cluster-api.viewer`.
* Primitive roles. These are global primitive IAM roles that include service roles (e.g., the primitive `admin` role includes both the service administration role and the administrative role to access the Kubernetes API).
* Standard Kubernetes roles. Inside the Kubernetes cluster itself, you can use Kubernetes tools to create both regular roles and cluster roles. Thus you can manage access for IAM accounts at the namespace level. To assign IAM roles at the namespace level, you can manually create RoleBinding objects in a relevant namespace stating the cloud user's IAM ID in the **subjects name** field.

| Requirement ID | Severity |
| --- | --- |
| K8S4 | High |

{% list tabs group=instructions %}

- Manual check {#manual}

  Make sure the above recommendations are met.

{% endlist %}

#### 7.5 Managed Service for Kubernetes uses a safe configuration {#kubernetes-safe-config}

In Managed Service for Kubernetes, the user is fully in control of all node group settings, but only partially in control of the [master](../../managed-kubernetes/concepts/index.md#master) settings. The user is responsible for the whole cluster's security.

The [CIS Kubernetes Benchmark](https://www.cisecurity.org/benchmark/kubernetes) standard is designed to build a secure Kubernetes configuration, including node configurations. In Yandex Cloud, the Kubernetes node groups are deployed by default with the configuration that complies with CIS Kubernetes Benchmark.

| Requirement ID | Severity |
| --- | --- |
| K8S5 | Medium |

{% list tabs group=instructions %}

- Manual check {#manual}

  * Using the [kube-bench](https://github.com/aquasecurity/kube-bench) tool, check whether the node group configuration is compliant with CIS Kubernetes Benchmark. The tool officially supports the Yandex Cloud node groups.
  * [Starboard Operator](https://blog.aquasec.com/automate-kubernetes-compliance) is a free tool that helps you automate scanning of images for vulnerabilities and checking that the configuration is compliant with CIS Kubernetes Benchmark. Starboard Operator supports integration with kube-bench and is used for its automatic startup.

{% endlist %}

#### 7.6 Managed Service for Kubernetes data encryption and secret management are done in _ESO as a Service_ format {#data-encryption}

At the Kubernetes etcd level, encrypt secrets using an in-built [mechanism from Yandex Cloud](../../managed-kubernetes/concepts/encryption.md).

We recommend that you use SecretManager solutions to work with Kubernetes secrets. [Yandex Lockbox](../../lockbox/index.md) is such a solution in Yandex Cloud.

 Yandex Lockbox was integrated with Kubernetes using the [External Secrets](https://external-secrets.io/latest/) open-source project. In Cloud Marketplace, the solution is available in the basic simplified scenario: [External Secrets Operator with Yandex Lockbox support](https://yandex.cloud/en/marketplace/products/yc/external-secrets).

The most secure recommended option for encrypting secrets is ESO as a Service (External Secrets Operator as a service). When using ESO, the global administrator has access to the namespace where ESO is installed, and administrators of individual namespaces create their own [SecretStore](https://external-secrets.io/latest/api/secretstore/) objects (where they specify IAM-authorized access keys for their Lockbox secrets). If this SecretStore object is compromised, the authorized key of only one namespace will be compromised – not all of them, as in the case of Shared ClusterSecretStore.

| Requirement ID | Severity |
| --- | --- |
| K8S6 | Medium |

**Guides and solutions to use:**

* [Guide on External Secrets and Yandex Lockbox from the project description](https://external-secrets.io/latest/provider/yandex-lockbox/).
* [Guide on External Secrets and Yandex Lockbox from the Yandex Cloud documentation](../../lockbox/tutorials/kubernetes-lockbox-secrets.md).

#### 7.7 Docker images are stored in a Container Registry registry configured for regular image scanning {#docker-images-periodic-scan}

To ensure effective security, we recommend using [Container Registry](../../container-registry/index.md) to store Docker images to be deployed in Managed Service for Kubernetes. This allows you to quickly respond to new vulnerabilities in images using built-in recurrent vulnerability scanning.

You should perform vulnerability scanning at least once a week. This will help you detect and eliminate vulnerabilities in images in a timely manner, which significantly reduces risks of unauthorized access to your resources and increases security level of your infrastructure.

Using Container Registry to store images will also provide centralized image version control for simpler updates and security management.

| Requirement ID | Severity |
| --- | --- |
| K8S7 | Medium |

{% list tabs group=instructions %}

- Performing a check in the management console {#console}

  1. In the [management console](https://console.yandex.cloud), select the folder the registry with Docker images belongs to.
  1. Select the appropriate registry in **Container Registry**.
  1. Navigate to the **Vulnerability scanner** tab and click **Edit settings**.
  1. Make sure that scheduled Docker image scans are enabled with a frequency of at least once a week.

{% endlist %}

**Guides and solutions to use:**

* [Guide on scheduled scanning of Docker images](../../container-registry/operations/scanning-docker-image.md#scheduled).

#### 7.8 One of the three latest Kubernetes versions is used, updates are monitored {#version-update}

In Kubernetes, both automatic and manual updates are available for [clusters](../../managed-kubernetes/concepts/index.md#kubernetes-cluster) and [node groups](../../managed-kubernetes/concepts/index.md#node-group). You can request a manual update of your Kubernetes cluster or its nodes to the latest supported [version](../../managed-kubernetes/concepts/release-channels-and-updates.md) at any time. Manual updates bypass any configured maintenance windows and maintenance exceptions. Kubernetes issues updates in a regular manner. To meet the Information Security standards:

* Select a relevant update channel and enable either automatic installation of updates, or manual installation immediately after publication in the selected channel.
* Check that the update settings meet the Information Security standards.
* Use one of the three latest Kubernetes versions, because updates (including security updates) are only released for these versions.

| Requirement ID | Severity |
| --- | --- |
| K8S8 | Medium |

{% list tabs group=instructions %}

- Performing a check in the management console {#console}

  To get a list of available versions for a Kubernetes cluster:
  1. Navigate to the [folder](../../resource-manager/concepts/resources-hierarchy.md#folder) dashboard and select **Managed Service for Kubernetes**.
  1. Click the name of the Kubernetes cluster.
  1. Click **Edit** in the top-right corner.
  1. View the list of available versions in the **Kubernetes version** field under **Master configuration**.

  To get a list of available versions for a Kubernetes node group:
  1. Navigate to the folder dashboard and select **Managed Service for Kubernetes**.
  1. Click the name of your Kubernetes cluster and open the **Node management** tab.
  1. Select the Kubernetes node group from the list and click **Edit** in the top-right corner.
  1. Get a list of available versions in the **Kubernetes version** field.

- Performing a check via the CLI {#cli}

  If you do not have the Yandex Cloud CLI yet, [install and initialize it](../../cli/quickstart.md#install).

  The folder used by default is the one specified when [creating](../../cli/operations/profile/profile-create.md) the CLI profile. To change the default folder, use the `yc config set folder-id <folder_ID>` command. You can also specify a different folder for any command using `--folder-name` or `--folder-id`. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

  To get a list of available versions, run this command:

  ```bash
  yc managed-kubernetes list-versions
  ```

- Checking via the API {#api}

  To get a list of available versions, use the [list](../../managed-kubernetes/managed-kubernetes/api-ref/Version/list.md) method.

{% endlist %}

**Guides and solutions to use:**

* [Guide on how to update a cluster automatically](../../managed-kubernetes/operations/update-kubernetes.md#cluster-upgrade).
* [Guide on how to update a cluster manually](../../managed-kubernetes/operations/update-kubernetes.md#cluster-manual-upgrade).

#### 7.9 Backup is configured {#backup}

To ensure continuous operation and data protection, we recommend using backups in Managed Service for Kubernetes. With backups, you can quickly recover the service without experiencing any data or time loss in the wake of a malfunction or accident. Yandex Cloud provides secure storage and replication for data in [Kubernetes](../../managed-kubernetes/concepts/index.md#kubernetes-cluster) clusters. However, you can back up data from your [Kubernetes cluster node groups](../../managed-kubernetes/concepts/index.md#node-group) at any time and store it in [Yandex Object Storage](../../storage/index.md) or other types of storage.

| Requirement ID | Severity |
| --- | --- |
| K8S9 | High |

{% list tabs group=instructions %}

- Manual check {#manual}

  Make sure that you have backups of your data from node groups of Kubernetes clusters.

{% endlist %}

**Guides and solutions to use:**

You can create backups of Kubernetes cluster node group data using [Velero](https://velero.io/). It supports Yandex Cloud [disks](../../compute/concepts/disk.md) through the Kubernetes CSI driver and helps create [disk and volume snapshots](../../compute/concepts/snapshot.md).

If installed manually, Velero allows you to use [nfs](https://kubernetes.io/docs/concepts/storage/volumes/#nfs), [emptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir), [local](https://kubernetes.io/docs/concepts/storage/volumes/#local), or any other volume type without built-in support for snapshots. To use one of these volume types, install Velero with the [restic plugin](https://velero.io/docs/v1.8/restic/). Velero installed from [Cloud Marketplace](https://yandex.cloud/en/marketplace/products/yc/velero-yc-csi) does not include the `restic` plugin.

* [Guide on Kubernetes cluster backup in Object Storage](../../managed-kubernetes/tutorials/kubernetes-backup.md#backup).

#### 7.10 Check lists are in place for security when creating and using Docker images {#check-list}

Secure Docker image creation and operation practices ensure protection against potential vulnerabilities, malware, and unauthorized access to data. They ensure image integrity and security compliance while also preventing potential threats coming from its deployment in the infrastructure. Use these check lists to meet requirements for secure creation of images:

* [Dockerfile best practices](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/).
* [Kubernetes Security Checklist and Requirements](https://github.com/Vinum-Security/kubernetes-security-checklist/blob/main/README.md).

You can control Dockerfile in your CI/CD pipeline using the [Conftest](https://www.conftest.dev/) utility.

When using minimal images or distroless images without a shell, we recommend using [ephemeral containers](https://kubernetes.io/docs/concepts/workloads/pods/ephemeral-containers/).

| Requirement ID | Severity |
| --- | --- |
| K8S10 | Informational |

{% list tabs group=instructions %}

- Manual check {#manual}

  Make sure you have the check lists in place to meet the requirements for secure creation of images.

{% endlist %}

#### 7.11 The Kubernetes security policy is in place {#security-standards}

The requirements listed in [Kubernetes Pod Security Standards](https://kubernetes.io/docs/concepts/security/pod-security-standards/) allow you to prevent threats related to Kubernetes objects, such as unauthorized access to confidential data and execution of malicious code.

These requirements allow you to ensure security and reliability of applications in a Kubernetes cluster. To achieve compliance, you can either use the built-in Kubernetes tool called [Pod Security Admission Controller](https://kubernetes.io/docs/setup/best-practices/enforcing-pod-security-standards/) or open-source software, e.g., [OPA Gatekeeper](https://github.com/open-policy-agent/gatekeeper) or [Kyverno](https://yandex.cloud/en/marketplace/products/yc/kyverno).

| Requirement ID | Severity |
| --- | --- |
| K8S11 | Medium |

{% list tabs group=instructions %}

- Manual check {#manual}

  Verify compliance with the Kubernetes Pod Security Standards.

{% endlist %}

**Guides and solutions to use:**

* To control compliance with Pod Security Standards, you can also use the following tools within CI/CD:

  * [Kyverno CLI](https://kyverno.io/docs/kyverno-cli/)
  * The gator CLI

* [Kubesec](https://kubesec.io/)

#### 7.12 Audit log collection is set up for incident investigation {#audit-logs}

Events available to the user in the Managed Service for Kubernetes service can be classified as levels:

* Kubernetes API events (Kubernetes audit logging)
* Kubernetes node events
* Kubernetes pod events
* Kubernetes metrics
* Kubernetes flow logs

For more information about setting up audit event logging at different levels, see [Collecting, monitoring, and analyzing Managed Service for Kubernetes audit logs](../domains/kubernetes.md#collection-monitoring-analysis-audit-logs).

In Managed Service for Kubernetes, you can audit the current role model used in the service. To do this, open the Kubernetes cluster page in the [management console](https://console.yandex.cloud), and go to the **Access management** tab.

You can also use:

* [KubiScan](https://github.com/cyberark/KubiScan)
* [Krane](https://github.com/appvia/krane)
* Yandex Audit Trails [audit logs](../../managed-kubernetes/at-ref.md)

| Requirement ID | Severity |
| --- | --- |
| K8S12 | High |

{% list tabs group=instructions %}

- Manual check {#manual}

  Make sure that audit logs are being collected.

{% endlist %}