[Yandex Cloud documentation](../../index.md) > [Tutorials](../index.md) > [Container infrastructure](index.md) > Managed Service for Kubernetes > Usage Container Registry > Signing and verifying Container Registry Docker images

# Signing and verifying Yandex Container Registry Docker images in Yandex Managed Service for Kubernetes


This scenario describes how to sign [Docker images](../../container-registry/concepts/docker-image.md) using [Cosign](https://docs.sigstore.dev/cosign/overview/) in [Yandex Container Registry](../../container-registry/index.md) and then set up signature verification in [Yandex Managed Service for Kubernetes](../../managed-kubernetes/index.md) using Yandex Key Management Service keys.

To sign Docker images and set up their verification:
1. [Sign a Docker image using Cosign](#cosign).
1. [Create a policy for signature verification](#kyverno).
1. [Check the result](#check-result).

If you no longer need the resources you created, [delete them](#clear-out).


## Required paid resources {#paid-resources}

The support cost for this solution includes:

* Fee for a Managed Service for Kubernetes cluster: using the master and outbound traffic (see [Managed Service for Kubernetes pricing](../../managed-kubernetes/pricing.md)).
* Fee for cluster nodes (VMs): using computing resources, OS, and storage (see [Compute Cloud pricing](../../compute/pricing.md)).
* Fee for public IP addresses if assigned to cluster nodes (see [Virtual Private Cloud pricing](../../vpc/pricing.md#prices-public-ip)).
* Fee for Container Registry [storage](../../container-registry/pricing.md).


## Getting started {#before-begin}

### Set up your infrastructure {#deploy-infrastructure}

{% list tabs group=instructions %}

- Manually {#manual}

  1. [Create these service accounts](../../iam/operations/sa/create.md):
     * [Service account](../../iam/concepts/users/service-accounts.md) for the resources with the `k8s.clusters.agent` and `vpc.publicAdmin` [roles](../../iam/concepts/access-control/roles.md) for the [folder](../../resource-manager/concepts/resources-hierarchy.md#folder) where the [Managed Service for Kubernetes cluster](../../managed-kubernetes/concepts/index.md#kubernetes-cluster) is being created. This service account will be used to create Managed Service for Kubernetes cluster resources.
     * Service account for nodes with the [container-registry.images.puller](../../container-registry/security/index.md#required-roles) role for the folder with the Docker image [registry](../../container-registry/concepts/registry.md). Nodes will use this account to pull the required Docker images from the registry.

     You can use the same service account for both operations.

  1. [Create security groups](../../managed-kubernetes/operations/connect/security-groups.md) for the Managed Service for Kubernetes cluster and its node groups.

        {% note warning %}
        
        The configuration of security groups determines performance and availability of the cluster and the services and applications running in it.
        
        {% endnote %}

  1. [Create a Managed Service for Kubernetes cluster](../../managed-kubernetes/operations/kubernetes-cluster/kubernetes-cluster-create.md#kubernetes-cluster-create) and [node group](../../managed-kubernetes/operations/node-group/node-group-create.md). When creating a cluster, specify the previously created service accounts for resources and nodes and the security group.
  1. [Create a registry in Container Registry](../../container-registry/operations/registry/registry-create.md).

- Terraform {#tf}

  1. If you do not have Terraform yet, [install it](../infrastructure-management/terraform-quickstart.md#install-terraform).
  1. [Get the authentication credentials](../infrastructure-management/terraform-quickstart.md#get-credentials). You can add them to environment variables or specify them later in the provider configuration file.
  1. [Configure and initialize a provider](../infrastructure-management/terraform-quickstart.md#configure-provider). There is no need to create a provider configuration file manually, you can [download it](https://github.com/yandex-cloud-examples/yc-terraform-provider-settings/blob/main/provider.tf).
  1. Place the configuration file in a separate working directory and [specify the parameter values](../infrastructure-management/terraform-quickstart.md#configure-provider). If you did not add the authentication credentials to environment variables, specify them in the configuration file.
  1. Download the [k8s-validate-cr-image.tf](https://github.com/yandex-cloud-examples/yc-mk8s-cr-images-verification/blob/main/k8s-validate-cr-image.tf) configuration file to the same working directory.

     This file describes:
     * [Network](../../vpc/concepts/network.md#network).
     * [Subnet](../../vpc/concepts/network.md#subnet).
     * Container Registry.
     * Managed Service for Kubernetes cluster.
     * Service account for the Managed Service for Kubernetes cluster and node group.
     * [Security groups](../../vpc/concepts/security-groups.md) which contain [rules](../../managed-kubernetes/operations/connect/security-groups.md) required for the Managed Service for Kubernetes cluster and its node groups.

        {% note warning %}
        
        The configuration of security groups determines performance and availability of the cluster and the services and applications running in it.
        
        {% endnote %}

  1. In `k8s-validate-cr-image.tf`, specify the following:
     * [Folder ID](../../resource-manager/operations/folder/get-id.md).
     * [Kubernetes version](../../managed-kubernetes/concepts/release-channels-and-updates.md) for the Managed Service for Kubernetes cluster and node groups.
     * Managed Service for Kubernetes cluster CIDR.
     * Name of the cluster service account.
     * Name of the Container Registry.
  1. Validate your Terraform configuration files using this command:

     ```bash
     terraform validate
     ```

     Terraform will display any configuration errors detected in your files.
  1. Create the required infrastructure:

     1. Run this command to view the planned changes:
     
        ```bash
        terraform plan
        ```
     
        If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
     
     1. If everything looks correct, apply the changes:
        1. Run this command:
     
           ```bash
           terraform apply
           ```
     
        1. Confirm updating the resources.
        1. Wait for the operation to complete.

     All the required resources will be created in the specified folder. You can check resource availability and their settings in the [management console](https://console.yandex.cloud).

{% endlist %}

### Get ready to work with the Managed Service for Kubernetes cluster

1. [Install kubect](https://kubernetes.io/docs/tasks/tools/install-kubectl) and [configure it to work with the new cluster](../../managed-kubernetes/operations/connect/index.md#kubectl-connect).
1. [Install Helm](https://helm.sh/docs/intro/install).

### Add multiple Docker images to the Container Registry registry {#add-docker-images}

1. [Configure](../../container-registry/operations/configure-docker.md) Docker and [get authenticated in Container Registry](../../container-registry/operations/authentication.md).
1. [Create multiple Docker images](../../container-registry/operations/docker-image/docker-image-create.md). One image will be signed using Cosign, while others will remain unsigned.
1. [Push Docker images](../../container-registry/operations/docker-image/docker-image-push.md) to the Container Registry registry.

## Sign a Docker image using Cosign {#cosign}

{% list tabs %}

- Image signature based on asymmetric keys Key Management Service

  1. Install a special Cosign build for your OS:

     * [Linux amd64](https://storage.yandexcloud.net/cosign/linux-amd64/cosign)
     * [MacOS Darwin amd64](https://storage.yandexcloud.net/cosign/macOS-amd64/cosign)
     * [MacOS Darwin arm64](https://storage.yandexcloud.net/cosign/macOS-arm64/cosign)
     * [Windows](https://storage.yandexcloud.net/cosign/windows/cosign.exe)

  1. Get an [IAM token](../../iam/concepts/authorization/iam-token.md) and save it to the `$YC_IAM_TOKEN` environment variable:
     * **Bash:**

       ```bash
       export YC_IAM_TOKEN=$(yc iam create-token)
       ```

     * **PowerShell:**

       ```powershell
       $env:YC_IAM_TOKEN = $(yc iam create-token)
       ```

  1. Log in to Container Registry:
     * **Bash:**

       ```bash
       docker login \
         --username iam \
         --password $YC_IAM_TOKEN \
         cr.yandex
       ```

     * **PowerShell:**

       ```powershell
       docker login `
         --username iam `
         --password $Env:YC_IAM_TOKEN `
         cr.yandex
       ```

     Result:

     ```text
     WARNING! Using --password via the CLI is insecure. Use --password-stdin.
     Login Succeeded
     ```

     {% note info %}

     To avoid using a credential helper for authentication, edit the `${HOME}/.docker/config.json` configuration file to delete the `cr.yandex` domain line from the `credHelpers` section.

     {% endnote %}

  1. Create a digital signature key pair and save it to Key Management Service:

     ```bash
     cosign generate-key-pair \
       --kms yckms:///folder/<folder_ID>/keyname/<key_pair_name>
     ```

     Where:
     * `<folder_ID>`: [ID of the folder](../../resource-manager/operations/folder/get-id.md) to save the new key pair to.
     * `<key_pair_name>`: Name of the signature key pair you are creating.

     Result:

     ```bash
     client.go:183: Using IAM Token from 'YC_IAM_TOKEN' environment variable as credentials
     client.go:310: generated yckms KEY_ID: '<key_pair_ID>'
     Public key written to cosign.pub
     ```

     The utility will return the ID of the created signature key pair and save a public signature key to a local file. Save the key pair ID, you will need it in the next steps.
      
     You can always get the ID of your signature key pair in the [management console](https://console.yandex.cloud) or using a [CLI command](../../cli/cli-ref/kms/cli-ref/asymmetric-signature-key/list.md).
  1. Sign the image in Container Registry:

     ```bash
     cosign sign \
       --key yckms:///<key_pair_ID> \
       cr.yandex/<registry_ID>/<Docker_image_name>:<tag> \
       --tlog-upload=false
     ```

     Where:
     * `<key_pair_ID>`: ID of the signature key pair you got in the previous step.
     * `<registry_ID>`: ID of the Container Registry [registry](../../container-registry/operations/registry/registry-list.md#registry-list) the image for signing is in.
     * `<Docker_image_name>`: Name of the [Docker image](../../container-registry/operations/docker-image/docker-image-list.md#docker-image-list) you are signing in Container Registry.
     * `<tag>`: Tag of the image version to sign.

     Result:

     ```bash
     Pushing signature to: cr.yandex/<registry_ID>/<Docker_image_name>
     ```

     A second object with the `sha256-....sig` tag and `cr.yandex/<registry_ID>/<Docker_image_name>@sha256:...` hash should appear in the Container Registry registry.
  1. Check manually that the Docker image signature is correct:

     ```bash
     cosign verify \
       --key yckms:///<key_pair_ID> \
       cr.yandex/<registry_ID>/<Docker_image_name>:<tag> \
       --insecure-ignore-tlog
     ```

     Where:
     * `<key_pair_ID>`: Signature key pair ID you got earlier.
     * `<registry_ID>`: ID of the Container Registry [registry](../../container-registry/operations/registry/registry-list.md#registry-list) the image is in.
     * `<Docker_image_name>`: [Docker image name](../../container-registry/operations/docker-image/docker-image-list.md#docker-image-list) in the Container Registry registry.
     * `<tag>`: Tag of the image version to verify the signature for.

     Result:

     ```bash
     Verification for cr.yandex/<registry_ID>/<Docker_image_name>:<tag> --
     The following checks were performed on each of these signatures:
     - The cosign claims were validated
     - The signatures were verified against the specified public key

     [{"critical":{"identity":{"docker-reference":"cr.yandex/<registry_ID>/<Docker_image_name>"},"image":{"docker-manifest-digest":"sha256:..."},"type":"cosign container image signature"},"optional":null}]
     ```

- Image signature based on local keys

  1. [Install Cosign](https://docs.sigstore.dev/cosign/installation).
  1. Generate a key pair using Cosign:

     ```bash
     cosign generate-key-pair
     ```

     Set a private key's password and enter it twice.

     Result:

     ```bash
     Enter password for private key:
     Enter password for private key again:
     Private key written to cosign.key
     Public key written to cosign.pub
     ```

  1. Sign the Docker image in the Container Registry registry:

     ```bash
     cosign sign \
         --key cosign.key \
         cr.yandex/<registry_ID>/<Docker_image_name>:<tag>
     ```

     The signed image will be used when [checking results](#check-result).

     Enter the password for the private key. Result:

     ```text
     Enter password for private key:
     Pushing signature to: cr.yandex/<registry_ID>/<Docker_image_name>
     ```

     A second object with the `sha256-....sig` tag and `cr.yandex/<registry_ID>/<Docker_image_name>@sha256:...` hash should appear in the Container Registry registry.
  1. Check manually that the Docker image signature is correct:

     ```bash
     cosign verify \
       --key cosign.pub \
       cr.yandex/<registry_ID>/<Docker_image_name>:<tag>
     ```

     Result:

     ```text
     Verification for cr.yandex/<registry_ID>/<Docker_image_name>:<tag> --
     The following checks were performed on each of these signatures:
     - The cosign claims were validated
     - The signatures were verified against the specified public key

     [{"critical":{"identity":{"docker-reference":"cr.yandex/<registry_ID>/<Docker_image_name>"},"image":{"docker-manifest-digest":"sha256:..."},"type":"cosign container image signature"},"optional":null}]
     ```

{% endlist %}

## Create a policy for signature verification {#kyverno}

1. Create an [authorized key](../../iam/concepts/authorization/key.md) for the service account with the [container-registry.images.puller](../../container-registry/security/index.md#required-roles) role and save it to the file:

   ```bash
   yc iam key create \
     --service-account-name=<service_account_name> \
     --output authorized-key.json
   ```

   Where `--service-account-name`: Name of the service account with the container-registry.images.puller role.
1. Install the [Kyverno](https://kyverno.io/docs/) app in the Managed Service for Kubernetes cluster. You need it to create a policy for verifying Docker image signatures.
   1. Add a repository named `kyverno`:

      ```bash
      helm repo add kyverno https://kyverno.github.io/kyverno/
      ```

      Result:

      ```text
      "kyverno" has been added to your repositories
      ```

   1. Install Kyverno to the `kyverno` namespace:

      ```bash
      helm install kyverno kyverno/kyverno \
        --namespace kyverno \
        --create-namespace \
        --set replicaCount=1 \
        --set imagePullSecrets.regcred.registry=cr.yandex \
        --set imagePullSecrets.regcred.username=json_key \
        --set-file imagePullSecrets.regcred.password=./authorized-key.json
      ```

      Result:

      ```text
      NAME: kyverno
      LAST DEPLOYED: Thu Sep 8 10:43:00 2022
      NAMESPACE: kyverno
      STATUS: deployed
      ...
      ```

1. Create a policy:
   1. Save the `ClusterPolicy` specification to a YAML file named `policy.yaml`:

      ```yaml
      apiVersion: kyverno.io/v1
      kind: ClusterPolicy
      metadata:
        name: check-image
      spec:
        validationFailureAction: enforce
        background: false
        webhookTimeoutSeconds: 30
        failurePolicy: Fail
        rules:
          - name: check-image
            match:
              any:
              - resources:
                  kinds:
                    - Pod
            verifyImages:
            - imageReferences:
              - "cr.yandex/<registry_ID>/*"
              attestors:
              - count: 1
                entries:
                - keys:
                    publicKeys: |-
                      <cosign.pub_contents>
      ```

      {% cut "Example of a filled out policy.yaml file" %}

      ```yaml
      apiVersion: kyverno.io/v1
      kind: ClusterPolicy
      metadata:
        name: check-image
      spec:
        validationFailureAction: enforce
        background: false
        webhookTimeoutSeconds: 30
        failurePolicy: Fail
        rules:
          - name: check-image
            match:
              any:
              - resources:
                  kinds:
                    - Pod
            verifyImages:
            - imageReferences:
              - "cr.yandex/crpd2f2bnrlb********/*"
              attestors:
              - count: 1
                entries:
                - keys:
                    publicKeys: |-
                      -----BEGIN PUBLIC KEY-----
                      MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE1jTu/9rJZZvUFi4bGhlvgMQdIY97
                      7NuGl2zzpV7olAyIu/WiywxI7Fny5tk6JmNPIFvSAtys3c08gfEc********
                      -----END PUBLIC KEY-----
      ```

      {% endcut %}

      {% note info %}

      By default, when you create a policy, a signature verification request is made to the Transparency Log immutable record storage. You can disable it by adding the `rekor: ignoreTlog: true` parameter to the `keys` element of the policy specification. For more information, see the [Kyverno guide](https://kyverno.io/docs/writing-policies/verify-images/sigstore/#ignoring-tlogs-and-sct-verification).

      {% endnote %}

   1. Run this command:

      ```bash
      kubectl apply -f ./policy.yaml
      ```

      Result:

      ```text
      clusterpolicy.kyverno.io/check-image configured
      ```

## Check the result {#check-result}

* Create a [pod](../../managed-kubernetes/concepts/index.md#pod) from the signed Docker image:

  ```bash
  kubectl run pod --image=cr.yandex/<registry_ID>/<Docker_image_name>:<tag>
  ```

  Result:

  ```text
  pod/pod created
  ```

* Create a pod from an unsigned Docker image:

  ```bash
  kubectl run pod2 --image=cr.yandex/<registry_ID>/<unsigned_Docker_image_name>:<tag>
  ```

  Result:

  ```text
  Error from server: admission webhook "mutate.kyverno.svc-fail" denied the request:

  resource Pod/default/pod2 was blocked due to the following policies

  check-image:
    check-image: 
      failed to verify signature for cr.yandex/crpsere9njsa********/alpine:2.0: .attestors[0].entries[0].keys: no matching signatures:
  ```

## Delete the resources you created {#clear-out}

Some resources are not free of charge. Delete the resources you no longer need to avoid paying for them:

{% list tabs group=instructions %}

- Manually {#manual}

  1. [Delete the Managed Service for Kubernetes cluster](../../managed-kubernetes/operations/kubernetes-cluster/kubernetes-cluster-delete.md).
  1. If you reserved a public static IP address for the cluster, [delete it](../../vpc/operations/address-delete.md).
  1. [Delete the service accounts](../../iam/operations/sa/delete.md).
  1. [Delete all Docker images](../../container-registry/operations/docker-image/docker-image-delete.md) from the Container Registry registry.
  1. [Delete the Container Registry](../../container-registry/operations/registry/registry-delete.md).

- Terraform {#tf}

  1. In the terminal window, go to the directory containing the infrastructure plan.
  
      {% note warning %}
  
      Make sure the directory has no Terraform manifests with the resources you want to keep. Terraform deletes all resources that were created using the manifests in the current directory.
  
      {% endnote %}
  
  1. Delete resources:
  
      1. Run this command:
  
          ```bash
          terraform destroy
          ```
  
      1. Confirm deleting the resources and wait for the operation to complete.
  
      All the resources described in the Terraform manifests will be deleted.

{% endlist %}