[Yandex Cloud documentation](../../index.md) > [Tutorials](../index.md) > [Container infrastructure](index.md) > Managed Service for Kubernetes > Monitoring > Managed Service for Kubernetes cluster monitoring with Filebeat OSS

# Yandex Managed Service for Kubernetes cluster monitoring with Filebeat OSS

# Cluster monitoring with Filebeat OSS


[Filebeat OSS](https://www.elastic.co/beats/filebeat) is a plugin for collecting and forwarding logs to the OpenSearch ecosystem. Installed in a [Managed Service for Kubernetes cluster](../../managed-kubernetes/concepts/index.md#kubernetes-cluster), Filebeat collects cluster and [pod](../../managed-kubernetes/concepts/index.md#pod) logs to then forward them to [Yandex Managed Service for OpenSearch](../../managed-opensearch/index.md).

To set up Managed Service for Kubernetes cluster monitoring with Filebeat OSS:

1. [Install Filebeat OSS](#filebeat-oss-install).
1. [Check the result](#check-result).

If you no longer need the resources you created, [delete them](#clear-out).

## Required paid resources {#paid-resources}

* Managed Service for Kubernetes master (see [Managed Service for Kubernetes pricing](../../managed-kubernetes/pricing.md)).
* Managed Service for Kubernetes cluster nodes: Use of computing resources and storage (see [Yandex Compute Cloud pricing](../../compute/pricing.md)).
* Managed Service for OpenSearch cluster: use of computing resources, storage and backup size (see [Managed Service for OpenSearch pricing](../../managed-opensearch/pricing.md)).
* Public IP addresses for Managed Service for OpenSearch cluster hosts and Managed Service for Kubernetes cluster nodes with public access enabled (see [Yandex Virtual Private Cloud pricing](../../vpc/pricing.md)).

## Getting started {#before-you-begin}

1. Create a Managed Service for Kubernetes cluster and [node group](../../managed-kubernetes/concepts/index.md#node-group).

   {% list tabs group=instructions %}

   - Manually {#manual}

     1. If you do not have a [network](../../vpc/concepts/network.md#network) yet, [create one](../../vpc/operations/network-create.md).
     1. If you do not have any [subnets](../../vpc/concepts/network.md#subnet) yet, [create them](../../vpc/operations/subnet-create.md) in the [availability zones](../../overview/concepts/geo-scope.md) where the new Managed Service for Kubernetes cluster and node group will reside.

     1. [Create security groups](../../managed-kubernetes/operations/connect/security-groups.md) for the Managed Service for Kubernetes cluster and its node groups.

        {% note warning %}
        
        The configuration of security groups determines performance and availability of the cluster and the services and applications running in it.
        
        {% endnote %}

     1. [Create a Managed Service for Kubernetes cluster](../../managed-kubernetes/operations/kubernetes-cluster/kubernetes-cluster-create.md) and [node group](../../managed-kubernetes/operations/node-group/node-group-create.md) with any suitable configuration. When creating, specify the preconfigured security groups.

   - Terraform {#tf}

     1. If you do not have Terraform yet, [install it](../infrastructure-management/terraform-quickstart.md#install-terraform).
     1. Download the [file with provider settings](https://github.com/yandex-cloud-examples/yc-terraform-provider-settings/blob/main/provider.tf). Place it in a separate working directory and [specify the parameter values](../infrastructure-management/terraform-quickstart.md#configure-provider).
     1. Download the [k8s-cluster.tf](https://github.com/yandex-cloud-examples/yc-mk8s-cluster-infrastructure/blob/main/k8s-cluster.tf) configuration file for the Managed Service for Kubernetes cluster to the same working directory. This file describes:
        * [Network](../../vpc/concepts/network.md#network).
        * [Subnet](../../vpc/concepts/network.md#subnet).
        * Managed Service for Kubernetes cluster.
        * [Service account](../../iam/concepts/users/service-accounts.md) for the Managed Service for Kubernetes cluster and node group.
        * [Security groups](../../vpc/concepts/security-groups.md) which contain [rules](../../managed-kubernetes/operations/connect/security-groups.md) required for the Managed Service for Kubernetes cluster and its node groups.

            {% note warning %}
            
            The configuration of security groups determines performance and availability of the cluster and the services and applications running in it.
            
            {% endnote %}

     1. Specify the following in the configuration file:
        * [Folder ID](../../resource-manager/operations/folder/get-id.md).
        * [Kubernetes version](../../managed-kubernetes/concepts/release-channels-and-updates.md) for the Managed Service for Kubernetes cluster and node groups.
        * Managed Service for Kubernetes cluster CIDR.
        * Service account name. The name must be unique within the [folder](../../resource-manager/concepts/resources-hierarchy.md#folder).
     1. Run `terraform init` in the directory with the configuration files. This command initializes the provider specified in the configuration files and enables you to use its resources and data sources.
     1. Validate your Terraform configuration files using this command:

        ```bash
        terraform validate
        ```

        Terraform will display any configuration errors detected in your files.
     1. Create the required infrastructure:

        1. Run this command to view the planned changes:
        
           ```bash
           terraform plan
           ```
        
           If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
        
        1. If everything looks correct, apply the changes:
           1. Run this command:
        
              ```bash
              terraform apply
              ```
        
           1. Confirm updating the resources.
           1. Wait for the operation to complete.

        All the required resources will be created in the specified folder. You can check resource availability and their settings in the [management console](https://console.yandex.cloud).

   {% endlist %}

1. [Install kubect](https://kubernetes.io/docs/tasks/tools/install-kubectl) and [configure it to work with the new cluster](../../managed-kubernetes/operations/connect/index.md#kubectl-connect).
1. [Create a Managed Service for OpenSearch cluster with any suitable configuration](../../managed-opensearch/operations/cluster-create.md).

## Install Filebeat OSS {#filebeat-oss-install}

Install Filebeat OSS by following [this guide](../../managed-kubernetes/operations/applications/filebeat-oss.md).

## Check the result {#check-result}

Make sure the Managed Service for OpenSearch cluster receives logs from the Managed Service for Kubernetes cluster:

{% list tabs group=programming_language %}

- Bash {#bash}

  Run this command:

  ```bash
  curl \
    --user admin:<Managed_Service_for_OpenSearch_cluster_admin_user_password> \
    --cacert CA.pem \
    --request GET 'https://<name_of_OpenSearch_host_with_DATA_role>:9200/_cat/indices?v'
  ```

- OpenSearch Dashboards {#opensearch}

  1. Connect to the Managed Service for OpenSearch cluster using [OpenSearch Dashboards](../../managed-opensearch/operations/connect/clients.md#dashboards).
  1. Select the `Global` tenant.
  1. Open the management panel by clicking ![os-dashboards-sandwich](../../_assets/os-dashboards-sandwich.svg).
  1. Under **OpenSearch Plugins**, select **Index Management**.
  1. Navigate to **Indexes**.

{% endlist %}

The list should contain `filebeat-7.12.1-<log_date>` indexes with the Managed Service for Kubernetes cluster logs.

## Delete the resources you created {#clear-out}

Some resources are not free of charge. Delete the resources you no longer need to avoid paying for them.

[Delete the Managed Service for OpenSearch cluster](../../managed-opensearch/operations/cluster-delete.md).

Delete the other resources depending on how you created them:

{% list tabs group=instructions %}

- Manually {#manual}

  1. [Delete the Managed Service for Kubernetes cluster](../../managed-kubernetes/operations/kubernetes-cluster/kubernetes-cluster-delete.md).
  1. [Delete the created subnets](../../vpc/operations/subnet-delete.md) and [networks](../../vpc/operations/network-delete.md).
  1. [Delete the created service account](../../iam/operations/sa/delete.md).

- Terraform {#tf}

  1. In the terminal window, go to the directory containing the infrastructure plan.
  
      {% note warning %}
  
      Make sure the directory has no Terraform manifests with the resources you want to keep. Terraform deletes all resources that were created using the manifests in the current directory.
  
      {% endnote %}
  
  1. Delete resources:
  
      1. Run this command:
  
          ```bash
          terraform destroy
          ```
  
      1. Confirm deleting the resources and wait for the operation to complete.
  
      All the resources described in the Terraform manifests will be deleted.

{% endlist %}