[Yandex Cloud documentation](../../index.md) > [Yandex Data Processing](../index.md) > [Step-by-step guides](index.md) > Logs and monitoring > Cluster and host state monitoring

# Yandex Data Processing cluster and host state monitoring

You can monitor the health of a Yandex Data Processing cluster and the state of its individual hosts using the monitoring tools in the management console. These tools display diagnostic information as charts.

To get started with Monitoring metrics, dashboards, or alerts, click **Open in Monium** in the top panel.

Chart update rate:

* Standard hosts and hosts with an increased RAM to vCPU ratio (`memory-optimized`): 15 seconds.
* Hosts with a guaranteed vCPU share under 100% (`burstable`): 150 seconds.

{% note info %}

The most appropriate multiple units (MB, GB, and more) are automatically used in charts.

{% endnote %}

## Cluster state monitoring {#monitoring-cluster}

To view detailed information on the health state of a Yandex Data Processing cluster:

{% list tabs group=instructions %}

- Management console {#console}

  1. Open the [folder dashboard](https://console.yandex.cloud).
  1. Navigate to **Yandex Data Processing**.
  1. Click the name of your cluster and select the **Monitoring** tab.

  This tab displays the following charts:

  * **Active nodes**: Number of running hosts (other than master hosts).
  * **Apps failed**: Number of applications with runtime errors.
  * **Available RAM**: Amount of free RAM (in bytes) available in YARN for hosts in data storage and data processing subclusters.
  * **Available virtual cores**: Number of available cores in YARN.
  * **Containers pending**: Number of containers pending execution in YARN ResourceManager.
  * **Decommissioned nodes**: Number of hosts that have been [decommissioned](../concepts/decommission.md).

{% endlist %}

## Host state monitoring {#monitoring-hosts}

To view detailed information on the state of individual Yandex Data Processing hosts:

{% list tabs group=instructions %}

- Management console {#console}

  1. Open the [folder dashboard](https://console.yandex.cloud).
  1. Navigate to **Yandex Data Processing**.
  1. Click the name of your cluster and select the **Hosts** tab.
  1. Open the VM of the host you want to monitor and select the **Monitoring** tab.

  This tab displays charts with VM resource usage details:

  * **CPU Utilization**: Processor core workload.
  * **Connections quota utilization**: Percentage of available host connections in use.
  * **Disk bytes**: Storage read and write rate, in bytes per second.
  * **Disk operations**: Number of disk operations per second.
  * **Network bytes**: Network data exchange rate, in bytes per second.
  * **Network packets**: Network packet exchange rate, in packets per second.

{% endlist %}


## Integration with Yandex Monitoring {#monitoring-integration}

{% list tabs group=instructions %}

- Management console {#console}

  To set up alerts for [cluster](#monitoring-cluster) health and [host](#monitoring-hosts) state metrics:

  1. In the [management console](https://console.yandex.cloud), select the folder with the cluster for which you want to set up alerts.
  1. Navigate to ![image](../../_assets/console-icons/display-pulse.svg) **Monitoring**.
  1. Under **Service dashboards**, select **Yandex Data Processing**.
  1. In the relevant metric chart, click ![options](../../_assets/console-icons/ellipsis.svg) and select **Create alert**.
  1. If the chart shows multiple metrics, select the data query to generate the metric and click **Continue**. Learn more about the query language in [this Yandex Monitoring guide](../../monitoring/concepts/querying.md).
  1. Set the `Alarm` and `Warning` thresholds for notifications.
  1. Click **Create**.

{% endlist %}

To have other cluster health indicators monitored automatically:

{% list tabs group=instructions %}

- Management console {#console}

    1. [Create an alert](../../monitoring/operations/alert/create-alert.md).
    1. Add a status metric.
    1. In the alert parameters, set the alert thresholds.

{% endlist %}

For a complete list of supported metrics, see [this Monitoring guide](../../monitoring/metrics-ref/data-processing-ref.md).



## Cluster health and status {#cluster-health-and-status}

The **State** of a cluster shows the health of its hosts, while the **Status** shows whether the cluster is started, stopped, or is at an intermediate stage.

To view the health state and status of a cluster:

1. Open the [folder dashboard](https://console.yandex.cloud).
1. Navigate to **Yandex Data Processing**.
1. In the cluster row, hover over the indicator in the **Availability** column.

### Cluster health states {#cluster-health}

State | Description | Suggested actions
:--- | :--- | :---
**ALIVE** | Cluster is operating normally. | No action is required.
**DEGRADED** | Cluster is not running at its full capacity: the state of at least one of the hosts is other than `ALIVE`. | Run the diagnostics:<ul><li>Go to the **Hosts** tab and see which hosts are not working.</li><li>Go to the **Operations** tab and make sure all operations are completed.</li><li>Make sure the cluster is not under maintenance.</li></ul>If you cannot find the cause yourself, [contact support](https://center.yandex.cloud/support).
**DEAD** | The cluster is down: none of its hosts are running. | [Make a support request](https://center.yandex.cloud/support) stating the following:<ul><li>Cluster ID.</li><li>IDs of the last operations performed on it.</li><li>Time the cluster entered the `DEAD` state according to the [availability charts](#monitoring-cluster).</li></ul>
**UNKNOWN** | Cluster state is unknown. | [Make a support request](https://center.yandex.cloud/support) stating the following:<ul><li>Cluster ID.</li><li>IDs of the last operations performed on it.</li><li>Time the cluster entered the `UNKNOWN` state according to the [availability charts](#monitoring-cluster).</li></ul>

### Cluster statuses {#cluster-status}

Status | Description | Suggested actions
:--- | :--- | :---
**CREATING** | Preparing for the first start | Wait a while and get started. The time it takes to create a cluster depends on the host class.
**RUNNING** | The cluster is operating normally | No action is required.
**STOPPING** | The cluster is stopping | After a while, the cluster status will switch to `STOPPED` and the cluster will be disabled. No action is required.
**STOPPED** | The cluster is stopped | Start the cluster to get it running again.
**STARTING** | Starting the cluster that was stopped earlier | After a while, the cluster status will switch to `RUNNING`. Wait a while and get started.
**UPDATING** | Updating the cluster's configuration | Once the update is complete, the cluster will get the status it had prior to the update: `RUNNING` or `STOPPED`.
**ERROR** | Error when performing an operation with the cluster or during a maintenance window | If the cluster remains in this status for a long time, [contact support](https://center.yandex.cloud/support). You can see whether a cluster is available by its status.
**STATUS_UNKNOWN** | The cluster is unable to determine its status | If the cluster remains in this status for a long time, [contact support](https://center.yandex.cloud/support).