[Yandex Cloud documentation](../../index.md) > [Yandex Data Processing](../index.md) > [Step-by-step guides](index.md) > Logs and monitoring > Working with logs

# Working with logs

[Yandex Cloud Logging](../../logging/index.md) collects and displays Yandex Data Processing cluster logs.

To monitor events on the cluster and its individual hosts, specify the relevant [log group](../../logging/concepts/log-group.md) in its settings. You can do this when [creating](cluster-create.md) or [updating](cluster-update.md) the cluster. If no log group is selected for the cluster, the default log group in the cluster directory will be used for sending and storing logs.

For more information, see [Logs](../concepts/logs.md).

## Viewing log entries {#logging-cluster}

{% list tabs group=instructions %}

- Management console {#console}

    1. Open the [folder dashboard](https://console.yandex.cloud).
    1. Navigate to **Yandex Data Processing**.
    1. Click the cluster name.
    1. Under **Configuration**, click the name of the cluster log group. The Cloud Logging page will open.
    1. Click the log group row. This will open the cluster logs.
    1. Optionally, specify the output settings:
        * [Message filter](../concepts/logs.md):
           * Getting the Yandex Data Processing job run output:

             ```ini
             job_id="<job_ID>"
             ```

           * Getting the `stdout` logs for all YARN application containers:

             ```ini
             application_id="<YARN_app_ID>" AND yarn_log_type="stdout"
             ```

           * Getting the `stderr` logs of a YARN container:

             ```ini
             container_id="<YARN_container_ID>" AND yarn_log_type="stderr"
             ```

           * Getting YARN Resource Manager log entries from the cluster master host:

             ```ini
             hostname="<master_host_FQDN>" AND log_type="hadoop-yarn-resourcemanager"
             ```

        * Message logging levels: From `TRACE` to `FATAL`.
        * Number of messages per page.
        * Message interval (either standard or custom).

- CLI {#cli}

  If you do not have the Yandex Cloud CLI yet, [install and initialize it](../../cli/quickstart.md#install).

  See the description of the CLI command for getting logs:

    ```bash
    yc logging read --help
    ```

  Examples:

    * To get logs of the Yandex Data Processing cluster's HDFS NameNode, run this command:

      ```bash
      yc logging read \
         --group-id=<log_group_ID> \
         --resource-ids=<cluster_ID> \
         --filter=log_type=hadoop-hdfs-namenode
      ```

    * To get logs for the last two hours from all Yandex Data Processing clusters assigned to a specific log group, run this command:

      ```bash
      yc logging read \
         --group-id=<log_group_ID> \
         --resource-types=dataproc.cluster \
         --since=2h
      ```

    * To get your cluster's system log over a specific period, run this command:

      ```bash
      yc logging read \
         --group-id <log_group_ID> \
         --resource-ids=<cluster_ID> \
         --filter 'syslog' \
         --since 'YYYY-MM-DDThh:mm:ssZ' \
         --until 'YYYY-MM-DDThh:mm:ssZ'
      ```

      Set the logging period in the `--since` and `--until` parameters. Time format: `YYYY-MM-DDThh:mm:ssZ`, e.g., `2020-08-10T12:00:00Z`. Use the UTC time zone.

    * To get the log for metrics sent from a specific host to [Yandex Monitoring](../../monitoring/index.md), run this command:

      ```bash
      yc logging read \
         --group-id <log_group_ID> \
         --resource-ids=<cluster_ID> \
         --filter 'telegraf and hostname="<host_FQDN>"' \
         --since 'YYYY-MM-DDThh:mm:ssZ' \
         --until 'YYYY-MM-DDThh:mm:ssZ'
      ```

   {% note info %}

   You can skip `--group-id` and specify the log group ID directly.

   {% endnote %}

      To get the host FQDN:

      1. Open the [folder dashboard](https://console.yandex.cloud).
      1. Navigate to **Yandex Data Processing**.
      1. Click the cluster name.
      1. Navigate to the **Hosts** tab.
      1. Copy the host FQDN.

{% endlist %}

## Disabling log sending {#disable-logs}

{% list tabs group=instructions %}

- Management console {#console}

    When [creating](cluster-create.md) or [updating your cluster](cluster-update.md), add the `dataproc:disable_cloud_logging` property set to `true`.
    
- CLI {#cli}

    If you do not have the Yandex Cloud CLI yet, [install and initialize it](../../cli/quickstart.md#install).

    The folder used by default is the one specified when [creating](../../cli/operations/profile/profile-create.md) the CLI profile. To change the default folder, use the `yc config set folder-id <folder_ID>` command. You can also specify a different folder for any command using `--folder-name` or `--folder-id`. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

    When [creating](cluster-create.md) or [updating](cluster-update.md) your cluster, specify `dataproc:disable_cloud_logging=true` in the `--property` parameter or provide an empty string (`""`) instead of the log group ID in the `--log-group-id` parameter:
    
    ```bash
    yc dataproc cluster create <cluster_name> \
       ... \
       --log-group-id=""
    ```

    ```bash
    yc dataproc cluster update <cluster_name_or_ID> \
       --property dataproc:disable_cloud_logging=true
    ```

{% endlist %}

## Storing logs {#logs-storage}

Log collection and storage are billed according to the Cloud Logging [pricing policy](../../logging/pricing.md). The default log retention period is three days. To update the retention period, [edit the log group settings](../../logging/operations/retention-period.md).

For more information about logs, see [this Cloud Logging guide](../../logging/operations/index.md).