[Yandex Cloud documentation](../../index.md) > [Yandex Cloud Logging](../index.md) > [Tutorials](index.md) > Visualizing logs in Grafana using the Yandex Cloud Logging plugin

# Visualizing logs in Grafana using the Cloud Logging plugin

# Visualizing logs in Grafana using the Yandex Cloud Logging plugin


The [Yandex Cloud Logging plugin for Grafana](https://github.com/yandex-cloud/grafana-logs-plugin/tree/master) is an extension for Grafana that allows you to add [Cloud Logging](https://yandex.cloud/en/services/logging) as a data source.

{% note info %}

The Cloud Logging plugin works with Grafana versions 11.2.0 or earlier. Menu titles and UI elements may vary depending on the version.

{% endnote %}

To visualize logs:

1. [Install the plugin](#install-plugin).
1. [Create a service account](#create-account).
1. [Create an authorized key for the service account](#create-key).
1. [Create a log group](#create-group).
1. [Add entries to the log group](#add-records).
1. [Connect a data source in Grafana](#connect-plugin).
1. [View the logs in Grafana](#see-logs).

If you no longer need the resources you created, [delete them](#clear-out).

## Get your cloud ready {#before-begin}

Sign up for Yandex Cloud and create a [billing account](../../billing/concepts/billing-account.md):
1. Navigate to the [management console](https://console.yandex.cloud) and log in to Yandex Cloud or create a new account.
1. On the **[Yandex Cloud Billing](https://center.yandex.cloud/billing/accounts)** page, make sure you have a billing account linked and it has the `ACTIVE` or `TRIAL_ACTIVE` [status](../../billing/concepts/billing-account-statuses.md). If you do not have a billing account, [create one](../../billing/quickstart/index.md) and [link](../../billing/operations/pin-cloud.md) a cloud to it.

If you have an active billing account, you can create or select a [folder](../../resource-manager/concepts/resources-hierarchy.md#folder) for your infrastructure on the [cloud page](https://console.yandex.cloud/cloud).

[Learn more about clouds and folders here](../../resource-manager/concepts/resources-hierarchy.md).

### Required paid resources {#paid-resources}

The cost of resources includes a fee for logging operations and log storage in a log group (see [Yandex Cloud Logging pricing](../pricing.md)).

## Install the plugin {#install-plugin}

1. [Download](https://github.com/yandex-cloud/grafana-logs-plugin/releases) the archive with the latest plugin version.
1. Unpack the archive to the directory with plugins. The path to the directory with plugins is specified in the [Grafana configuration](https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/).

   ```bash
   unzip <path_to_archive> -d <path_to_plugin_directory>
   ```

   {% note info %}

   In macOS, after you unpack the plugin archive, manually run the `/opt/homebrew/var/lib/grafana/plugins/yandexcloud-logging-datasource/yc-logs-plugin_darwin_arm64` file and [allow launching third-party applications](https://support.apple.com/en-us/HT202491#openanyway) in the system settings.

   {% endnote %}

1. Allow loading an unsigned plugin. To do this, specify the plugin name in the `allow_loading_unsigned_plugins` parameter of the Grafana configuration file:

   ```text
   allow_loading_unsigned_plugins = yandexcloud-logging-datasource
   ```

   For more information about loading unsigned plugins, see [this Grafana guide](https://grafana.com/docs/grafana/latest/administration/plugin-management/#allow-unsigned-plugins).

1. Restart the Grafana server:

   {% list tabs group=operating_system %}

   - Linux {#linux}

     ```bash
     sudo systemctl restart grafana-server
     ```

   - Windows {#windows}

     1. Press **Win+R**.
     1. In the window that opens, enter `services.msc` and click **OK**.
     1. Right-click the line with `Grafana` and select **Restart**.

   - macOS {#macos}

     ```bash
     brew services restart grafana
     ```

   {% endlist %}

## Create a service account {#create-account}

{% list tabs group=instructions %}

- Management console {#console}

  1. In the [management console](https://console.yandex.cloud), select the folder where you want to create a service account.
  1. Navigate to **Identity and Access Management**.
  1. Click **Create service account**.
  1. Name the service account: `grafana-plugin`.
  1. Click **Add role** and select `logging.reader`.
  1. Click **Create**.

- CLI {#cli}

  If you do not have the Yandex Cloud CLI yet, [install and initialize it](../../cli/quickstart.md#install).

  The folder used by default is the one specified when [creating](../../cli/operations/profile/profile-create.md) the CLI profile. To change the default folder, use the `yc config set folder-id <folder_ID>` command. You can also specify a different folder for any command using `--folder-name` or `--folder-id`. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

  1. Create a service account named `grafana-plugin`:

     ```bash
     yc iam service-account create --name grafana-plugin
     ```

     Result:

     ```text
     id: nfersamh4s**********
     folder_id: b1gc1t4cb6**********
     created_at: "2023-09-26T10:36:29.726397755Z"
     name: grafana-plugin
     ```

     Save the ID of the `grafana-plugin` service account (`id`) and the ID of the folder where you created it (`folder_id`).

  1. Assign the `logging.reader` role for the folder to the service account:

     ```bash
     yc resource-manager folder add-access-binding <folder_ID> \
       --role logging.reader \
       --subject serviceAccount:<service_account_ID>
     ```

     Result:

     ```text
     done (1s)
     ```

- Terraform {#tf}

  
  If you do not have Terraform yet, [install it and configure the Yandex Cloud provider](../../tutorials/infrastructure-management/terraform-quickstart.md#install-terraform).
  
  
  To manage infrastructure using Terraform under a service account or user accounts (a Yandex account, a federated account, or a local user), [authenticate](../../terraform/authentication.md) using the appropriate method.


  1. In the configuration file, specify the service account properties:

     ```hcl
     resource "yandex_iam_service_account" "grafana-plugin" {
       name        = "grafana-plugin"
       folder_id   = "<folder_ID>"
     }

     resource "yandex_resourcemanager_folder_iam_member" "reader" {
       folder_id = "<folder_ID>"
       role      = "logging.reader"
       member    = "serviceAccount:${yandex_iam_service_account.grafana-plugin id}"
     }
     ```

     Where:

     * `name`: Service account name. This is a required setting.
     * `folder_id`: [Folder ID](../../resource-manager/operations/folder/get-id.md). This is an optional setting. It defaults to the value specified in the provider settings.
     * `role`: Role to assign.

     For more on the properties of the `yandex_iam_service_account` resource in Terraform, see [this provider guide](../../terraform/resources/iam_service_account.md).

  1. Make sure the configuration files are correct.

     1. In the terminal, navigate to the directory where you created your configuration file.
     1. Run a check using this command:

         ```bash
         terraform plan
         ```

     If the configuration is correct, the terminal will display information about the service account. Otherwise, Terraform will show any detected errors.

  1. Deploy the cloud resources.

  1. If the configuration is correct, run this command:

     ```bash
     terraform apply
     ```

  1. Confirm creating the service account by typing `yes` in the terminal and pressing **Enter**.

     This will create the service account. You can check it using the [management console](https://console.yandex.cloud) or this [CLI](../../cli/quickstart.md) command:

     ```bash
     yc iam service-account list
     ```

- API {#api}

  To create a service account, use the [create](../../iam/api-ref/ServiceAccount/create.md) REST API method for the [ServiceAccount](../../iam/api-ref/ServiceAccount/index.md) resource or the [ServiceAccountService/Create](../../iam/api-ref/grpc/ServiceAccount/create.md) gRPC API call.

  To assign the `logging.reader` role for a folder to a service account, use the [setAccessBindings](../../iam/api-ref/ServiceAccount/setAccessBindings.md) method for the [ServiceAccount](../../iam/api-ref/ServiceAccount/index.md) resource or the [ServiceAccountService/SetAccessBindings](../../iam/api-ref/grpc/ServiceAccount/setAccessBindings.md) gRPC API call.

{% endlist %}

## Create an authorized key for the service account {#create-key}

{% list tabs group=instructions %}

- Management console {#console}

  1. In the [management console](https://console.yandex.cloud), select the folder the service account belongs to.
  1. Navigate to **Identity and Access Management**.
  1. In the left-hand panel, select ![FaceRobot](../../_assets/console-icons/face-robot.svg) **Service accounts**.
  1. In the list that opens, select the `grafana-plugin` service account.
  1. Click **Create new key** in the top panel.
  1. Select **Create authorized key**.
  1. Select the encryption algorithm.
  1. Enter a description of the key so that you can easily find it in the management console.
  1. Click **Create**.
  1. In the window that opens, click **Download file with keys**.
  1. Click **Close**.

- CLI {#cli}

  Create authorized keys for the `grafana-plugin` service account:

  ```bash
  yc iam key create --service-account-name grafana-plugin -o authorized_key.json
  ```

  If successful, a private key (`privateKey`) and a public key ID (`id`) will be written to the `authorized_key.json` file.

  Key file example:

  ```json
  {
     "id": "lfkoe35hsk**********",
     "service_account_id": "ajepg0mjt0**********",
     "created_at": "2023-10-10T10:04:56Z",
     "key_algorithm": "RSA_2048",
     "public_key": "-----BEGIN PUBLIC KEY-----\n...\n-----END PUBLIC KEY-----\n",
     "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n"
  }
  ```

- Terraform {#tf}

  1. In the configuration file, specify the properties of the resources you want to create:

     * `service_account_id`: Service account [ID](../../iam/operations/sa/get-id.md). This is a required setting.
     * `description`: Key description. This is an optional setting.
     * `key_algorithm`: Key generation algorithm. This is an optional setting. The default algorithm is `RSA_2048`. For more information about the acceptable values, see [this API guide](../../iam/api-ref/Key/index.md).

     Here is an example of the configuration file structure:

     ```hcl
     resource "yandex_iam_service_account_key" "sa-auth-key" {
       service_account_id = "<service_account_ID>"
       description        = "<key_description>"
       key_algorithm      = "<key_generation_algorithm>"
     }
     ```

     For more information about the resources you can create with Terraform, see [this provider guide](../../terraform/resources/iam_service_account_key.md).

  1. Make sure the configuration files are correct.

     1. In the terminal, navigate to the directory where you created your configuration file.
     1. Run a check using this command:

        ```bash
        terraform plan
        ```

     If the configuration is correct, the terminal will display a list of the resources and their settings. Otherwise, Terraform will show any detected errors.

  1. Deploy the cloud resources.

     1. If the configuration is correct, run this command:

        ```bash
        terraform apply
        ```

     1. Confirm creating the resources by typing `yes` and pressing **Enter**.

        This will create all the resources you need in the specified folder. You can check the new resources and their settings using the [management console](https://console.yandex.cloud) or this CLI command:

        ```bash
        yc iam key list --service-account-id <service_account_ID>
        ```

- API {#api}

  To create an access key, use the [create](../../iam/api-ref/Key/create.md) REST API method for the [Key](../../iam/api-ref/Key/index.md) resource or the [KeyService/Create](../../iam/api-ref/grpc/Key/create.md) gRPC API call.

  Example of a request using cURL for the `create` REST API method:

  ```bash
  curl --request POST \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer <IAM_token>" \
    --data '{"serviceAccountId": "<service_account_ID>"}' \
    https://iam.api.cloud.yandex.net/iam/v1/keys
  ```

  Where:

  * `<IAM_token>`: IAM token of the user with permissions to create keys for the specified service account.
  * `<service_account_id>`: ID of the service account for which the keys are being created.

  If successful, the server response will contain the private key (`privateKey`) and public key ID (`id`). Save this data. You will not be able to get the secret key again.

  Example of a server response:

  ```json
  {
      "key": {
          "createdAt": "2023-10-10T10:55:00+00:00",
          "description": "",
          "id": "lfkoe35hsk**********",
          "keyAlgorithm": "RSA_2048",
          "publicKey": "-----BEGIN PUBLIC KEY-----\n...\n-----END PUBLIC KEY-----\n",
          "serviceAccountId": "ajepg0mjt0**********"
      },
      "privateKey": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n"
  }
  ```

{% endlist %}

## Create a log group {#create-group}

{% list tabs group=instructions %}

- Management console {#console}

  1. In the [management console](https://console.yandex.cloud), go to the folder where you created the `grafana-plugin` service account.
  1. Navigate to **Cloud Logging**.
  1. Click **Create group**.
  1. Enter `grafana-plugin` as the log group name.
  1. Set the log group entry retention period.
  1. Click **Create group**.

- CLI {#cli}

  To create a log group, run this command:

  ```bash
  yc logging group create \
    --name=grafana-plugin \
    --retention-period=<retention_period> \
  ```

  Where:

  * `--name`: Log group name.
  * `--retention-period`: Retention period for log group entries.

  Result:

  ```text
  done (1s)
  id: af3flf29t8**********
  folder_id: aoek6qrs8t**********
  cloud_id: aoegtvhtp8**********
  created_at: "2023-09-26T09:56:38.970Z"
  name: grafana-plugin
  status: ACTIVE
  retention_period: 3600s
  ```

- Terraform {#tf}

  1. In the configuration file, specify the log group properties:

     ```hcl
     provider "yandex" {
       cloud_id  = "<cloud_ID>"
       folder_id = "<folder_ID>"
       zone      = "ru-central1-a"
     }

     resource "yandex_logging_group" "grafana-plugin" {
       name             = "grafana-plugin"
       folder_id        = "<folder_ID>"
       retention_period = "1h"
     }
     ```

     Where:

     * `name`: Log group name.
     * `folder_id`: [Folder ID](../../resource-manager/operations/folder/get-id.md).
     * `retention_period`: Retention period for log group entries.

     For more on the properties of the `yandex_logging_group` resource in Terraform, see [this provider guide](../../terraform/resources/logging_group.md).

  1. Make sure the configuration files are correct.

     1. In the terminal, navigate to the directory where you created your configuration file.
     1. Run a check using this command:

        ```bash
        terraform plan
        ```

     If the configuration is correct, the terminal will display a list of the resources and their settings. Otherwise, Terraform will show any detected errors.

  1. Deploy the cloud resources.

  1. If the configuration is correct, run this command:

     ```bash
     terraform apply
     ```

  1. Confirm creating the resources by typing `yes` and pressing **Enter**.

     This will create all the resources you need in the specified folder. You can check the new resources and their settings using the [management console](https://console.yandex.cloud) or this [CLI](../../cli/quickstart.md) command:

     ```bash
     yc logging group list
     ```

- API {#api}

  To create a log group, use the [create](../api-ref/LogGroup/create.md) REST API method for the [LogGroup](../api-ref/LogGroup/index.md) resource or the [LogGroupService/Create](../api-ref/grpc/LogGroup/create.md) gRPC API call.

{% endlist %}

## Add entries to the log group {#add-records}

{% list tabs group=instructions %}

- CLI {#cli}

  To add entries to a log group, run this command:

  * Linux, macOS:

    ```bash
    yc logging write \
      --group-name=grafana-plugin \
      --message="My message" \
      --level=INFO
    ```

  * Windows (cmd):

    ```bash
    yc logging write ^
      --group-name=grafana-plugin ^
      --message="My message" ^
      --level=INFO
    ```

  * Windows (PowerShell):

    ```bash
    yc logging write `
      --group-name=grafana-plugin `
      --message="My message" `
      --level=INFO
    ```

    Where:

    * `--group-name`: Name of the log group to add entries to.
    * `--message`: Message.
    * `--level`: Logging level.

   {% note info %}

   You can skip the `--group-name` and `--message` parameters and provide only the values, e.g., `grafana-plugin "My message"`.

   {% endnote %}

- API {#api}

  To add entries to the log group, use the [LogIngestionService/Write](../api-ref/grpc/LogIngestion/write.md) gRPC API call.

{% endlist %}

## Connect a data source in Grafana {#connect-plugin}

1. In your browser, go to `http://localhost:3000/`.

   {% note info %}

   By default, Grafana uses port 3000, unless you [specified a different one](https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/#http_port) in the configuration file.

   {% endnote %}

1. In the left-hand panel, select **Connections** → **Add new connection**.
1. In the list of sources, select **Yandex Cloud Logging**.
1. Click **Add new data source**.
1. Under **Secret config**, in the **API Key** field, paste the contents of the `authorized_key.json` file with the [authorized keys](#create-key).
1. Under **SDK config**, in the **Folder ID** field, specify the ID of the folder with the `grafana-plugin` log group.
1. Click **Save & test**.

## View the logs in Grafana {#see-logs}

1. In the Grafana interface, select **Explore** in the left-hand panel.
1. In the top-left corner, select the **Yandex Cloud Logging** data source from the drop-down list.
1. In the query editor for the data source:

   1. Select the ID of the `grafana-plugin` log group in the **Group** field.
   1. Enter your query written in the [filter expression language](../concepts/filter.md) in the **Filter query** field.
   1. In the top-right corner, click **Run query**.

      You will see a histogram with log group entries in the **Logs volume** section.

## How to delete the resources you created {#clear-out}

To stop paying for the resources you created, [delete the log group](../operations/delete-group.md).