[Yandex Cloud documentation](../../index.md) > [Yandex Cloud Logging](../index.md) > [Tutorials](index.md) > Transferring Yandex MPP Analytics for PostgreSQL cluster logs to Yandex Cloud Logging

# Transferring a Yandex MPP Analytics for PostgreSQL cluster's logs to Yandex Cloud Logging

# Transferring a Yandex MPP Analytics for PostgreSQL cluster's logs to Yandex Cloud Logging

You can set up regular collection of Yandex MPP Analytics for PostgreSQL cluster performance logs. Logs will be delivered to a [log group](../concepts/log-group.md) in Cloud Logging. You can choose between these two types of log groups:

* [Default log group of the cluster folder](#default)
* [Custom log group](#custom)

## Getting started {#before-you-begin}

[Create a service account](../../iam/operations/sa/create.md#create-sa) with the `logging.writer` [role](../security/index.md#logging-writer).

## Transferring data to the default log group {#default}

1. Create a Yandex MPP Analytics for PostgreSQL cluster with active logging and a service account [created earlier](#before-you-begin):

   {% list tabs group=instructions %}

   * Management console {#console}

      1. In the [management console](https://console.yandex.cloud), select the folder where you want to create a cluster.
      1. Navigate to **Yandex MPP Analytics for PostgreSQL**.
      1. Click **Create cluster**.
      1. Enter a name for the cluster.
      1. Select the DBMS version (Greenplum® or Apache Cloudberry™). Learn more about [available DBMS versions](../../managed-greenplum/concepts/overview.md).

      1. Under **Network settings**, select:

         * Cloud network.
         * Security group.
         * Availability zone and subnet.

      1. Under **User**, specify the admin user credentials:

         * **Username** may contain Latin letters, numbers, hyphens, and underscores, but cannot start with a hyphen. It must be from 1 to 32 characters long.
         * The **Password** must be from 8 to 128 characters long.

      1. Under **Advanced settings**:

         * Select the service account with the `logging.writer` role.
         * Enable **Write logs**.
         * To write logs to the default log group, select **Folder** in the **Destination** field.
         * Select the folder whose log group you want to use.
         * Enable the **Yandex MPP Analytics for PostgreSQL logs** and **Command Center logs** options. Use [Log min messages](../../managed-greenplum/concepts/settings-list.md#setting-log-min-messages) under **DBMS settings** to specify the logging level.

      1. Click **Create**.

   * CLI {#cli}

      ```bash
      yc managed-greenplum cluster create <cluster_name> \
         --greenplum-version=<DBMS_version> \
         --environment=PRODUCTION \
         --network-name=<cluster_network_name> \
         --user-name=<VM_user_name> \
         --user-password=<cluster_user_password> \
         --master-config resource-id=s3-c8-m32,`
                        `disk-size=10,`
                        `disk-type=network-hdd \
         --segment-config resource-id=s3-c8-m32,`
                        `disk-size=93,`
                        `disk-type=network-ssd-nonreplicated \
         --zone-id=<availability_zone_ID> \
         --subnet-id=<cluster_subnet_ID> \
         --assign-public-ip=true \
         --security-group-ids=<cluster_security_group_IDs> \
         --service-account <cluster_service_account_ID> \
         --log-enabled \
         --log-greenplum-enabled \
         --log-command-center-enabled \
         --log-folder-id <folder_ID>
      ```

      Where:

      * `--greenplum-version`: DBMS version (Greenplum® or Apache Cloudberry™). Learn more about [available DBMS versions](../../managed-greenplum/concepts/overview.md).

      * `--service-account`: ID of the service account you [created earlier](#before-you-begin).

      * `--log-enabled`: Enables log transfer. Required for other flags responsible for transferring specific logs, e.g., `--log-greenplum-enabled`.

      * `--log-greenplum-enabled`: Transferring DBMS logs.

          You can [set](../../managed-greenplum/operations/update.md#change-gp-settings) the DBMS logging level in the `Log min messages` [parameter](../../managed-greenplum/concepts/settings-list.md#setting-log-min-messages).

      * `--log-command-center-enabled`: Transferring [Command Center](../../managed-greenplum/concepts/command-center.md) logs.

          Command Center logs are trasferred in full; you cannot change the logging level.

      * `--log-folder-id`: ID of the folder the Yandex MPP Analytics for PostgreSQL cluster was created in.

   * Terraform {#tf}

      1. If you do not have Terraform yet, [install it](../../tutorials/infrastructure-management/terraform-quickstart.md#install-terraform).
      1. [Get the authentication credentials](../../tutorials/infrastructure-management/terraform-quickstart.md#get-credentials). You can add them to environment variables or specify them later in the provider configuration file.
      1. [Configure and initialize a provider](../../tutorials/infrastructure-management/terraform-quickstart.md#configure-provider). There is no need to create a provider configuration file manually, you can [download it](https://github.com/yandex-cloud-examples/yc-terraform-provider-settings/blob/main/provider.tf).
      1. Place the configuration file in a separate working directory and [specify the parameter values](../../tutorials/infrastructure-management/terraform-quickstart.md#configure-provider). If you did not add the authentication credentials to environment variables, specify them in the configuration file.

      1. Specify the following parameters in the configuration file with the cluster description:

          ```hcl
          resource "yandex_mdb_greenplum_cluster" "gp-cluster" {
            name               = "<cluster_name>"
            environment        = "PRODUCTION"
            network_id         = "<network_ID>"
            zone               = "<availability_zone_ID>"
            subnet_id          = "<cluster_subnet_ID>"
            assign_public_ip   = true
            version            = "<DBMS_version>"
            master_host_count  = 2
            segment_host_count = 2
            segment_in_host    = 2
            service_account_id = "<service_account_ID>"

            master_subcluster {
              resources {
                resource_preset_id = "s3-c8-m32" # 8 vCPU, 32 GB RAM
                disk_size          = 10          # GB
                disk_type_id       = "network-hdd"
              }
            }

            segment_subcluster {
              resources {
                resource_preset_id = "s3-c8-m32" # 8 vCPU, 32 GB RAM
                disk_size          = 93          # GB
                disk_type_id       = "network-ssd-nonreplicated"
              }
            }

            user_name          = "<VM_user_name>"
            user_password      = "<cluster_user_password>"
            security_group_ids = ["<cluster_security_group_IDs>"]

            logging {
              enabled = true
              greenplum_enabled  = true
              command_center_enabled = true
              folder_id = <folder_ID>
            }
          }
          ```

          Where:

          * `version`: DBMS version (Greenplum® or Apache Cloudberry™). Learn more about [available DBMS versions](../../managed-greenplum/concepts/overview.md).

          * `service_account_id`: ID of the service account you [created earlier](#before-you-begin).

          * `enabled`: Manages log transferring. To enable parameters responsible for transferring specific logs, provide the `true` value.

          * `greenplum_enabled`: Transferring DBMS logs.

              You can [set](../../managed-greenplum/operations/update.md#change-gp-settings) the DBMS logging level in the `Log min messages` [parameter](../../managed-greenplum/concepts/settings-list.md#setting-log-min-messages).

          * `command_center_enabled`: Transferring [Command Center](../../managed-greenplum/concepts/command-center.md) logs.

              Command Center logs are trasferred in full; you cannot change the logging level.

          * `folder_id`: Specify the ID of the folder whose log group you want to use.

      1. Make sure the Terraform configuration files are correct using this command:

          ```bash
          terraform validate
          ```

          Terraform will display any configuration errors detected in your files.

      1. Create the required infrastructure:

          1. Run this command to view the planned changes:
          
             ```bash
             terraform plan
             ```
          
             If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
          
          1. If everything looks correct, apply the changes:
             1. Run this command:
          
                ```bash
                terraform apply
                ```
          
             1. Confirm updating the resources.
             1. Wait for the operation to complete.

   * API {#api}

      Specify the following parameters in the body of the cluster create request ([Cluster.Create](../../managed-greenplum/api-ref/Cluster/create.md) in the REST API or [ClusterService.Create](../../managed-greenplum/api-ref/grpc/Cluster/create.md) in the gRPC API):

      ```json
      {
        ...
        "config": {
          "version": "<DBMS_version>",
          ...
          },
          ...
        },
        ...
        "serviceAccountId": "<service_account_ID>",
        "logging": {
          "enabled": "true",
          "greenplumEnabled": "true",
          "commandCenterEnabled": "true",
          "folderId": "<folder_ID>"
        }
        ...
      }
      ```

      Where:

      * `version`: DBMS version (Greenplum® or Apache Cloudberry™). Learn more about [available DBMS versions](../../managed-greenplum/concepts/overview.md).

      * `serviceAccountId`: ID of the service account you [created earlier](#before-you-begin).

      * `enabled`: Manages log transferring. To enable parameters responsible for transferring specific logs, provide the `true` value.

      * `greenplumEnabled`: Transferring DBMS logs.

          You can [set](../../managed-greenplum/operations/update.md#change-gp-settings) the DBMS logging level in the `Log min messages` [parameter](../../managed-greenplum/concepts/settings-list.md#setting-log-min-messages).

      * `commandCenterEnabled`: Transferring [Command Center](../../managed-greenplum/concepts/command-center.md) logs.

          Command Center logs are trasferred in full; you cannot change the logging level.

      * `folderId`: Specify the ID of the folder whose log group you want to use.

   {% endlist %}

1. Test the transfer of cluster logs to the log group:

   {% list tabs group=instructions %}

   * Management console {#console}

      1. In the management console, navigate to the relevant folder.
      1. Navigate to **Cloud Logging**.
      1. Click the row with the `default` log group.

      The page that opens will show the log group entries.

   * CLI {#cli}

      To view the entries in JSON format, run this command:

      ```bash
      yc logging read --group-name=default --format=json
      ```

      Result:

      ```text
      [
        {
          "uid": "22:********:15",
          "resource": {
            "type": "mdb.greenplum.cluster",
            "id": "c9qao2tkhbdt********"
          },
          "timestamp": "2025-04-25T07:50:21.871157Z",
          "ingested_at": "2025-04-25T07:50:24.553Z",
          "saved_at": "2025-04-25T07:50:26.125189Z",
          "level": "INFO",
          "message": "2025-04-25 10:50:20.209664 MSK,\"monitor\",\"postgres\",p29034,th1258748352,\"localhost\",\"47384\",2025-04-25 07:38:20 MSK,0,con40,cmd20190,seg-1,,,,sx1,\"LOG\",\"00000\",\"statement: \n\t\treset log_min_messages\n\t\",,,,,,,0,,\"postgres.c\",1664,",
          "json_payload": {
            "cid": "c9qao2tkhbdt********",
            "gp_host_type": "master",
            "gp_preferred_role": "primary",
            "hostname": "rc1d-jds7okr4********.mdb.yandexcloud.net",
            "segment_id": "-1",
            "username": "monitor"
          },
          "stream_name": "greenplum-logs-master-primary"
        }
      ]
      ```

   * API {#api}

      To view log group records, use the [Cluster.ListLogs](../../managed-greenplum/api-ref/Cluster/listLogs.md) REST API method or the [LogReadingService/Read](../api-ref/grpc/LogReading/read.md) gRPC API call.

   {% endlist %}

   For more information, see [Reading records](../operations/read-logs.md).

## Sending data to a custom log group {#custom}

1. [Create a log group](../operations/create-group.md) named `my-log-group`.
1. Create a Yandex MPP Analytics for PostgreSQL cluster with active logging and a service account [created earlier](#before-you-begin):

   {% list tabs group=instructions %}

   * Management console {#console}

      1. In the [management console](https://console.yandex.cloud), select the folder where you want to create a cluster.
      1. Navigate to **Yandex MPP Analytics&nbsp;for&nbsp;PostgreSQL**.
      1. Click **Create cluster**.
      1. Enter a name for the cluster.
      1. Select the DBMS version (Greenplum® or Apache Cloudberry™). Learn more about [available DBMS versions](../../managed-greenplum/concepts/overview.md).

      1. Under **Network settings**, select:

         * Cloud network.
         * Security group.
         * Availability zone and subnet.

      1. Under **User**, specify the admin user credentials:

         * **Username** may contain Latin letters, numbers, hyphens, and underscores, but cannot start with a hyphen. It must be from 1 to 32 characters long.
         * The **Password** must be from 8 to 128 characters long.

      1. Under **Advanced settings**:

         * Select the service account with the `logging.writer` role.
         * Enable **Write logs**.
         * To write logs to a custom log group, select **Log group** in the **Destination** field.
         * Select the `my-log-group` log group.
         * Enable the **Yandex MPP Analytics for PostgreSQL logs** and **Command Center logs** options. Use [Log min messages](../../managed-greenplum/concepts/settings-list.md#setting-log-min-messages) under **DBMS settings** to specify the logging level.

      1. Click **Create**.

   * CLI {#cli}

      ```bash
      yc managed-greenplum cluster create <cluster_name> \
         --greenplum-version=<DBMS_version> \
         --environment=PRODUCTION \
         --network-name=<cluster_network_name> \
         --user-name=<VM_user_name> \
         --user-password=<cluster_user_password> \
         --master-config resource-id=s3-c8-m32,`
                        `disk-size=10,`
                        `disk-type=network-hdd \
         --segment-config resource-id=s3-c8-m32,`
                        `disk-size=93,`
                        `disk-type=network-ssd-nonreplicated \
         --zone-id=<availability_zone_ID> \
         --subnet-id=<cluster_subnet_ID> \
         --assign-public-ip=true \
         --security-group-ids=<cluster_security_group_IDs> \
         --service-account <cluster_service_account_ID> \
         --log-enabled \
         --log-greenplum-enabled \
         --log-command-center-enabled \
         --log-group-id <log_group_ID>
      ```

      Where:

      * `--greenplum-version`: DBMS version (Greenplum® or Apache Cloudberry™). Learn more about [available DBMS versions](../../managed-greenplum/concepts/overview.md).

      * `--service-account`: ID of the service account you [created earlier](#before-you-begin).

      * `--log-enabled`: Enables log transfer. Required for other flags responsible for transferring specific logs, e.g., `--log-greenplum-enabled`.

      * `--log-greenplum-enabled`: Transferring DBMS logs.

          You can [set](../../managed-greenplum/operations/update.md#change-gp-settings) the DBMS logging level in the `Log min messages` [parameter](../../managed-greenplum/concepts/settings-list.md#setting-log-min-messages).

      * `--log-command-center-enabled`: Transferring [Command Center](../../managed-greenplum/concepts/command-center.md) logs.

          Command Center logs are trasferred in full; you cannot change the logging level.

      * `--log-group-id`: ID of the log group to write logs to.

   * Terraform {#tf}

      1. If you do not have Terraform yet, [install it](../../tutorials/infrastructure-management/terraform-quickstart.md#install-terraform).
      1. [Get the authentication credentials](../../tutorials/infrastructure-management/terraform-quickstart.md#get-credentials). You can add them to environment variables or specify them later in the provider configuration file.
      1. [Configure and initialize a provider](../../tutorials/infrastructure-management/terraform-quickstart.md#configure-provider). There is no need to create a provider configuration file manually, you can [download it](https://github.com/yandex-cloud-examples/yc-terraform-provider-settings/blob/main/provider.tf).
      1. Place the configuration file in a separate working directory and [specify the parameter values](../../tutorials/infrastructure-management/terraform-quickstart.md#configure-provider). If you did not add the authentication credentials to environment variables, specify them in the configuration file.

      1. Specify the following parameters in the configuration file with the cluster description:

          ```hcl
          resource "yandex_mdb_greenplum_cluster" "gp-cluster" {
            name               = "<cluster_name>"
            environment        = "PRODUCTION"
            network_id         = "<network_ID>"
            zone               = "<availability_zone_ID>"
            subnet_id          = "<cluster_subnet_ID>"
            assign_public_ip   = true
            version            = "<DBMS_version>"
            master_host_count  = 2
            segment_host_count = 2
            segment_in_host    = 2
            service_account_id = "<service_account_ID>"

            master_subcluster {
              resources {
                resource_preset_id = "s3-c8-m32" # 8 vCPU, 32 GB RAM
                disk_size          = 10          # GB
                disk_type_id       = "network-hdd"
              }
            }

            segment_subcluster {
              resources {
                resource_preset_id = "s3-c8-m32" # 8 vCPU, 32 GB RAM
                disk_size          = 93          # GB
                disk_type_id       = "network-ssd-nonreplicated"
              }
            }

            user_name          = "<VM_user_name>"
            user_password      = "<cluster_user_password>"
            security_group_ids = ["<cluster_security_group_IDs>"]

            logging {
              enabled = true
              greenplum_enabled  = true
              command_center_enabled = true
              log_group_id = <log_group_ID>
            }
          }
          ```

          Where:

          * `version`: DBMS version (Greenplum® or Apache Cloudberry™). Learn more about [available DBMS versions](../../managed-greenplum/concepts/overview.md).

          * `service_account_id`: ID of the service account you [created earlier](#before-you-begin).

          * `enabled`: Manages log transferring. To enable parameters responsible for transferring specific logs, provide the `true` value.

          * `greenplum_enabled`: Transferring DBMS logs.

              You can [set](../../managed-greenplum/operations/update.md#change-gp-settings) the DBMS logging level in the `Log min messages` [parameter](../../managed-greenplum/concepts/settings-list.md#setting-log-min-messages).

          * `command_center_enabled`: Transferring [Command Center](../../managed-greenplum/concepts/command-center.md) logs.

              Command Center logs are trasferred in full; you cannot change the logging level.

          * `log_group_id`: ID of the log group to write logs to.

      1. Make sure the Terraform configuration files are correct using this command:

          ```bash
          terraform validate
          ```

          Terraform will display any configuration errors detected in your files.

      1. Create the required infrastructure:

          1. Run this command to view the planned changes:
          
             ```bash
             terraform plan
             ```
          
             If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
          
          1. If everything looks correct, apply the changes:
             1. Run this command:
          
                ```bash
                terraform apply
                ```
          
             1. Confirm updating the resources.
             1. Wait for the operation to complete.

   * API {#api}

      Specify the following parameters in the body of the cluster create request ([Cluster.Create](../../managed-greenplum/api-ref/Cluster/create.md) in the REST API or [ClusterService.Create](../../managed-greenplum/api-ref/grpc/Cluster/create.md) in the gRPC API):

      ```json
      {
        ...
        "config": {
          "version": "<DBMS_version>",
          ...
          },
          ...
        },
        ...
        "serviceAccountId": "<service_account_ID>",
        "logging": {
          "enabled": "true",
          "greenplumEnabled": "true",
          "commandCenterEnabled": "true",
          "logGroupId": "<log_group_ID>"
        }
        ...
      }
      ```

      Where:

      * `version`: DBMS version (Greenplum® or Apache Cloudberry™). Learn more about [available DBMS versions](../../managed-greenplum/concepts/overview.md).

      * `serviceAccountId`: ID of the service account you [created earlier](#before-you-begin).

      * `enabled`: Manages log transferring. To enable parameters responsible for transferring specific logs, provide the `true` value.

      * `greenplumEnabled`: Transferring DBMS logs.

          You can [set](../../managed-greenplum/operations/update.md#change-gp-settings) the DBMS logging level in the `Log min messages` [parameter](../../managed-greenplum/concepts/settings-list.md#setting-log-min-messages).

      * `commandCenterEnabled`: Transferring [Command Center](../../managed-greenplum/concepts/command-center.md) logs.

          Command Center logs are trasferred in full; you cannot change the logging level.

      * `logGroupId`: Log group ID.

   {% endlist %}

1. Test the transfer of cluster logs to the log group:

   {% list tabs group=instructions %}

   * Management console {#console}

      1. In the [management console](https://console.yandex.cloud), navigate to the relevant folder.
      1. Navigate to **Cloud Logging**.
      1. Click the row with the `my-log-group` log group.

      The page that opens will show the records.

   * CLI {#cli}

      To view the entries in JSON format, run this command:

      ```bash
      yc logging read --group-name=my-log-group --format=json
      ```

      Result:

      ```text
      [
        {
          "uid": "22:********:15",
          "resource": {
            "type": "mdb.greenplum.cluster",
            "id": "c9qao2tkhbdt********"
          },
          "timestamp": "2025-04-25T07:50:21.871157Z",
          "ingested_at": "2025-04-25T07:50:24.553Z",
          "saved_at": "2025-04-25T07:50:26.125189Z",
          "level": "INFO",
          "message": "2025-04-25 10:50:20.209664 MSK,\"monitor\",\"postgres\",p29034,th1258748352,\"localhost\",\"47384\",2025-04-25 07:38:20 MSK,0,con40,cmd20190,seg-1,,,,sx1,\"LOG\",\"00000\",\"statement: \n\t\treset log_min_messages\n\t\",,,,,,,0,,\"postgres.c\",1664,",
          "json_payload": {
            "cid": "c9qao2tkhbdt********",
            "gp_host_type": "master",
            "gp_preferred_role": "primary",
            "hostname": "rc1d-jds7okr4********.mdb.yandexcloud.net",
            "segment_id": "-1",
            "username": "monitor"
          },
          "stream_name": "greenplum-logs-master-primary"
        }
      ]
      ```

   * API {#api}

      To view log group records, use the [Cluster.ListLogs](../../managed-greenplum/api-ref/Cluster/listLogs.md) REST API method or the [LogReadingService/Read](../api-ref/grpc/LogReading/read.md) gRPC API call.

   {% endlist %}

   For more information, see [Reading records](../operations/read-logs.md).

_Greenplum® and Greenplum Database® are registered trademarks or trademarks of Broadcom Inc. in the United States and/or other countries._

_Apache® and Apache Cloudberry™ are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries._