[Yandex Cloud documentation](../../index.md) > [Yandex MPP Analytics for PostgreSQL](../index.md) > [Step-by-step guides](index.md) > Clusters > Expanding a cluster

# Expanding a cluster

You can expand a Yandex MPP Analytics for PostgreSQL cluster by adding segment hosts to it. You need to add at least two hosts. You can also increase the [number of segments per host](../concepts/index.md).

When a cluster is being expanded, its data is automatically redistributed evenly across all hosts in the cluster (including added hosts). For more information, see [Expanding a Yandex MPP Analytics for PostgreSQL cluster](../concepts/expand.md).

## Expanding a cluster {#add-hosts}

{% list tabs group=instructions %}

- Management console {#console}

    1. Open the [folder dashboard](https://console.yandex.cloud).
    1. Navigate to **Yandex MPP Analytics for PostgreSQL**.
    1. Select a cluster and open the ![hosts-edit](../../_assets/console-icons/cube.svg) **Hosts** tab.
    1. Click **Expand cluster** in the top-right corner.
    1. Specify the cluster expansion settings:

        * **Number of new segment hosts**: Number of segment hosts to expand the cluster by.

            The minimum value (default) is `2`, the maximum value cannot exceed the [limit on the number of segment hosts in the cluster](../concepts/limits.md#limits) (minus existing hosts).

        * **Add segments to host**: Number of segments added per host.

            {% note warning %}
            
            Segments [will be added to all hosts](../concepts/index.md), both new and existing ones.
            
            {% endnote %}
            
            The minimum value (default) is `0` (number of segments per host remains unchanged); the maximum value depends on the [host class](../concepts/instance-types.md).

            The segment host class and the number of segments per host [affect](../concepts/instance-types.md#select-class-segment-hosts) the maximum amount of memory allocated to each Yandex MPP Analytics for PostgreSQL server process. If you select a host class with small RAM and specify a large number of segments, an [error](../qa/cluster-hosts.md#memory-limit) may occur.

        * **Block cluster from load**: [Closing the cluster for access](../concepts/expand.md#setting-close-cluster).

            If this setting is enabled, you cannot connect to the cluster and it receives no new user requests. This will accelerate cluster expansion.
            
            {% note warning %}
            
            If you close the cluster from load and disable [background data redistribution](../concepts/expand.md#setting-delay-redistribution), you will lose access to the cluster until its expansion is complete.
            
            The expansion process can be [time-consuming](../concepts/expand.md#duration).
            
            {% endnote %}

        * **Background data redistribution**: [Using background data redistribution](../concepts/expand.md#setting-delay-redistribution).

            * If background mode is disabled, the redistribution process will start once as soon as preparation for the cluster expansion is over.

                The cluster will remain in the `Updating` status until all cluster expansion stages are completed.

                {% note warning %}

                If data redistribution terminates upon reaching the timeout with some tables remaining unprocessed, you will need to [redistribute those tables manually](cluster-expand.md#start-redistribute).

                {% endnote %}

            * If background mode is enabled, the redistribution process will be delayed. The process will run on a schedule as a [routine maintenance operation](../concepts/maintenance.md#regular-ops).

                The cluster will remain in the `Updating` status only during preparation for cluster expansion.

        * **Redistribution timeout**: [Timeout after which the data redistribution process will be interrupted, in seconds](../concepts/expand.md#setting-duration).

            Reaching the timeout does not stop the process immediately: data redistribution will be completed for the tables currently being processed (`IN PROGRESS` status). You can request the [status of the tables](cluster-expand.md#redistribute-monitoring) if the cluster is not closed from load.
            
            The minimum value is `0`. The timeout will be calculated automatically depending on the cluster configuration and data size.
            
            The maximum value depends on whether background data redistribution is enabled:
            
            * If enabled, the maximum value is `28800` (eight hours).
            * If disabled, the maximum value is not limited.

        * **Number of redistricting streams**: [Number of threads that will be started during the data redistribution process](../concepts/expand.md#setting-parallel).

            Using more threads will speed up data redistribution but it will also increase the cluster load. 
            
            The minimum value (default) is `0` (the number will be calculated automatically depending on the cluster configuration and data size), and the maximum one is `25`.

    1. Click **Expand**.

- CLI {#cli}

    If you do not have the Yandex Cloud CLI yet, [install and initialize it](../../cli/quickstart.md#install).

    The folder used by default is the one specified when [creating](../../cli/operations/profile/profile-create.md) the CLI profile. To change the default folder, use the `yc config set folder-id <folder_ID>` command. You can also specify a different folder for any command using `--folder-name` or `--folder-id`. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

    To expand a cluster:

    1. View the description of the CLI command for expanding a cluster:

        ```bash
        yc managed-greenplum cluster expand --help
        ```

    1. To expand a cluster, run this command:

        ```bash
        yc managed-greenplum cluster expand <cluster_name_or_ID> \
           --segment-host-count <number_of_added_segment_hosts> \
           --add-segments-per-host-count <number_of_added_segments_per_host> \
           --close-cluster <temporary_ban_on_connection_to_cluster> \
           --delay-redistribution <background_data_redistribution> \
           --duration-seconds <data_redistribution_timeout> \
           --parallel <number_of_data_redistribution_threads>
        ```

        Where:

        * `--segment-host-count`: Number of segment hosts to expand the cluster by.

            The minimum value (default) is `2`, the maximum value cannot exceed the [limit on the number of segment hosts in the cluster](../concepts/limits.md#limits) (minus existing hosts).

        * `--add-segments-per-host-count`: Number of segments added per host.

            {% note warning %}
            
            Segments [will be added to all hosts](../concepts/index.md), both new and existing ones.
            
            {% endnote %}
            
            The minimum value (default) is `0` (number of segments per host remains unchanged); the maximum value depends on the [host class](../concepts/instance-types.md).

            The segment host class and the number of segments per host [affect](../concepts/instance-types.md#select-class-segment-hosts) the maximum amount of memory allocated to each Yandex MPP Analytics for PostgreSQL server process. If you select a host class with small RAM and specify a large number of segments, an [error](../qa/cluster-hosts.md#memory-limit) may occur.

        * `--close-cluster`: [Closing the cluster for access](../concepts/expand.md#setting-close-cluster): `true` or `false`.

            If this setting is enabled (`true`), you cannot connect to the cluster and it does not receive new user requests. This will accelerate cluster expansion.
            
            {% note warning %}
            
            If you close the cluster from load and disable [background data redistribution](../concepts/expand.md#setting-delay-redistribution), you will lose access to the cluster until its expansion is complete.
            
            The expansion process can be [time-consuming](../concepts/expand.md#duration).
            
            {% endnote %}

        * `--delay-redistribution`: [Using background data redistribution](../concepts/expand.md#setting-delay-redistribution): `true` or `false`.

            * If background mode is disabled (`false`), the redistribution process will be run once as soon as the cluster expansion preparation is over.
            
                The cluster will remain in the `Updating` status until all cluster expansion stages are completed.
            
                {% note warning %}
            
                If data redistribution terminates upon reaching the timeout with some tables remaining unprocessed, you will need to [redistribute those tables manually](cluster-expand.md#start-redistribute).
            
                {% endnote %}
            
            * If background mode is enabled (`true`), the redistribution process will be delayed. The process will run on a schedule as a [routine maintenance operation](../concepts/maintenance.md#regular-ops).
            
                The cluster will remain in the `Updating` status only during preparation for cluster expansion.

        * `--duration-seconds`: [Timeout after which the data redistribution process will be interrupted, in seconds](../concepts/expand.md#setting-duration).

            Reaching the timeout does not stop the process immediately: data redistribution will be completed for the tables currently being processed (`IN PROGRESS` status). You can request the [status of the tables](cluster-expand.md#redistribute-monitoring) if the cluster is not closed from load.
            
            The minimum value is `0`. The timeout will be calculated automatically depending on the cluster configuration and data size.
            
            The maximum value depends on whether background data redistribution is enabled:
            
            * If enabled, the maximum value is `28800` (eight hours).
            * If disabled, the maximum value is not limited.

        * `--parallel`: [Number of threads that will be started during the data redistribution process](../concepts/expand.md#setting-parallel).

            Using more threads will speed up data redistribution but it will also increase the cluster load. 
            
            The minimum value (default) is `0` (the number will be calculated automatically depending on the cluster configuration and data size), and the maximum one is `25`.

        You can get the cluster ID and name with the [list of clusters in the folder](cluster-list.md#list-clusters).

- REST API {#api}

    1. [Get an IAM token for API authentication](../api-ref/authentication.md) and put it into an environment variable:

        ```bash
        export IAM_TOKEN="<IAM_token>"
        ```

    1. Call the [Cluster.Expand](../api-ref/Cluster/expand.md) method, e.g., via the following [cURL](https://curl.se/) request:

        ```bash
        curl \
            --request POST \
            --header "Authorization: Bearer $IAM_TOKEN" \
            --header "Content-Type: application/json" \
            --url 'https://mdb.api.cloud.yandex.net/managed-greenplum/v1/clusters/<cluster_ID>/expand' \
            --data '{
                      "segmentHostCount": "<number_of_added_segment_hosts>",
                      "addSegmentsPerHostCount": "<number_of_added_segments_per_host>",
                      "duration": "<data_redistribution_timeout>",
                      "parallel": "<number_of_data_redistribution_threads>",
                      "closeCluster": "<temporary_ban_on_connection_to_cluster>",
                      "delayRedistribution": "<background_data_redistribution>"
                    }'
        ```

        You can request the cluster ID with the [list of clusters in the folder](cluster-list.md#list-clusters).

        Request body parameters:

        * `segmentHostCount`: Number of segment hosts to expand the cluster by.

            The minimum value (default) is `2`, the maximum value cannot exceed the [limit on the number of segment hosts in the cluster](../concepts/limits.md#limits) (minus existing hosts).

        * `addSegmentsPerHostCount`: Number of segments added per host.

            {% note warning %}
            
            Segments [will be added to all hosts](../concepts/index.md), both new and existing ones.
            
            {% endnote %}
            
            The minimum value (default) is `0` (number of segments per host remains unchanged); the maximum value depends on the [host class](../concepts/instance-types.md).

            The segment host class and the number of segments per host [affect](../concepts/instance-types.md#select-class-segment-hosts) the maximum amount of memory allocated to each Yandex MPP Analytics for PostgreSQL server process. If you select a host class with small RAM and specify a large number of segments, an [error](../qa/cluster-hosts.md#memory-limit) may occur.

        * `duration`: [Timeout](../concepts/expand.md#setting-duration) after which the data redistribution process will be interrupted, in seconds.

            Reaching the timeout does not stop the process immediately: data redistribution will be completed for the tables currently being processed (`IN PROGRESS` status). You can request the [status of the tables](cluster-expand.md#redistribute-monitoring) if the cluster is not closed from load.
            
            The minimum value is `0`. The timeout will be calculated automatically depending on the cluster configuration and data size.
            
            The maximum value depends on whether background data redistribution is enabled:
            
            * If enabled, the maximum value is `28800` (eight hours).
            * If disabled, the maximum value is not limited.

        * `parallel`: [Number of threads](../concepts/expand.md#setting-parallel) that will be started during the data redistribution process.

            Using more threads will speed up data redistribution but it will also increase the cluster load. 
            
            The minimum value (default) is `0` (the number will be calculated automatically depending on the cluster configuration and data size), and the maximum one is `25`.

        * `closeCluster`: [Closing the cluster](../concepts/expand.md#setting-close-cluster) for access, `true` or `false`.

            If this setting is enabled (`true`), you cannot connect to the cluster and it does not receive new user requests. This will accelerate cluster expansion.
            
            {% note warning %}
            
            If you close the cluster from load and disable [background data redistribution](../concepts/expand.md#setting-delay-redistribution), you will lose access to the cluster until its expansion is complete.
            
            The expansion process can be [time-consuming](../concepts/expand.md#duration).
            
            {% endnote %}

        * `delayRedistribution`: [Using background data redistribution](../concepts/expand.md#setting-delay-redistribution), `true` or `false`.

            * If background mode is disabled (`false`), the redistribution process will be run once as soon as the cluster expansion preparation is over.
            
                The cluster will remain in the `Updating` status until all cluster expansion stages are completed.
            
                {% note warning %}
            
                If data redistribution terminates upon reaching the timeout with some tables remaining unprocessed, you will need to [redistribute those tables manually](cluster-expand.md#start-redistribute).
            
                {% endnote %}
            
            * If background mode is enabled (`true`), the redistribution process will be delayed. The process will run on a schedule as a [routine maintenance operation](../concepts/maintenance.md#regular-ops).
            
                The cluster will remain in the `Updating` status only during preparation for cluster expansion.

    1. Check the [server response](../api-ref/Cluster/expand.md#yandex.cloud.operation.Operation) to make sure your request was successful.

- gRPC API {#grpc-api}

    1. [Get an IAM token for API authentication](../api-ref/authentication.md) and put it into an environment variable:

        ```bash
        export IAM_TOKEN="<IAM_token>"
        ```

    1. Clone the [cloudapi](https://github.com/yandex-cloud/cloudapi) repository:
       
       ```bash
       cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi
       ```
       
       Below, we assume that the repository contents reside in the `~/cloudapi/` directory.

    1. Call the [ClusterService.Expand](../api-ref/grpc/Cluster/expand.md) method, e.g., via the following [gRPCurl](https://github.com/fullstorydev/grpcurl) request:

        ```bash
        grpcurl \
            -format json \
            -import-path ~/cloudapi/ \
            -import-path ~/cloudapi/third_party/googleapis/ \
            -proto ~/cloudapi/yandex/cloud/mdb/greenplum/v1/cluster_service.proto \
            -rpc-header "Authorization: Bearer $IAM_TOKEN" \
            -d '{
                  "cluster_id": "<cluster_ID>",
                  "segment_host_count": "<number_of_added_segment_hosts>",
                  "add_segments_per_host_count": "<number_of_added_segments_per_host>",
                  "duration": "<data_redistribution_timeout>",
                  "parallel": "<number_of_data_redistribution_threads>",
                  "close_cluster": "<temporary_ban_on_connection_to_cluster>",
                  "delay_redistribution": "<background_data_redistribution>"
                }' \
            mdb.api.cloud.yandex.net:443 \
            yandex.cloud.mdb.greenplum.v1.ClusterService.Expand
        ```

        You can request the cluster ID with the [list of clusters in the folder](cluster-list.md#list-clusters).

        Request body parameters:

        * `segment_host_count`: Number of segment hosts to expand the cluster by.

            The minimum value (default) is `2`, the maximum value cannot exceed the [limit on the number of segment hosts in the cluster](../concepts/limits.md#limits) (minus existing hosts).

        * `add_segments_per_host_count`: Number of segments added per host.

            {% note warning %}
            
            Segments [will be added to all hosts](../concepts/index.md), both new and existing ones.
            
            {% endnote %}
            
            The minimum value (default) is `0` (number of segments per host remains unchanged); the maximum value depends on the [host class](../concepts/instance-types.md).

            The segment host class and the number of segments per host [affect](../concepts/instance-types.md#select-class-segment-hosts) the maximum amount of memory allocated to each Yandex MPP Analytics for PostgreSQL server process. If you select a host class with small RAM and specify a large number of segments, an [error](../qa/cluster-hosts.md#memory-limit) may occur.

        * `duration`: [Timeout](../concepts/expand.md#setting-duration) after which the data redistribution process will be interrupted, in seconds.

            Reaching the timeout does not stop the process immediately: data redistribution will be completed for the tables currently being processed (`IN PROGRESS` status). You can request the [status of the tables](cluster-expand.md#redistribute-monitoring) if the cluster is not closed from load.
            
            The minimum value is `0`. The timeout will be calculated automatically depending on the cluster configuration and data size.
            
            The maximum value depends on whether background data redistribution is enabled:
            
            * If enabled, the maximum value is `28800` (eight hours).
            * If disabled, the maximum value is not limited.

        * `parallel`: [Number of threads](../concepts/expand.md#setting-parallel) that will be started during the data redistribution process.

            Using more threads will speed up data redistribution but it will also increase the cluster load. 
            
            The minimum value (default) is `0` (the number will be calculated automatically depending on the cluster configuration and data size), and the maximum one is `25`.

        * `close_cluster`: [Closing the cluster](../concepts/expand.md#setting-close-cluster) for access, `true` or `false`.

            If this setting is enabled (`true`), you cannot connect to the cluster and it does not receive new user requests. This will accelerate cluster expansion.
            
            {% note warning %}
            
            If you close the cluster from load and disable [background data redistribution](../concepts/expand.md#setting-delay-redistribution), you will lose access to the cluster until its expansion is complete.
            
            The expansion process can be [time-consuming](../concepts/expand.md#duration).
            
            {% endnote %}

        * `delay_redistribution`: [Using background data redistribution](../concepts/expand.md#setting-delay-redistribution), `true` or `false`.

            * If background mode is disabled (`false`), the redistribution process will be run once as soon as the cluster expansion preparation is over.
            
                The cluster will remain in the `Updating` status until all cluster expansion stages are completed.
            
                {% note warning %}
            
                If data redistribution terminates upon reaching the timeout with some tables remaining unprocessed, you will need to [redistribute those tables manually](cluster-expand.md#start-redistribute).
            
                {% endnote %}
            
            * If background mode is enabled (`true`), the redistribution process will be delayed. The process will run on a schedule as a [routine maintenance operation](../concepts/maintenance.md#regular-ops).
            
                The cluster will remain in the `Updating` status only during preparation for cluster expansion.

    1. Check the [server response](../api-ref/Cluster/expand.md#yandex.cloud.operation.Operation) to make sure your request was successful.

{% endlist %}

## Data redistribution monitoring {#redistribute-monitoring}

There are several ways to monitor the [data redistribution process](../concepts/expand.md#redistribution):

* If the cluster is [closed from load](../concepts/expand.md#setting-close-cluster) and [background data redistribution](../concepts/expand.md#setting-delay-redistribution) is disabled, you can only track progress using Yandex Cloud monitoring tools.

    [Follow](monitoring.md#monitoring-cluster) the **Expansion progress** chart on the **Monitoring** tab of the cluster management page or request the `gp_expand.*` [metrics](../metrics.md#managed-greenplum-expand-metrics) using [Yandex Monitoring](../../monitoring/concepts/index.md).

* If the cluster is not closed from load, you can also run SQL queries to the `gpexpand` data schema's objects.

    {% note info %}
    
    This schema is available only during cluster expansion, then it gets deleted.
    
    {% endnote %}

    To track the progress of the data redistribution process using SQL queries:

    1. [Connect](connect/index.md) to the `postgres` database as a user with the `mdb_admin` [role](../concepts/cluster-users.md#mdb_admin).
    1. Run a query to get the current process status:

        ```sql
        SELECT dbname, fq_name, rank, status, expansion_started, expansion_finished
        FROM gpexpand.status_detail
        ORDER BY rank;
        ```

    The query result will contain [various details](https://techdocs.broadcom.com/us/en/vmware-tanzu/data-solutions/tanzu-greenplum/7/greenplum-database/ref_guide-system_catalogs-gp_expansion_tables.html) on the data redistribution status. The `status` column will indicate the current processing status for specific tables:

    * `NOT STARTED`: Table data redistribution is scheduled, and the table is in the redistribution queue.

        If the cluster is not [closed from load](../concepts/expand.md#setting-close-cluster), [you can modify the table priority](#table-priority) to change its position in the queue if needed.

    * `IN PROGRESS`: Table data redistribution is in progress.
    * `COMPLETED`: Table data redistribution is completed.
    * `NO LONGER EXISTS`: The table no longer exists.

## Additional actions `resgroup`

### Changing the table redistribution priority {#table-priority}

During [preparation for cluster expansion](../concepts/expand.md#preparation), each table is assigned a rank that affects its redistribution queue priority. The higher its rank, the higher the table's queue priority and, therefore, the faster its data redistribution will be completed.

If you need to, you can change the rank for one or more tables provided the cluster is [not closed from load](../concepts/expand.md#setting-close-cluster). To do this, you must modify the table in the `gpexpand` data schema.

{% note info %}

This schema is available only during cluster expansion, then it gets deleted.

{% endnote %}

To edit the ranks of redistributed tables:

1. [Connect](connect/index.md) to the `postgres` database as a user with the `mdb_admin` [role](../concepts/cluster-users.md#mdb_admin).

1. Run a query to get the current status of the data redistribution process:

    ```sql
    SELECT dbname, fq_name, rank, status, expansion_started, expansion_finished
    FROM gpexpand.status_detail
    ORDER BY rank;
    ```

    This will include full table names (`fq_name`) and their ranks (`rank`).

1. Change the rank for one or more tables:

    ```sql
    UPDATE gpexpand.status_detail
    SET rank=<required_rank>
    WHERE fq_name IN (<list_of_full_table_names>);
    ```

    For more information, see [this Greenplum® guide](https://techdocs.broadcom.com/us/en/vmware-tanzu/data-solutions/tanzu-greenplum/7/greenplum-database/admin_guide-expand-expand-redistribute.html#ranking-tables-for-redistribution).

### Running data redistribution manually {#start-redistribute}

If [background data redistribution](../concepts/expand.md#setting-delay-redistribution) was not used during cluster expansion and the set [timeout for the data redistribution process](../concepts/expand.md#setting-duration) was not long enough, the process may end before data redistribution is over for all tables.

This will leave some of the data distributed unevenly. As a result, the `Unknown error: Partially Distributed Data` error will occur in [cluster logs](cluster-logs.md). To fix the error, run data redistribution manually:

1. Learn the total number of segments in the cluster:

    1. [Get detailed information about a cluster](cluster-list.md#get-cluster).
    1. Multiply the number of segments per host by the number of segment hosts.

1. [Connect](connect/index.md) to the database as a user with the `mdb_admin` [role](../concepts/cluster-users.md#mdb_admin).

1. Find tables with unevenly distributed data:

    ```sql
    SELECT *
        FROM
            (SELECT oid, relname FROM pg_class) pgc
            JOIN
            (SELECT localoid AS oid, numsegments FROM gp_distribution_policy) gpdp
            USING (oid)
        WHERE numsegments != <total_number_of_segments>;
    ```

    A list of tables will be displayed stating the following for each table:
    * `oid`: Table ID.
    * `relname`: Table name.
    * `numsegment`: Number of segments storing the data.

    For more information, see the Greenplum® guide on the [gp_distribution_policy table](https://techdocs.broadcom.com/us/en/vmware-tanzu/data-solutions/tanzu-greenplum/7/greenplum-database/ref_guide-system_catalogs-gp_distribution_policy.html).

1. Run data redistribution using the information you got earlier.

    To redistribute the data of an individual table, run the following request:

    * For an ordinary table:

        ```sql
        ALTER TABLE ONLY <table_name> EXPAND TABLE;
        ```

    * For a partitioned table:

        ```sql
        ALTER TABLE <table_name>
        SET WITH (REORGANIZE=true) <distribution_policy>;
        ```

        To learn the distribution policy value for an individual table partition, run the following query:

        ```sql
        SELECT pg_get_table_distributedby(<partition_OID>) AS distribution_policy;
        ```

_Greenplum® and Greenplum Database® are registered trademarks or trademarks of Broadcom Inc. in the United States and/or other countries._