[Yandex Cloud documentation](../../index.md) > [Yandex Managed Service for OpenSearch](../index.md) > [Step-by-step guides](index.md) > Clusters > Creating a cluster # Creating an OpenSearch cluster A Managed Service for OpenSearch cluster is a group of multiple interlinked OpenSearch and [Dashboards](https://opensearch.org/docs/latest/dashboards/index/) hosts. A cluster provides high search performance by distributing search and indexing tasks across all cluster hosts with the `DATA` role. To learn more about roles in the cluster, see [Host roles](../concepts/host-roles.md). The available disk types [depend](../concepts/storage.md) on the selected [host class](../concepts/instance-types.md). For more information, see [Resource relationships](../concepts/index.md). ## Roles for creating a cluster {#roles} To create a Managed Service for OpenSearch cluster, you need the [vpc.user](../../vpc/security/index.md#vpc-user) role and the [managed-opensearch.editor](../security/index.md#managed-opensearch-editor) role or higher. To attach your service account to a cluster, e.g., to [use Yandex Object Storage](s3-access.md), make sure your Yandex Cloud account has the [iam.serviceAccounts.user](../../iam/security/index.md#iam-serviceAccounts-user) role or higher. For more information about assigning roles, see [this Yandex Identity and Access Management guide](../../iam/operations/roles/grant.md). ## Creating a cluster {#create-cluster} When creating a cluster, you need to specify individual parameters for each [host group](../concepts/host-roles.md). To reduce load on hosts with the `DATA` role, we recommend you put hosts with the `MANAGER` role into a separate group. {% list tabs group=instructions %} - Management console {#console} To create a Managed Service for OpenSearch cluster: 1. In the [management console](https://console.yandex.cloud), select the folder where you want to create a cluster. 1. Navigate to **Managed Service for OpenSearch**. 1. Click **Create cluster**. 1. Under **Basic parameters**: 1. Enter a name for the cluster. It must be unique within the folder. 1. Optionally, enter a description for the cluster. 1. Select the environment where you want to create your cluster (you cannot change the environment once the cluster is created): * `PRODUCTION`: For stable versions of your applications. * `PRESTABLE`: For testing purposes. The prestable environment is similar to the production environment and likewise covered by an SLA, but it is the first to get new features, improvements, and bug fixes. In the prestable environment, you can test new versions for compatibility with your application. 1. Select the OpenSearch version. 1. Select the [plugins](plugins.md#supported-plugins) you want to install in the cluster. 1. Under **Network settings**, select the cloud network to host your cluster and security groups for cluster network traffic. You may need to additionally [set up security groups](connect/index.md#security-groups) to be able connect to the cluster. 1. Under **Virtual node group 1**, configure your `OpenSearch` [host group](../concepts/host-roles.md): 1. Select the host group type: `OpenSearch`. 1. Enter a name for the host group. It must be unique within the cluster. 1. Select the `DATA` and `MANAGER` [host roles](../concepts/host-roles.md). 1. Select the platform, host type, and host class. The host class defines the specifications of virtual machines OpenSearch nodes will be deployed on. All available options are listed under [Host classes](../concepts/instance-types.md). 1. Select the [disk type](../concepts/storage.md) and data storage size. The type you select determines the increments for changing the disk size: * Network HDDs and SSDs: In increments of 1 GB. * Local SSDs: * For **Intel Cascade Lake**: In increments of 100 GB. * For **Intel Ice Lake** and **AMD Zen 4**: In 368 GB increments. * Non-replicated SSDs: In increments of 93 GB. 1. Optionally, select **Encrypted disk** to encrypt the disk with a [custom KMS key](../../kms/concepts/key.md). * To [create](../../kms/operations/key.md#create) a new key, click **Create**. * To use a previously created key, select it in the **KMS key** field. Learn more about disk encryption in [Storage](../concepts/storage.md#disk-encryption). 1. Optionally, set up automatic storage expansion: * In the **Increase size** field, specify the conditions for the actions below: * Storage expansion during the [next maintenance window](../concepts/maintenance.md#maintenance-window) once the fill level exceeds the specified percentage. If you set this condition, configure the maintenance window schedule. * Storage expansion immediately once the fill level exceeds the specified percentage. {% note warning %} If you set both conditions, make sure the immediate expansion threshold is higher than the scheduled one. {% endnote %} * Specify the maximum storage size after the expansion in the **Maximum storage size** field. {% note warning %} * You cannot decrease the storage size after automatic expansion. * When scaling your storage, the cluster hosts will be unavailable. {% endnote %} 1. Specify how hosts should be distributed across [availability zones](../../overview/concepts/geo-scope.md) and subnets. 1. Select the number of hosts to create. 1. Enable **Public access** if you want to allow [connections](connect/index.md) to hosts over the internet. 1. Configure the `Dashboards` [host group](../concepts/host-roles.md#dashboards) under **Virtual node group 2**, if required: 1. Select the platform, host type, and host class. 1. Configure the storage the same way as for `OpenSearch` hosts. 1. Specify how hosts should be distributed across availability zones and subnets. 1. Select the number of hosts to create. 1. Enable **Public access** if you want to allow [connections](connect/index.md) to hosts over the internet. {% note tip %} You can use OpenSearch Dashboards even if you cannot get public access to your hosts, e.g., for security reasons. To do this, proxy the connections via the virtual machine in Yandex Compute Cloud that resides in the same [network](../../vpc/concepts/network.md#network) as the cluster. For more information, see [Connecting to OpenSearch Dashboards](connect/clients.md#dashboards). {% endnote %} 1. If required, click **Add virtual node group** to add another host group or more. 1. Under **Service settings**: 1. Enter the password for `admin`. The password must include three groups of characters out of these four: * Lowercase Latin letters * Uppercase Latin letters * Numbers * Special characters The password must be from 10 to 72 characters long. `admin` is a dedicated user required to manage the cluster and cannot be deleted. Such a user has the `superuser` role and can perform any operations with the cluster. {% note tip %} However, for routine jobs, we recommend creating individual regular users. For more information, see [Managing OpenSearch users](cluster-users.md). {% endnote %} 1. If required, change additional cluster settings: * **Maintenance**: [Maintenance window](../concepts/maintenance.md) settings: * To enable maintenance at any time, select **At any time** (default). * To specify the preferred maintenance start time, select **By schedule** and specify the day of the week and the UTC time interval. For example, you can choose a time when the cluster is least loaded. Both active and stopped clusters are subject to maintenance. Maintenance operations include DBMS updates, patches, etc. * **Service account**: Account for [accessing Yandex Object Storage](s3-access.md) as a repository of OpenSearch snapshots. Learn more about service accounts in [this Yandex Identity and Access Management guide](../../iam/concepts/users/service-accounts.md). * **Deletion protection**: Manages cluster protection against accidental deletion. Even with cluster deletion protection enabled, it is still possible to delete a user or connect to the cluster manually and delete the data. 1. Click **Create cluster**. - CLI {#cli} If you do not have the Yandex Cloud CLI yet, [install and initialize it](../../cli/quickstart.md#install). The folder used by default is the one specified when [creating](../../cli/operations/profile/profile-create.md) the CLI profile. To change the default folder, use the `yc config set folder-id ` command. You can also specify a different folder for any command using `--folder-name` or `--folder-id`. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions. To create a Managed Service for OpenSearch cluster: 1. View the description of the CLI command for creating a cluster: ```bash yc managed-opensearch cluster create --help ``` 1. Specify the cluster properties in this command (the example does not show all that are available): ```bash yc managed-opensearch cluster create \ --name \ --description \ --labels \ --environment \ --network-name \ --security-group-ids \ --service-account-name \ --delete-protection \ --maintenance schedule=,` `weekday=,` `hour= \ --disk-encryption-key-id \ --version \ --read-admin-password \ --serverless-access= \ --plugins \ --advanced-params \ --opensearch-node-group name=,` `resource-preset-id=,` `disk-size=,` `disk-type-id=,` `hosts-count=,` `zone-ids=,` `subnet-names=,` `assign-public-ip=,` `roles= \ --dashboards-node-group name=,` `resource-preset-id=,` `disk-size=,` `disk-type-id=,` `hosts-count=,` `zone-ids=,` `subnet-names=,` `assign-public-ip= ``` Where: * `--labels`: [Yandex Cloud labels](../../resource-manager/concepts/labels.md) in `=` format. You can use them to logically separate resources. * `--environment`: Environment: * `production`: For stable versions of your applications. * `prestable`: For testing purposes. The prestable environment is similar to the production environment and likewise covered by an SLA, but it is the first to get new features, improvements, and bug fixes. In the prestable environment, you can test new versions for compatibility with your application. * `--service-account-name`: Name of the service account for [accessing Yandex Object Storage](s3-access.md) as a repository of OpenSearch snapshots. Learn more about service accounts in [this Yandex Identity and Access Management guide](../../iam/concepts/users/service-accounts.md). * `--deletion-protection`: Cluster protection from accidental deletion, `true` or `false`. Even with cluster deletion protection enabled, it is still possible to delete a user or connect to the cluster manually and delete the data. * `--maintenance`: Maintenance window settings: * To allow maintenance at any time, do not specify the `--maintenance` parameter in the command (the default configuration) or specify `--maintenance schedule=anytime`. * To specify the preferred start time for maintenance, specify this parameter in the command: `--maintenance schedule=weekly,weekday=,hour=`. In this case, maintenance will take place every week on a specified day at a specified time. Both active and stopped clusters undergo maintenance. Maintenance may involve such operations as applying patches or updating DBMS's. * `--disk-encryption-key-id`: Disk encryption using a [custom KMS key](../../kms/concepts/key.md). Learn more about disk encryption in [Storage](../concepts/storage.md#disk-encryption). * `--read-admin-password`: `admin` password. If you specify this parameter in the command, it will prompt you to enter a password. The password must include three groups of characters out of these four: * Lowercase Latin letters * Uppercase Latin letters * Numbers * Special characters The password must be from 10 to 72 characters long. `admin` is a dedicated user required to manage the cluster and cannot be deleted. Such a user has the `superuser` role and can perform any operations with the cluster. {% note tip %} However, for routine jobs, we recommend creating individual regular users. For more information, see [Managing OpenSearch users](cluster-users.md). {% endnote %} * `--serverless-access`: Access from [Yandex Serverless Containers](../../serverless-containers/index.md), `true` or `false`. * `--plugins`: [OpenSearch plugins](../concepts/plugins.md) to install in the cluster. * `--advanced-params`: Additional cluster parameters. The possible values are: * `max-clause-count`: Maximum allowed number of boolean clauses per query. For more information, see [this OpenSearch guide](https://opensearch.org/docs/latest/query-dsl/compound/bool/). * `fielddata-cache-size`: JVM heap size allocated for the `fielddata` data structure. You can specify either an absolute value or percentage, e.g., `512mb` or `50%`. For more information, see [this OpenSearch guide](https://opensearch.org/docs/latest/install-and-configure/configuring-opensearch/index-settings/#cluster-level-index-settings). * `reindex-remote-whitelist`: List of remote hosts whose indexes contain documents to copy for reindexing. Specify the parameter value in `:` format. If you need to specify more than one host, list values separated by commas. For more information, see [this OpenSearch guide](https://opensearch.org/docs/latest/im-plugin/reindex-data/#reindex-from-a-remote-cluster). * `--opensearch-node-group`: `OpenSearch` host group configuration, where: * `resource-preset-id`: Host class that defines the specifications of virtual machines the OpenSearch nodes will be deployed on. All available options are listed under [Host classes](../concepts/instance-types.md). * `disk-size`: Disk size, in bytes. The minimum and maximum values depend on the selected host class. * `disk-type-id`: [Disk type](../concepts/storage.md). * `zone-ids`: [Availability zones](../../overview/concepts/geo-scope.md). Separate zones with commas and enclose them in square brackets, e.g.: ```bash zone-ids=[ru-central1-a,ru-central1-b,ru-central1-d] ``` * `subnet-names`: Names of the subnets in the specified availability zones. Separate subnets with commas and enclose them in square brackets, e.g.: ```bash subnet-names=[default-ru-central1-a,default-ru-central1-b,default-ru-central1-d] ``` You can specify the `subnet-ids` parameter with network IDs instead of `subnet-names`. Separate IDs with commas and enclosed them in square brackets, e.g.: ```bash subnet-ids=[e9bp8qmchqh2********,e2l963gkhobo********,fl8klaabecc3********] ``` * `roles`: [Host roles](../concepts/host-roles.md). The possible values are: * `data`: Assigns the `DATA` role only. * `manager`: Assigns the `MANAGER` role only. * `data+manager` or `manager+data`: Assigns both roles. * `--dashboards-node-group`: `Dashboards` host group configuration. It is configured just as the `OpenSearch` host group, except for the host roles. You do not need to set up any roles for the `Dashboards` group. - Terraform {#tf} With [Terraform](https://www.terraform.io/), you can quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. These files store the infrastructure description written in HashiCorp Configuration Language (HCL). If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed. Terraform is distributed under the [Business Source License](https://github.com/hashicorp/terraform/blob/main/LICENSE). The [Yandex Cloud provider for Terraform](https://github.com/yandex-cloud/terraform-provider-yandex) is distributed under the [MPL-2.0](https://www.mozilla.org/en-US/MPL/2.0/) license. For more information about the provider resources, see the guides on the [Terraform](https://www.terraform.io/docs/providers/yandex/index.html) website or [its mirror](../../terraform/index.md). If you do not have Terraform yet, [install it and configure the Yandex Cloud provider](../../tutorials/infrastructure-management/terraform-quickstart.md#install-terraform). To manage infrastructure using Terraform under a service account or user accounts (a Yandex account, a federated account, or a local user), [authenticate](../../terraform/authentication.md) using the appropriate method. To create a Managed Service for OpenSearch cluster: 1. In the configuration file, describe the resources you want to create: * Database cluster: Description of the Managed Service for OpenSearch cluster and its hosts. * Network: Description of the [cloud network](../../vpc/concepts/network.md#network) where a cluster will be located. If you already have a suitable network, you don't have to describe it again. * Subnets: Description of the [subnets](../../vpc/concepts/network.md#subnet) to connect the cluster hosts to. If you already have suitable subnets, you don't have to describe them again. Here is an example of the configuration file structure: ```hcl resource "yandex_mdb_opensearch_cluster" "" { name = "" environment = "" network_id = "" security_group_ids = [""] disk_encryption_key_id = deletion_protection = "" config { version = "" admin_password = "" opensearch { node_groups { name = "" assign_public_ip = hosts_count = zone_ids = [""] subnet_ids = [""] roles = [""] resources { resource_preset_id = "" disk_size = disk_type_id = "" } } plugins = [""] } dashboards { node_groups { name = "" assign_public_ip = hosts_count = zone_ids = [""] subnet_ids = [""] resources { resource_preset_id = "" disk_size = disk_type_id = "" } } } } maintenance_window { type = "" day = "" hour = } } resource "yandex_vpc_network" "" { name = "" } resource "yandex_vpc_subnet" "" { name = "" zone = "" network_id = "" v4_cidr_blocks = [""] } ``` Where: * `environment`: Environment, `PRESTABLE` or `PRODUCTION`. * `disk_encryption_key_id`: Disk encryption using a [custom KMS key](../../kms/concepts/key.md). Learn more about disk encryption in [Storage](../concepts/storage.md#disk-encryption). * `deletion_protection`: Cluster deletion protection, `true` or `false`. Even with cluster deletion protection enabled, it is still possible to delete a user or connect to the cluster manually and delete the data. * `admin_password`: `admin` password. The password must include three groups of characters out of these four: * Lowercase Latin letters * Uppercase Latin letters * Numbers * Special characters The password must be from 10 to 72 characters long. `admin` is a dedicated user required to manage the cluster and cannot be deleted. Such a user has the `superuser` role and can perform any operations with the cluster. {% note tip %} However, for routine jobs, we recommend creating individual regular users. For more information, see [Managing OpenSearch users](cluster-users.md). {% endnote %} * `assign_public_ip`: Public access to the host, `true` or `false`. * `roles`: Host roles, `DATA` and `MANAGER`. * `maintenance_window`: [Maintenance window](../concepts/maintenance.md) (for stopped clusters as well): * `type`: Maintenance type. The possible values include: * `ANYTIME`: Any time. * `WEEKLY`: On a schedule. * `day`: Day of week in `DDD` format for the `WEEKLY` type, e.g., `MON`. * `hour`: Time of day (UTC) in `HH` format for the `WEEKLY` type, e.g., `21`. For a complete list of configurable Managed Service for OpenSearch cluster fields, see [this Terraform provider guide](../../terraform/resources/mdb_opensearch_cluster.md). 1. Make sure the settings are correct. 1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure. 1. Run this command: ```bash terraform validate ``` Terraform will show any errors found in your configuration files. 1. Create a cluster. 1. Run this command to view the planned changes: ```bash terraform plan ``` If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources. 1. If everything looks correct, apply the changes: 1. Run this command: ```bash terraform apply ``` 1. Confirm updating the resources. 1. Wait for the operation to complete. {% note warning "Timeouts" %} The Terraform provider sets the following timeouts for Managed Service for OpenSearch cluster operations: * Creating a cluster, including by restoring it from a backup: 30 minutes. * Updating a cluster: 60 minutes. * Deleting a cluster: 15 minutes. Operations exceeding the timeout are aborted. {% cut "How do I change these limits?" %} Add a `timeouts` section to the cluster description, e.g.: ```hcl resource "yandex_mdb_opensearch_cluster" "" { ... timeouts { create = "1h30m" # 1 hour 30 minutes update = "2h" # 2 hours delete = "30m" # 30 minutes } } ``` {% endcut %} {% endnote %} - REST API {#api} 1. [Get an IAM token for API authentication](../api-ref/authentication.md) and put it into an environment variable: ```bash export IAM_TOKEN="" ``` 1. Create a file named `body.json` and paste the following code into it: ```json { "folderId": "", "name": "", "environment": "", "networkId": "", "securityGroupIds": [ "", "", ... "" ], "serviceAccountId": "", "deletionProtection": , "configSpec": { "version": "", "adminPassword": "", "opensearchSpec": { "plugins": [ "", "", ... "" ], "nodeGroups": [ { "name": "", "resources": { "resourcePresetId": "", "diskSize": "", "diskTypeId": "" }, "roles": ["",""], "hostsCount": "", "zoneIds": [ "", "", "" ], "subnetIds": [ "", "", "" ], "assignPublicIp": , "diskSizeAutoscaling": { "plannedUsageThreshold": "", "emergencyUsageThreshold": "", "diskSizeLimit": "" } }, ... ] }, "dashboardsSpec": { "nodeGroups": [ { "name": "", "resources": { "resourcePresetId": "", "diskSize": "", "diskTypeId": "" }, "hostsCount": "", "zoneIds": [""], "subnetIds": [""], "assignPublicIp": , "diskSizeAutoscaling": { "plannedUsageThreshold": "", "emergencyUsageThreshold": "", "diskSizeLimit": "" } } ] }, "access": { "dataTransfer": , "serverless": } }, "maintenanceWindow": { "weeklyMaintenanceWindow": { "day": "", "hour": "" } } } ``` Where: * `folderId`: Folder ID. You can get it from the [list of your cloud folders](../../resource-manager/operations/folder/get-id.md). * `name`: Cluster name. * `environment`: Cluster environment, `PRODUCTION` or `PRESTABLE`. * `networkId`: ID of the [network](../../vpc/concepts/network.md#network) where your cluster will be deployed. * `securityGroupIds`: [Security group](../concepts/network.md#security-groups) IDs. * `serviceAccountId`: ID of the [service account](../../iam/concepts/users/service-accounts.md) used for cluster operations. * `deletionProtection`: Cluster protection against accidental deletion, `true` or `false`. Even with cluster deletion protection enabled, it is still possible to delete a user or connect to the cluster manually and delete the data. * `configSpec`: Cluster settings: * `version`: OpenSearch version. * `adminPassword`: `admin` password. The password must include three groups of characters out of these four: * Lowercase Latin letters * Uppercase Latin letters * Numbers * Special characters The password must be from 10 to 72 characters long. `admin` is a dedicated user required to manage the cluster and cannot be deleted. Such a user has the `superuser` role and can perform any operations with the cluster. {% note tip %} However, for routine jobs, we recommend creating individual regular users. For more information, see [Managing OpenSearch users](cluster-users.md). {% endnote %} * `opensearchSpec`: `OpenSearch` host group settings: * `plugins`: List of [OpenSearch plugins](../concepts/plugins.md) to additionally install in the cluster. * `nodeGroups`: Host settings as an array of elements, one per host group. Each element has the following structure: * `name`: Host group name. * `resources`: Cluster resources: * `resourcePresetId`: [Host class](../concepts/instance-types.md). * `diskSize`: Disk size, in bytes. * `diskTypeId`: [Disk type](../concepts/storage.md). * `roles`: List of [host roles](../concepts/host-roles.md). A cluster must include at least one group of `DATA` hosts and one group of `MANAGER` hosts. This can be a single group with two roles or multiple groups with different roles. * `hostsCount`: Number of hosts in the group. Minimum number of `DATA` hosts: one; minimum number of `MANAGER` hosts: three. * `zoneIds`: List of availability zones the cluster hosts are located in. * `subnetIds`: List of subnet IDs. * `assignPublicIp`: Permission to [connect](connect/index.md) to the host from the internet, `true` or `false`. * `diskSizeAutoscaling`: Automatic storage expansion settings: * `plannedUsageThreshold`: Storage usage percentage to trigger a storage expansion during the next maintenance window. Use a value between `0` and `100`%. The default value is `0`, i.e., automatic expansion is disabled. If you set this condition, configure the maintenance window schedule in the `maintenanceWindow` parameter. * `emergencyUsageThreshold`: Storage usage percentage to trigger an immediate storage expansion. Use a value between `0` and `100`%. The default value is `0`, i.e., automatic expansion is disabled. {% note warning %} If you specify both thresholds, `emergencyUsageThreshold` must not be less than `plannedUsageThreshold`. {% endnote %} * `diskSizeLimit`: Maximum storage size, in bytes, to set when storage usage reaches one of the specified thresholds. {% note warning %} * You cannot decrease the storage size after automatic expansion. * When scaling your storage, the cluster hosts will be unavailable. {% endnote %} * `dashboardsSpec`: `Dashboards` host group settings. These contain the `nodeGroups` parameter of the same structure as `opensearchSpec.nodeGroups`, except for the `roles` parameter. The `Dashboards` hosts have only one role, `DASHBOARDS`, so there is no need to specify it. * `access`: Settings for access to the cluster from the following Yandex Cloud services: * `serverless`: [Yandex Serverless Containers](../../serverless-containers/index.md) The possible setting values are `true` or `false`. * `maintenance_window.weeklyMaintenanceWindow`: Maintenance window schedule: * `day`: Day of the week, in `DDD` format, for scheduled maintenance. * `hour`: Hour, in `HH` format, for scheduled maintenance. The possible values range from `1` to `24`. Use the UTC time zone. 1. Call the [Cluster.Create](../api-ref/Cluster/create.md) method, e.g., via the following [cURL](https://curl.se/) request: ```bash curl \ --request POST \ --header "Authorization: Bearer $IAM_TOKEN" \ --header "Content-Type: application/json" \ --url 'https://mdb.api.cloud.yandex.net/managed-opensearch/v1/clusters' \ --data "@body.json" ``` 1. Check the [server response](../api-ref/Cluster/create.md#yandex.cloud.operation.Operation) to make sure your request was successful. - gRPC API {#grpc-api} 1. [Get an IAM token for API authentication](../api-ref/authentication.md) and put it into an environment variable: ```bash export IAM_TOKEN="" ``` 1. Clone the [cloudapi](https://github.com/yandex-cloud/cloudapi) repository: ```bash cd ~/ && git clone --depth=1 https://github.com/yandex-cloud/cloudapi ``` Below, we assume that the repository contents reside in the `~/cloudapi/` directory. 1. Create a file named `body.json` and paste the following code into it: ```json { "folder_id": "", "name": "", "environment": "", "network_id": "", "security_group_ids": [ "", "", ... "" ], "service_account_id": "", "deletion_protection": , "config_spec": { "version": "", "admin_password": "", "opensearch_spec": { "plugins": [ "", "", ... "" ], "node_groups": [ { "name": "", "resources": { "resource_preset_id": "", "disk_size": "", "disk_type_id": "" }, "roles": ["",""], "hosts_count": "", "zone_ids": [ "", "", "" ], "subnet_ids": [ "", "", "" ], "assign_public_ip": , "disk_size_autoscaling": { "planned_usage_threshold": "", "emergency_usage_threshold": "", "disk_size_limit": "" } }, ... ] }, "dashboards_spec": { "node_groups": [ { "name": "", "resources": { "resource_preset_id": "", "disk_size": "", "disk_type_id": "" }, "hosts_count": "", "zone_ids": [""], "subnet_ids": [""], "assign_public_ip": , "disk_size_autoscaling": { "planned_usage_threshold": "", "emergency_usage_threshold": "", "disk_size_limit": "" } } ] }, "access": { "serverless": } }, "maintenance_window": { "weekly_maintenance_window": { "day": "", "hour": "" } } } ``` Where: * `folder_id`: Folder ID. You can get it with the [list of folders in the cloud](../../resource-manager/operations/folder/get-id.md). * `name`: Cluster name. * `environment`: Cluster environment, `PRODUCTION` or `PRESTABLE`. * `network_id`: ID of the [network](../../vpc/concepts/network.md#network) where your cluster will be deployed. * `security_group_ids`: [Security group](../concepts/network.md#security-groups) IDs. * `service_account_id`: ID of the [service account](../../iam/concepts/users/service-accounts.md) used for cluster operations. * `deletion_protection`: Cluster deletion protection, `true` or `false`. Even with cluster deletion protection enabled, it is still possible to delete a user or connect to the cluster manually and delete the data. * `config_spec`: Cluster settings: * `version`: OpenSearch version. * `admin_password`: `admin` password. The password must include three groups of characters out of these four: * Lowercase Latin letters * Uppercase Latin letters * Numbers * Special characters The password must be from 10 to 72 characters long. `admin` is a dedicated user required to manage the cluster and cannot be deleted. Such a user has the `superuser` role and can perform any operations with the cluster. {% note tip %} However, for routine jobs, we recommend creating individual regular users. For more information, see [Managing OpenSearch users](cluster-users.md). {% endnote %} * `opensearch_spec`: `OpenSearch` host group settings: * `plugins`: List of [OpenSearch plugins](../concepts/plugins.md) to additionally install in the cluster. * `node_groups`: Host settings as an array of elements, one per host group. Each element has the following structure: * `name`: Host group name. * `resources`: Cluster resources: * `resource_preset_id`: [Host class](../concepts/instance-types.md). * `disk_size`: Disk size, in bytes. * `disk_type_id`: [Disk type](../concepts/storage.md). * `roles`: List of [host roles](../concepts/host-roles.md). A cluster must include at least one group of `DATA` hosts and one group of `MANAGER` hosts. This can be a single group with two roles or multiple groups with different roles. * `hosts_count`: Number of hosts in the group. Minimum number of `DATA` hosts: one; minimum number of `MANAGER` hosts: three. * `zone_ids`: List of availability zones the cluster hosts are located in. * `subnet_ids`: List of subnet IDs. * `assign_public_ip`: Permission to [connect](connect/index.md) to the host from the internet, `true` or `false`. * `disk_size_autoscaling`: Automatic storage expansion settings: * `planned_usage_threshold`: Storage usage percentage to trigger a storage expansion during the next maintenance window. Use a value between `0` and `100`%. The default value is `0`, i.e., automatic expansion is disabled. If you set this condition, configure the maintenance window schedule in the `maintenance_window` parameter. * `emergency_usage_threshold`: Storage usage percentage to trigger an immediate storage expansion. Use a value between `0` and `100`%. The default value is `0`, i.e., automatic expansion is disabled. {% note warning %} If you specify both thresholds, `emergency_usage_threshold` must not be less than `planned_usage_threshold`. {% endnote %} * `disk_size_limit`: Maximum storage size, in bytes, to set when storage usage reaches one of the specified thresholds. {% note warning %} * You cannot decrease the storage size after automatic expansion. * When scaling your storage, the cluster hosts will be unavailable. {% endnote %} * `dashboards_spec`: `Dashboards` host group settings. These contain the `node_groups` parameter of the same structure as `opensearch_spec.node_groups`, except for the `roles` parameter. The `Dashboards` hosts have only one role, `DASHBOARDS`, so there is no need to specify it. * `access`: Settings for access to the cluster from the following Yandex Cloud services: * `serverless`: [Yandex Serverless Containers](../../serverless-containers/index.md) The possible values are `true` or `false`. * `maintenance_window.weekly_maintenance_window`: Maintenance window schedule: * `day`: Day of the week, in `DDD` format, for scheduled maintenance. * `hour`: Hour, in `HH` format, for scheduled maintenance. The possible values range from `1` to `24`. Use the UTC time zone. 1. Call the [ClusterService.Create](../api-ref/grpc/Cluster/create.md) method, e.g., via the following [gRPCurl](https://github.com/fullstorydev/grpcurl) request: ```bash grpcurl \ -format json \ -import-path ~/cloudapi/ \ -import-path ~/cloudapi/third_party/googleapis/ \ -proto ~/cloudapi/yandex/cloud/mdb/opensearch/v1/cluster_service.proto \ -rpc-header "Authorization: Bearer $IAM_TOKEN" \ -d @ \ mdb.api.cloud.yandex.net:443 \ yandex.cloud.mdb.opensearch.v1.ClusterService.Create \ < body.json ``` 1. View the [server response](../api-ref/grpc/Cluster/create.md#yandex.cloud.operation.Operation) to make sure your request was successful. {% endlist %} ## Creating a cluster copy {#duplicate} You can create an OpenSearch cluster with the settings of another one created earlier. Do this by importing the original OpenSearch cluster configuration into Terraform. This way, you can either create an identical copy or use the imported configuration as the baseline and modify it as needed. The import feature is handy when the original OpenSearch cluster has a lot of settings and you need to create a similar one. To create an OpenSearch cluster copy: {% list tabs group=instructions %} - Terraform {#tf} 1. If you do not have Terraform yet, [install it](../../tutorials/infrastructure-management/terraform-quickstart.md#install-terraform). 1. [Get the authentication credentials](../../tutorials/infrastructure-management/terraform-quickstart.md#get-credentials). You can add them to environment variables or specify them later in the provider configuration file. 1. [Configure and initialize a provider](../../tutorials/infrastructure-management/terraform-quickstart.md#configure-provider). There is no need to create a provider configuration file manually, you can [download it](https://github.com/yandex-cloud-examples/yc-terraform-provider-settings/blob/main/provider.tf). 1. Place the configuration file in a separate working directory and [specify the parameter values](../../tutorials/infrastructure-management/terraform-quickstart.md#configure-provider). If you did not add the authentication credentials to environment variables, specify them in the configuration file. 1. In the same working directory, place a `.tf` file with the following contents: ```hcl resource "yandex_mdb_opensearch_cluster" "old" { } ``` 1. Save the ID of the original OpenSearch cluster to an environment variable: ```bash export OPENSEARCH_CLUSTER_ID= ``` You can get the ID with the [list of clusters in the folder](cluster-list.md#list-clusters). 1. Import the original OpenSearch cluster settings to the Terraform configuration: ```bash terraform import yandex_mdb_opensearch_cluster.old ${OPENSEARCH_CLUSTER_ID} ``` 1. Get the imported configuration: ```bash terraform show ``` 1. Copy it from your terminal and paste it into the `.tf` file. 1. Create a new directory `imported-cluster` and move your configuration file there. 1. Modify the configuration so that you can use it to create a new cluster: * Specify the new cluster name in the `resource` string and the `name` parameter. * Delete the `created_at`, `health`, `id`, and `status` parameters. * Add the `admin_password` parameter to the `config` section. * If you have `type = "ANYTIME"` in the `maintenance_window` section, delete the `hour` parameter. * Optionally, make further changes if you need a customized configuration. 1. [Get the authentication credentials](../../tutorials/infrastructure-management/terraform-quickstart.md#get-credentials) in the `imported-cluster` directory. 1. In the same directory, [configure and initialize the provider](../../tutorials/infrastructure-management/terraform-quickstart.md#configure-provider). Instead of manually creating the provider configuration file, you can [download it](https://github.com/yandex-cloud-examples/yc-terraform-provider-settings/blob/main/provider.tf). 1. Move the configuration file to the `imported-cluster` directory and [specify the arguments](../../tutorials/infrastructure-management/terraform-quickstart.md#configure-provider). If you have not set the authentication credentials as environment variables, specify them in the configuration file. 1. Validate your Terraform configuration: ```bash terraform validate ``` Terraform will display any configuration errors detected in your files. 1. Create the required infrastructure: 1. Run this command to view the planned changes: ```bash terraform plan ``` If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources. 1. If everything looks correct, apply the changes: 1. Run this command: ```bash terraform apply ``` 1. Confirm updating the resources. 1. Wait for the operation to complete. All the required resources will be created in the specified folder. You can check resource availability and their settings in the [management console](https://console.yandex.cloud). {% note warning "Timeouts" %} The Terraform provider sets the following timeouts for Managed Service for OpenSearch cluster operations: * Creating a cluster, including by restoring it from a backup: 30 minutes. * Updating a cluster: 60 minutes. * Deleting a cluster: 15 minutes. Operations exceeding the timeout are aborted. {% cut "How do I change these limits?" %} Add a `timeouts` section to the cluster description, e.g.: ```hcl resource "yandex_mdb_opensearch_cluster" "" { ... timeouts { create = "1h30m" # 1 hour 30 minutes update = "2h" # 2 hours delete = "30m" # 30 minutes } } ``` {% endcut %} {% endnote %} {% endlist %} ## Examples {#examples} {% list tabs group=instructions %} - CLI {#cli} Create a Managed Service for OpenSearch cluster with the following test specifications: * Name: `my-os-clstr`. * Description: `My OS cluster`. * Label: `label-key` with `label-value`. * Environment: `production`. * Network name: `default`. * Security group ID: `enp6saqnq4ie244g67sb`. * Service account name: `os-account`. * Deletion protection: Disabled. * Maintenance time: Every Monday from 13:00 till 14:00. * OpenSearch version: `2.12`. * `admin` password: Specified after entering the cluster create command. * Access to Serverless Containers: Enabled. * OpenSearch added plugin: analysis-icu. * OpenSearch additional parameter: `fielddata-cache-size=50%`. * `OpenSearch` node group configuration: * Node group name: `os-group`. * Host class: `s2.micro`. * Disk size: `10737418240` (in bytes). * Disk type: `network-ssd`. * Number of hosts: `3`. * Availability zone: `ru-central1-a`. * Subnet: `default-ru-central1-a`. * Public address: Assigned. * Host group roles: `DATA` and `MANAGER`. * `Dashboards` host group configuration: * Host group name: `dashboard-group`. * Host class: `s2.micro`. * Disk size: `10737418240` (in bytes). * Disk type: `network-ssd`. * Number of hosts: `1`. * Availability zone: `ru-central1-a`. * Subnet: `default-ru-central1-a`. * Public address: Assigned. Run this command: ```bash yc managed-opensearch cluster create \ --name my-os-clstr \ --description "My OS cluster" \ --labels label-key=label-value \ --environment production \ --network-name default \ --security-group-ids enp6saqnq4ie244g67sb \ --service-account-name os-account \ --delete-protection \ --maintenance schedule=weekly,` `weekday=mon,` `hour=14 \ --version 2.12 \ --read-admin-password \ --serverless-access=true \ --plugins analysis-icu \ --advanced-params fielddata-cache-size=50% \ --opensearch-node-group name=os-group,` `resource-preset-id=s2.micro,` `disk-size=10737418240,` `disk-type-id=network-ssd,` `hosts-count=3,` `zone-ids=ru-central1-a,` `subnet-names=default-ru-central1-a,` `assign-public-ip=true,` `roles=data+manager \ --dashboards-node-group name=dashboard-group,` `resource-preset-id=s2.micro,` `disk-size=10737418240,` `disk-type-id=network-ssd,` `hosts-count=1,` `zone-ids=ru-central1-a,` `subnet-names=default-ru-central1-a,` `assign-public-ip=true ``` - Terraform {#tf} Create a Managed Service for OpenSearch cluster with the following test specifications: * Name: `my-os-clstr`. * Environment: `PRODUCTION`. * OpenSearch version: `2.12`. * `admin` password: `osAdminpwd1`. * `OpenSearch` node group configuration: * `OpenSearch` node group name: `os-group`. * Host class: `s2.micro`. * Disk size: `10737418240` (in bytes). * Disk type: `network-ssd`. * Number of hosts: `3`. * Availability zone: `ru-central1-a`. * Public address: Assigned. * Host group roles: `DATA` and `MANAGER`. * `Dashboards` host group configuration: * Host group name: `dashboard-group`. * Host class: `s2.micro`. * Disk size: `10737418240` (in bytes). * Disk type: `network-ssd`. * Number of hosts: `1`. * Availability zone: `ru-central1-a`. * Public address: Assigned. * Maintenance time: Every Monday from 13:00 till 14:00. * Network name: `mynet`. * Subnet name: `mysubnet`. * Availability zone: `ru-central1-a`. * Address range: `10.1.0.0/16`. * Security group name: `os-sg`. The security group allows connecting to the cluster host from any network (including the internet) on port `9200`. The configuration file for this cluster is as follows: ```hcl resource "yandex_mdb_opensearch_cluster" "my-os-clstr" { name = "my-os-clstr" environment = "PRODUCTION" network_id = yandex_vpc_network.mynet.id security_group_ids = [yandex_vpc_security_group.os-sg.id] config { version = "2.12" admin_password = "osAdminpwd1" opensearch { node_groups { name = "os-group" assign_public_ip = true hosts_count = 3 zone_ids = ["ru-central1-a"] subnet_ids = [yandex_vpc_subnet.mysubnet.id] roles = ["DATA", "MANAGER"] resources { resource_preset_id = "s2.micro" disk_size = 10737418240 disk_type_id = "network-ssd" } } } dashboards { node_groups { name = "dashboard-group" assign_public_ip = true hosts_count = 1 zone_ids = ["ru-central1-a"] subnet_ids = [yandex_vpc_subnet.mysubnet.id] resources { resource_preset_id = "s2.micro" disk_size = 10737418240 disk_type_id = "network-ssd" } } } } maintenance_window { type = "WEEKLY" day = "MON" hour = 14 } } resource "yandex_vpc_network" "mynet" { name = "mynet" } resource "yandex_vpc_subnet" "mysubnet" { name = "mysubnet" zone = "ru-central1-a" network_id = yandex_vpc_network.mynet.id v4_cidr_blocks = ["10.1.0.0/16"] } resource "yandex_vpc_security_group" "os-sg" { name = "os-sg" network_id = yandex_vpc_network.mynet.id ingress { description = "Allow connections to the Managed Service for OpenSearch cluster from the Internet" protocol = "TCP" port = 9200 v4_cidr_blocks = ["0.0.0.0/0"] } egress { description = "The rule allows all outgoing traffic" protocol = "ANY" v4_cidr_blocks = ["0.0.0.0/0"] from_port = 0 to_port = 65535 } } ``` {% endlist %} ## Managing database connection parameters using Connection Manager {#connection-manager} If your cloud or folder has access to the [Connection Manager](../../metadata-hub/concepts/index.md) public preview, a new _connection_ entity will appear in your folder after you create a cluster. You can use it to manage database connection parameters. Passwords and other sensitive data will be stored in a _Yandex Lockbox secret_. To see which secrets store connection information for your cluster, select **Lockbox** in the list of services in your folder. You will find you cluster's ID on the **Secrets** page in the secret dependencies column. Also, you can use Connection Manager to configure access to connections. To configure integration with Connection Manager, contact [support](https://center.yandex.cloud/support).