# Managing endpoints

The source [endpoint](../../concepts/index.md#endpoint) describes the settings of the database from which the information will be transferred using Data Transfer, while the target endpoint, the database you are going to transfer the data to. You can [create](#create), [edit](#update), [clone](#clone), or [delete](#delete) such endpoints.

## Getting a list of endpoints {#list}

{% list tabs group=instructions %}

- Management console {#console}

  1. Open the [folder dashboard](https://console.yandex.cloud).
  1. Navigate to **Data Transfer**.
  1. In the left-hand panel, select ![image](../../../_assets/console-icons/aperture.svg) **Endpoints**.

- CLI {#cli}

  If you do not have the Yandex Cloud CLI yet, [install and initialize it](../../../cli/quickstart.md#install).

  The folder used by default is the one specified when [creating](../../../cli/operations/profile/profile-create.md) the CLI profile. To change the default folder, use the `yc config set folder-id <folder_ID>` command. You can also specify a different folder for any command using `--folder-name` or `--folder-id`. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

  To get a list of [transfers](../../concepts/index.md#transfer) in a [folder](../../../resource-manager/concepts/resources-hierarchy.md#folder), run the following command:

  ```bash
  yc datatransfer endpoint list
  ```


- API {#api}

  Use the [list](../../api-ref/Endpoint/list.md) API method.


{% endlist %}

## Getting detailed information about an endpoint {#get}

{% list tabs group=instructions %}

- Management console {#console}

    1. Open the [folder dashboard](https://console.yandex.cloud).
    1. Navigate to **Data Transfer**.
    1. In the left-hand panel, select ![image](../../../_assets/console-icons/aperture.svg) **Endpoints**.
    1. Click the required endpoint name.

- CLI {#cli}

    If you do not have the Yandex Cloud CLI yet, [install and initialize it](../../../cli/quickstart.md#install).

    The folder used by default is the one specified when [creating](../../../cli/operations/profile/profile-create.md) the CLI profile. To change the default folder, use the `yc config set folder-id <folder_ID>` command. You can also specify a different folder for any command using `--folder-name` or `--folder-id`. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

    To get information about an endpoint, run this command:

    ```bash
    yc datatransfer endpoint get <endpoint_ID>
    ```

    You can get the endpoint ID with a [list of endpoints in the folder](#list).


- API {#api}

    Use the [get](../../api-ref/Endpoint/get.md) API method and provide the endpoint ID value in the `endpointId` request parameter.

    To find out the endpoint ID, [get a list of endpoints in the folder](#list).


{% endlist %}

## Creating an endpoint {#create}


{% note warning %}

To create or edit an endpoint of a managed database, you need the service or primitive [`viewer` role](../../../iam/roles-reference.md#viewer) assigned for the folder hosting a cluster of this managed database.

{% endnote %}


{% list tabs group=instructions %}

- Management console {#console}

  1. Open the [folder dashboard](https://console.yandex.cloud).
  1. Navigate to **Data Transfer**.
  1. In the left-hand panel, select ![image](../../../_assets/console-icons/aperture.svg) **Endpoints**.
  1. Click **Create endpoint**.
  1. In the **Direction** field, select **Source** or **Target**.
  1. Name the endpoint. Use lowercase Latin letters and numbers.
  1. Optionally, enter a description of the endpoint.
  1. Select your preferred option in the **Database type** field. This will create an endpoint of the relevant type.
  1. Set the endpoint parameters:

     * Sources:
      
         * Airbyte®:
             * [AWS CloudTrail](source/aws-cloudtrail.md)
             * [BigQuery](source/bigquery.md)
             * [MS SQL Server](source/mssql.md)
             * [S3](source/s3.md)
         * [Apache Kafka®](source/kafka.md)
         * [ClickHouse®](source/clickhouse.md)
         * [Greenplum®](source/greenplum.md)
         * [Yandex Metrica](source/metrika.md)
         * [MongoDB](source/mongodb.md)
         * [MySQL®](source/mysql.md)
         * [Object Storage](source/object-storage.md)
         * [Oracle](source/oracle.md)
         * [PostgreSQL](source/postgresql.md)
         * [Yandex Data Streams](source/data-streams.md)
         * [Yandex Managed Service for YDB](source/ydb.md)
         * [YTsaurus](target/yt.md)
     * Targets:
         * [Apache Iceberg™](target/iceberg.md)
         * [Apache Kafka®](target/kafka.md)
         * [ClickHouse®](target/clickhouse.md)
         * [Greenplum®](target/greenplum.md)
         * [MongoDB](target/mongodb.md)
         * [MySQL®](target/mysql.md)
         * [Object Storage](target/object-storage.md)
         * [OpenSearch](target/opensearch.md)
         * [PostgreSQL](target/postgresql.md)
         * [Yandex Data Streams](target/data-streams.md)
         * [Yandex Managed Service for YDB](target/yandex-database.md)
         * YTsaurus:
             * [YTSaurus Dynamic](target/yt.md)
             * [YTSaurus Static](target/yt.md)

  1. Click **Create endpoint**.

- CLI {#cli}

  {% note info %}

  You can create endpoints using the [CLI](../../../cli/index.md) commands only for sources and targets of the ClickHouse®, MongoDB, MySQL®, and PostgreSQL types.

  {% endnote %}

  If you do not have the Yandex Cloud CLI yet, [install and initialize it](../../../cli/quickstart.md#install).

  The folder used by default is the one specified when [creating](../../../cli/operations/profile/profile-create.md) the CLI profile. To change the default folder, use the `yc config set folder-id <folder_ID>` command. You can also specify a different folder for any command using `--folder-name` or `--folder-id`. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

  1. View a description of the CLI create endpoint command:

     ```bash
     yc datatransfer endpoint create --help
     ```

  1. Specify the endpoint parameters in the create command (not all parameters are given in the example):

     ```bash
     yc datatransfer endpoint create <endpoint_type> \
       --name <endpoint_name> \
       --description <endpoint_description> \
       <endpoint_parameters>
     ```

     You can view the endpoint type and parameters in the settings section for relevant data source or target.

- Terraform {#tf}

  
  {% note info %}

  You can create endpoints with Terraform only for ClickHouse®, MongoDB, MySQL®, and PostgreSQL sources and targets.

  {% endnote %}


  
  With [Terraform](https://www.terraform.io/), you can quickly create a cloud infrastructure in Yandex Cloud and manage it using configuration files. These files store the infrastructure description written in HashiCorp Configuration Language (HCL). If you change the configuration files, Terraform automatically detects which part of your configuration is already deployed, and what should be added or removed.
  
  Terraform is distributed under the [Business Source License](https://github.com/hashicorp/terraform/blob/main/LICENSE). The [Yandex Cloud provider for Terraform](https://github.com/yandex-cloud/terraform-provider-yandex) is distributed under the [MPL-2.0](https://www.mozilla.org/en-US/MPL/2.0/) license.
  
  For more information about the provider resources, see the relevant documentation on the [Terraform](https://www.terraform.io/docs/providers/yandex/index.html) website or [its mirror](../../../terraform/index.md).

  If you do not have Terraform yet, [install it and configure the Yandex Cloud provider](../../../tutorials/infrastructure-management/terraform-quickstart.md#install-terraform).
  
  
  To manage infrastructure using Terraform under a service account or user accounts (a Yandex account, a federated account, or a local user), [authenticate](../../../terraform/authentication.md) using the appropriate method.


  1. Create a configuration file with a description of your endpoint.

     Here is an example of the configuration file structure:

     ```hcl
     resource "ycp_datatransfer_endpoint" "<endpoint_name_in_Terraform>" {
       name = "<endpoint_name>"
       settings {
         <endpoint_type> {
           <endpoint_parameters>
         }
       }
     }
     ```

  You can retrieve a complete resource schema by running `terraform providers schema`.
  1. You can view the endpoint type and parameters in the settings section for relevant data source or target.
  1. Make sure the settings are correct.

     1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
     1. Run this command:
     
        ```bash
        terraform validate
        ```
     
        Terraform will show any errors found in your configuration files.

  1. Confirm resource changes.

     1. Run this command to view the planned changes:
     
        ```bash
        terraform plan
        ```
     
        If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
     
     1. If everything looks correct, apply the changes:
        1. Run this command:
     
           ```bash
           terraform apply
           ```
     
        1. Confirm updating the resources.
        1. Wait for the operation to complete.

  
  1. For more information, see [this Terraform provider guide](../../../terraform/resources/datatransfer_endpoint.md).


- API {#api}

  {% note info %}

  You can create endpoints with the API only for ClickHouse®, Apache Kafka®, MongoDB, MySQL®, and PostgreSQL sources and targets.

  {% endnote %}

  Call the [create](../../api-ref/Endpoint/create.md) API method and provide the following in the request:
  * ID of the [folder](../../../resource-manager/concepts/resources-hierarchy.md#folder) to host the endpoint, in the `folderId` parameter.
  * Endpoint name in the `name` parameter.
  * Endpoint description in the `description` parameter.
  * Endpoint parameters in the `settings` parameter.

   You can view the endpoint parameters in the settings section for relevant data source or target.

{% endlist %}

## Updating an endpoint {#update}


{% note warning %}

To create or edit an endpoint of a managed database, you need the service or primitive [`viewer` role](../../../iam/roles-reference.md#viewer) assigned for the folder hosting a cluster of this managed database.

{% endnote %}


{% note info %}

You cannot change the endpoint type and the database type. For some endpoints, you cannot change the connection type, either.

{% endnote %}

{% list tabs group=instructions %}

- Management console {#console}

  1. Open the [folder dashboard](https://console.yandex.cloud).
  1. Navigate to **Data Transfer**.
  1. In the left-hand panel, select ![image](../../../_assets/console-icons/aperture.svg) **Endpoints**.
  1. Select an endpoint and click ![pencil](../../../_assets/console-icons/pencil.svg) **Edit** in the top panel.
  1. Edit the endpoint parameters:

     * Sources:
      
         * Airbyte®:
             * [AWS CloudTrail](source/aws-cloudtrail.md)
             * [BigQuery](source/bigquery.md)
             * [MS SQL Server](source/mssql.md)
             * [S3](source/s3.md)
         * [Apache Kafka®](source/kafka.md)
         * [ClickHouse®](source/clickhouse.md)
         * [Greenplum®](source/greenplum.md)
         * [Yandex Metrica](source/metrika.md)
         * [MongoDB](source/mongodb.md)
         * [MySQL®](source/mysql.md)
         * [Object Storage](source/object-storage.md)
         * [Oracle](source/oracle.md)
         * [PostgreSQL](source/postgresql.md)
         * [Yandex Data Streams](source/data-streams.md)
         * [Yandex Managed Service for YDB](source/ydb.md)
         * [YTsaurus](target/yt.md)
     * Targets:
         * [Apache Iceberg™](target/iceberg.md)
         * [Apache Kafka®](target/kafka.md)
         * [ClickHouse®](target/clickhouse.md)
         * [Greenplum®](target/greenplum.md)
         * [MongoDB](target/mongodb.md)
         * [MySQL®](target/mysql.md)
         * [Object Storage](target/object-storage.md)
         * [OpenSearch](target/opensearch.md)
         * [PostgreSQL](target/postgresql.md)
         * [Yandex Data Streams](target/data-streams.md)
         * [Yandex Managed Service for YDB](target/yandex-database.md)
         * YTsaurus:
             * [YTSaurus Dynamic](target/yt.md)
             * [YTSaurus Static](target/yt.md)

  1. Click **Apply**.

- CLI {#cli}

  {% note info %}

  You can change endpoints using the CLI commands only for ClickHouse®, MongoDB, MySQL®, and PostgreSQL sources and targets.

  {% endnote %}

  If you do not have the Yandex Cloud CLI yet, [install and initialize it](../../../cli/quickstart.md#install).

  The folder used by default is the one specified when [creating](../../../cli/operations/profile/profile-create.md) the CLI profile. To change the default folder, use the `yc config set folder-id <folder_ID>` command. You can also specify a different folder for any command using `--folder-name` or `--folder-id`. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

  To update the endpoint settings:
  1. View a description of the update transfer CLI command:

     ```bash
     yc datatransfer endpoint update --help
     ```

  1. Run the following command with the list of settings you want to update (not all settings are listed in the example below):

     ```bash
     yc datatransfer transfer update <endpoint_type> <endpoint_ID> \
       --name <endpoint_name> \
       --description <endpoint_description> \
       <endpoint_parameters>
     ```

     You can view the endpoint type and parameters in the settings section for relevant data source or target. You can get the endpoint ID with a [list of endpoints in the folder](#list).

- Terraform {#tf}

  1. Open the current Terraform configuration file with the endpoint description.

     For more info on how to create such a file, see the [Creating an endpoint](#create) subsection.
  1. Edit the value in the `name` field (endpoint name) and the endpoint parameters under `settings`.
  1. Make sure the settings are correct.

     1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
     1. Run this command:
     
        ```bash
        terraform validate
        ```
     
        Terraform will show any errors found in your configuration files.

  1. Confirm resource changes.

     1. Run this command to view the planned changes:
     
        ```bash
        terraform plan
        ```
     
        If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
     
     1. If everything looks correct, apply the changes:
        1. Run this command:
     
           ```bash
           terraform apply
           ```
     
        1. Confirm updating the resources.
        1. Wait for the operation to complete.

    For more information, see [this Terraform provider guide](../../../terraform/resources/datatransfer_endpoint.md).

- API {#api}

  {% note info %}

  You can edit endpoints using the API only for ClickHouse®, Apache Kafka®, MongoDB, MySQL®, and PostgreSQL sources and targets.

  {% endnote %}

  Use the [update](../../api-ref/Endpoint/update.md) API method, providing the following in your request:
  * Endpoint ID in the `endpointId` parameter.
  * Endpoint name in the `name` parameter.
  * Endpoint description in the `description` parameter.
  * Endpoint parameters in the `settings` parameter.

  You can get the endpoint ID with a [list of endpoints in the folder](#list).

{% endlist %}

Editing the endpoint settings of a transfer with the **Replication** type and the **Replicating** status will result in restarting the transfer.

## Cloning an endpoint {#clone}

{% note info %}

When cloning, you cannot change the endpoint type and the database type.

{% endnote %}

{% list tabs group=instructions %}

- Management console {#console}

  1. Open the [folder dashboard](https://console.yandex.cloud).
  1. Navigate to **Data Transfer**.
  1. In the left-hand panel, select ![image](../../../_assets/console-icons/aperture.svg) **Endpoints**.
  1. Select the endpoint to clone.
  1. Click **Clone** in the top panel.
  1. Specify a new endpoint name and edit other parameters, if required:

     * Sources:
      
         * Airbyte®:
             * [AWS CloudTrail](source/aws-cloudtrail.md)
             * [BigQuery](source/bigquery.md)
             * [MS SQL Server](source/mssql.md)
             * [S3](source/s3.md)
         * [Apache Kafka®](source/kafka.md)
         * [ClickHouse®](source/clickhouse.md)
         * [Greenplum®](source/greenplum.md)
         * [Yandex Metrica](source/metrika.md)
         * [MongoDB](source/mongodb.md)
         * [MySQL®](source/mysql.md)
         * [Object Storage](source/object-storage.md)
         * [Oracle](source/oracle.md)
         * [PostgreSQL](source/postgresql.md)
         * [Yandex Data Streams](source/data-streams.md)
         * [Yandex Managed Service for YDB](source/ydb.md)
         * [YTsaurus](target/yt.md)
     * Targets:
         * [Apache Iceberg™](target/iceberg.md)
         * [Apache Kafka®](target/kafka.md)
         * [ClickHouse®](target/clickhouse.md)
         * [Greenplum®](target/greenplum.md)
         * [MongoDB](target/mongodb.md)
         * [MySQL®](target/mysql.md)
         * [Object Storage](target/object-storage.md)
         * [OpenSearch](target/opensearch.md)
         * [PostgreSQL](target/postgresql.md)
         * [Yandex Data Streams](target/data-streams.md)
         * [Yandex Managed Service for YDB](target/yandex-database.md)
         * YTsaurus:
             * [YTSaurus Dynamic](target/yt.md)
             * [YTSaurus Static](target/yt.md)

  1. Click **Create endpoint**.

{% endlist %}

## Deleting an endpoint {#delete}

{% note warning %}

Before deleting an endpoint, [delete](../transfer.md#delete) all the transfers it is included in.

{% endnote %}

To delete an endpoint:

{% list tabs group=instructions %}

- Management console {#console}

  1. Open the [folder dashboard](https://console.yandex.cloud).
  1. Navigate to **Data Transfer**.
  1. In the left-hand panel, select ![image](../../../_assets/console-icons/aperture.svg) **Endpoints**.
  1. Select the endpoint to delete.
  1. Click **Delete** in the top panel.
  1. In the window that opens, click **Delete**.

- CLI {#cli}

  If you do not have the Yandex Cloud CLI yet, [install and initialize it](../../../cli/quickstart.md#install).

  The folder used by default is the one specified when [creating](../../../cli/operations/profile/profile-create.md) the CLI profile. To change the default folder, use the `yc config set folder-id <folder_ID>` command. You can also specify a different folder for any command using `--folder-name` or `--folder-id`. If you access a resource by its name, the search will be limited to the default folder. If you access a resource by its ID, the search will be global, i.e., through all folders based on access permissions.

  To delete an endpoint, run the command:

  ```bash
  yc datatransfer endpoint delete <endpoint_ID>
  ```

  You can get the endpoint ID with a [list of endpoints in the folder](#list).

- Terraform {#tf}

  To delete an endpoint created using Terraform:
  1. Open the current Terraform configuration file with the infrastructure plan.
  
     For more on how to create such a file, see [Creating an endpoint](index.md#create).
  1. Make sure that the endpoint being deleted is not bound to a transfer.
  1. Delete the endpoint description.
  1. Make sure the settings are correct.
  
     1. In the command line, navigate to the directory that contains the current Terraform configuration files defining the infrastructure.
     1. Run this command:
     
        ```bash
        terraform validate
        ```
     
        Terraform will show any errors found in your configuration files.
  
  1. Type `yes` and press **Enter**.
  
     1. Run this command to view the planned changes:
     
        ```bash
        terraform plan
        ```
     
        If you described the configuration correctly, the terminal will display a list of the resources to update and their parameters. This is a verification step that does not apply changes to your resources.
     
     1. If everything looks correct, apply the changes:
        1. Run this command:
     
           ```bash
           terraform apply
           ```
     
        1. Confirm updating the resources.
        1. Wait for the operation to complete.
  
  For more information, see [this Terraform provider guide](../../../terraform/resources/datatransfer_endpoint.md).

- API {#api}

  Use the [delete](../../api-ref/Endpoint/delete.md) API method and provide the ID of the required endpoint in the `endpointId` request parameter.

  You can get the endpoint ID with a [list of endpoints in the folder](#list).

{% endlist %}

_Greenplum® and Greenplum Database® are registered trademarks or trademarks of Broadcom Inc. in the United States and/or other countries._

_ClickHouse® is a registered trademark of [ClickHouse, Inc](https://clickhouse.com)._