# Transferring data to a Apache Iceberg™ target endpoint


Yandex Data Transfer enables you to migrate data to Apache Iceberg™ tables in a Apache Hive™ Metastore cluster and implement various data transfer, processing, and transformation scenarios. To implement a transfer:

1. [Explore possible data transfer scenarios](#scenarios).
1. [Configure one of the supported data sources](#supported-sources).
1. [Configure the target endpoint](#endpoint-settings) in Yandex Data Transfer.
1. [Create](../../transfer.md#create) a transfer and [start](../../transfer.md#activate) it.
1. Perform the required operations with the tables and [see how the transfer is going](../../monitoring.md).

## Scenarios for transferring data to Apache Iceberg™ {#scenarios}

For a detailed description of possible Yandex Data Transfer scenarios, see [Tutorials](../../../tutorials/index.md).

## Configuring the data source {#supported-sources}

Configure one of the supported data sources:

* [ClickHouse®](../source/clickhouse.md)
* [Greenplum®](../source/greenplum.md)
* [MongoDB](../source/mongodb.md)
* [MySQL®](../source/mysql.md)
* [PostgreSQL](../source/postgresql.md)
* [Elasticsearch](../source/elasticsearch.md)
* [Yandex Object Storage](../source/object-storage.md)
* [Oracle](../source/oracle.md)
* [Managed Service for YDB](../source/ydb.md)
* [YTsaurus](../source/yt.md)

For a complete list of supported sources and targets in Yandex Data Transfer, see [Available transfers](../../../transfer-matrix.md).

## Configuring the Apache Iceberg™ target endpoint {#endpoint-settings}

When [creating](../index.md#create) or [updating](../index.md#update) an endpoint, you can define:

* [Settings for connecting to a Apache Hive™ Metastore cluster](#managed-service).
* [Configuration settings](#bucket-config) for an Yandex Object Storage bucket or custom S3-compatible storage.
* [Additional parameters](#additional-settings).

### Apache Hive™ Metastore cluster {#managed-service}


{% note warning %}

To create or edit an endpoint of a managed database, you will need the [`managed-metastore.viewer`](../../../../metadata-hub/security/metastore-roles.md#managed-metastore-viewer) role or the primitive [`viewer`](../../../../iam/roles-reference.md#viewer) role for the folder the cluster of this managed database resides in.

{% endnote %}


Connection with the cluster specified in Yandex Cloud.

{% list tabs group=instructions %}

- Management console {#console}

    * **Apache Hive™ Metastore cluster**: ID of the cluster whose folder is used for Apache Iceberg™ tables.

    
    * **Security groups**: Select the cloud network to host the endpoint and security groups for network traffic. This will allow you to apply the specified security group rules to the VMs and clusters in the selected network without changing their settings. For more information, see [Networking in Yandex Data Transfer](../../../concepts/network.md).

      Make sure the selected security groups are [configured](../../../../metadata-hub/operations/metastore/configure-security-group.md).


{% endlist %}

### Bucket configurations {#bucket-config}

{% list tabs group=instructions %}

- Yandex Object Storage bucket {#obj-storage}

    
    * **Bucket**: Name of the [bucket](../../../../storage/concepts/bucket.md) to upload source data to.
    * **Service account**: Select or create a [service account](../../../../iam/concepts/users/service-accounts.md) with the `storage.uploader` role that Data Transfer will use to connect to the bucket.


- Custom S3-compatible storage {#s3-storage}

    * (Optional) **Endpoint**: Endpoint for an Amazon S3-compatible service. Leave this field empty to use Amazon.
    * **Region**: Region to send requests.
    * **Bucket**: Bucket name.
    * **Access Key ID** and **Secret Access Key**: [ID and contents of the AWS key](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys) used to access a private bucket.

{% endlist %}

* **Path prefix**: Path prefix for writing objects to the bucket. This is optional.

### Additional settings {#additional-settings}

{% list tabs group=instructions %}

- Management console {#console}

   * **Cleanup policy**: Select a way to clean up data in the target database before the transfer:

     * `DISABLED`: Use the existing tables to write new data.

     * `DROP`: Remove all tables involved in the transfer.

       Use this option to always transfer the latest version of the table schema to the target database from the source whenever the transfer is activated.

{% endlist %}

After configuring the data source and target, [create and start the transfer](../../transfer.md#create).