# Transferring data to a Yandex Object Storage target endpoint

Yandex Data Transfer enables you to migrate data to the Yandex Object Storage storage and implement various data transfer, processing, and transformation scenarios. To implement a transfer:

1. [Explore possible data transfer scenarios](#scenarios).
1. [Configure one of the supported data sources](#supported-sources).
1. [Configure the target endpoint](#endpoint-settings) in Yandex Data Transfer.
1. [Create](../../transfer.md#create) a transfer and [start](../../transfer.md#activate) it.
1. Perform required operations with the storage and [control the transfer](../../monitoring.md).
1. In case of any issues, [use ready-made solutions](../../../troubleshooting/index.md) to resolve them.

## Scenarios for transferring data to Yandex Object Storage {#scenarios}

1. Data delivery is a process of delivering arbitrary data to target storage. It includes data retrieval from a queue and its deserialization with subsequent transformation to target storage format.
    
    * [Delivering data from YDS to Object Storage](../../../tutorials/yds-to-objstorage.md).   

1. Uploading data to scalable Object Storage storage allows you to save on data storage and simplifies the exchange with contractors.
    
    * [Loading data from MySQL® to Object Storage](../../../tutorials/mmy-objs-migration.md).
    * [Loading data from PostgreSQL to Object Storage](../../../tutorials/mpg-to-objstorage.md).
    * [Loading data from OpenSearch to Object Storage](../../../tutorials/opensearch-to-object-storage.md).
    * [Loading data from Managed Service for YDB to Object Storage](../../../tutorials/ydb-to-object-storage.md).

For a detailed description of possible Yandex Data Transfer scenarios, see [Tutorials](../../../tutorials/index.md).

## Configuring the data source {#supported-sources}

Configure one of the supported data sources:

* [PostgreSQL](../source/postgresql.md)
* [MySQL®](../source/mysql.md)
* [MongoDB](../source/mongodb.md)
* [Apache Kafka®](../source/kafka.md)
* [Airbyte®](../../../transfer-matrix.md#airbyte)
* [YDS](../source/data-streams.md)
* [Oracle](../source/oracle.md)
* [Managed Service for YDB](../source/ydb.md)
* [OpenSearch](../source/opensearch.md).

For a complete list of supported sources and targets in Yandex Data Transfer, see [Available transfers](../../../transfer-matrix.md).

{% note warning %}

Object Storage only supports inserting new data but not updating it. If there is a data update going on in the source, it should not be used to supply data to Object Storage; otherwise, the transfer will fail with an [error](#update-not-supported).

{% endnote %}

## Configuring the Object Storage target endpoint {#endpoint-settings}

When [creating](../index.md#create) or [updating](../index.md#update) an endpoint, you can define:

* [Configuration settings](#bucket-config) for an Yandex Object Storage bucket or custom S3-compatible storage.
* [Additional parameters](#additional-settings).

### Bucket configurations {#bucket-config}

{% list tabs group=instructions %}

- Yandex Object Storage bucket {#obj-storage}

    
    * **Bucket**: Name of the [bucket](../../../../storage/concepts/bucket.md) to upload source data to.
    * **Service account**: [Service account](../../../../iam/concepts/users/service-accounts.md) with the `storage.uploader` role Data Transfer will use to connect to the bucket.


- Custom S3-compatible storage {#s3-storage}

    * **Bucket**: Bucket name.
    * **AWS Access Key ID** and **AWS Secret Access Key**: [ID and contents of the AWS key](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys) used to access a private bucket.
    * (Optional) **Endpoint**: Endpoint for an Amazon S3-compatible service. Leave this field empty to use Amazon.
    * **Region**: Region to send requests.
    * **Use SSL**: Select this option if the remote server uses a secure SSL/TLS connection.
    * **Verify SSL certificate**: Allow self-signed certificates.

{% endlist %}

### Additional settings {#additional-settings}

* **Serialization format**: Format in which the data will be written to the bucket: `JSON`, `CSV`, `PARQUET`, or `Raw data`. For more information, see [Serialization on delivery to Object Storage](../../../concepts/serializer.md#serializer-s3).

* **Convert complex data to strings**: Conversion of complex values to strings for `JSON` output format.

* **Encoding format**: Compression of output data (`Gzip` or `Uncompressed`).

* **Buffer size**: Size of files the data will be split into.

* **Flush interval**: Time after which the file will be written, regardless of its size.

* **Folder name**: Object folder name. It supports the data layout pattern by date. Here is an example: `2006/01/02/<folder_name>`.

* **Time zone**: Time zone according to which time the files are distributed. Only affects the distribution of files to folders in the bucket, but does not affect the data within the files.

* **Row time column name**: Column name to specify logical time for the data. The default value is the system recording time. When recording data to the target, the time is converted to UTC. This behavior cannot be changed.

After configuring the data source and target, [create and start the transfer](../../transfer.md#create).

## Troubleshooting data transfer issues {#troubleshooting}

For more troubleshooting tips, see [Troubleshooting](../../../troubleshooting/index.md).

### Source data update error {#update-not-supported}

Error message:

```text
Push failed: kind: update not supported
```

Object Storage only supports inserting new data but does not support updating it. If data is updated at the source, the transfer will fail with the above error.

**Solution**: Use sources supporting data insertion only or select a target other than Object Storage.