[Yandex Cloud documentation](../../index.md) > [Yandex Managed Service for Apache Kafka®](../index.md) > [Concepts](index.md) > Storage in Managed Service for Apache Kafka®

# Storage in Managed Service for Apache Kafka®


Managed Service for Apache Kafka® allows you to use network and local storage drives for database clusters. Network drives are based on network blocks, i.e., virtual drives in the Yandex Cloud infrastructure. Local disks are physically located on [broker servers](brokers.md).

When creating a cluster, you can select the following disk types for data storage:

* **Network HDDs** (`network-hdd`): Most cost-effective option for clusters with low requirements for read and write performance.
* **Network SSDs** (`network-ssd`): Compromise solution: slower than local SSDs, network SSDs ensure data integrity in the event of Yandex Cloud hardware failure.
* **Non-replicated SSDs** (`network-ssd-nonreplicated`): Network disks with higher performance achieved by eliminating redundancy.

  You can only expand this type of storage in 93 GB increments.


* **Ultra high-speed network SSDs with three replicas** (`network-ssd-io-m3`): Network disks that deliver performance equivalent to non-replicated SSDs while ensuring redundancy.

  You can only increase the size of these disks in 93 GB increments.


* **Local SSDs** (`local-ssd`): Highest-performing disks.

  You can expand such a storage as follows:

  * For **Intel Cascade Lake**: Only in 100 GB increments.
  * For **Intel Ice Lake** and **AMD Zen 4**: Only in 368 GB increments.

  For clusters with hosts residing in the `ru-central1-d` [availability zone](../../overview/concepts/geo-scope.md), local SSD storage is not available if using Intel Cascade Lake.

[Block](https://en.wikipedia.org/wiki/Block_(data_storage)) size for all disk types is 4 KB.

{% note info %}

Up to 5% of disk space is reserved for system use, so the disks may have less available space than indicated when creating a cluster.


For more information about sizes and performance of different disk types, see [this Yandex Compute Cloud guide](../../compute/concepts/limits.md#compute-limits-disks).


{% endnote %}

## Selecting disk type when creating a cluster {#storage-type-selection}


The number of broker hosts you can create together with an Apache Kafka® cluster depends on the selected disk type:

* You can create a cluster of only three or more broker hosts using the following disk types:

    * Local SSDs (`local-ssd`)
    * Non-replicated SSDs (`network-ssd-nonreplicated`)

    This cluster will be highly available only if all the [conditions](ha-cluster.md) are met.

* You can add any number of broker hosts within the current quota when using the following disk types:

    * Network HDDs (`network-hdd`)
    * Network SSDs (`network-ssd`)
    * Ultra high-speed network SSDs with three replicas (`network-ssd-io-m3`)

Learn more about limits on the number of broker hosts per cluster [here](limits.md).



## Disk encryption {#disk-encryption}

When creating a cluster, you can encrypt the storage disk with a [custom KMS key](../../kms/concepts/key.md).

To create an encrypted disk, you need the `kms.keys.user` [role](../../kms/security/index.md#kms-keys-user) or higher.

If you deactivate the key used to encrypt a disk, access to the data will be suspended until you reactivate the key.

{% note alert %}

If you delete the key used to encrypt a disk or its [version](../../kms/concepts/version.md), you will irrevocably lose access to your data. For more information, see [this Key Management Service guide](../../kms/concepts/version.md#version-distruct).

{% endnote %}


## Minimum storage size {#minimal-storage-size}

In order to work, each [topic](topics.md#topics) requires space in broker host storage. The amount of such space depends on the replication factor and the number of [partitions](topics.md#partitions). If there is not enough available storage space, you will not be able to create a new topic.

{% note tip %}

You can always [increase](../operations/storage-space.md#change-disk-size) the storage size up to the current [quota](https://console.yandex.cloud/cloud?section=quotas).

{% endnote %}

You can calculate the minimum storage size for all topics using the formula below:

`2 × maximum log segment size × number of partitions in cluster × replication factor`.

If topic partitions are evenly distributed, divide the value calculated with this formula by the number of broker hosts.

## Maximum log segment size {#maximum-log-segment-size}

At least two log segments are required for each replica of a topic partition. You can set the maximum size of such a segment:
* [At the topic level](../operations/cluster-topics.md#update-topic) using the [Segment bytes](settings-list.md#settings-topic-segment-bytes) setting.
* Globally at the [cluster level](../operations/cluster-update.md#change-kafka-settings) using the [Log segment bytes](settings-list.md#settings-log-segment-bytes) setting.

Thus, the minimum storage size for all topics is: `2 × maximum log segment size × number of partitions in cluster × replication factor`. If the cluster partitions are evenly distributed, you can divide the resulting value by the number of brokers to determine the required storage size per broker.

By default, the segment size is 1 GB.

## Managing disk space {#manage-storage-space}

As soon as Apache Kafka® logs take up 97% of storage capacity, the host automatically enters read-only mode. The Managed Service for Apache Kafka® cluster blocks requests to write to the topic.


You can monitor storage utilization on cluster hosts [by setting up alerts in Yandex Monitoring](../operations/storage-space.md#set-alert):


### Recovering a cluster from read-only mode {#read-only-solutions}

Use one of these methods:

* [Increase the storage capacity](../operations/storage-space.md#change-disk-size) to exceed the threshold value. The Managed Service for Apache Kafka® cluster will then automatically clear read-only mode.
* [Configure](../operations/storage-space.md#disk-size-autoscale) automatic storage expansion.

### Automatic storage expansion {#auto-rescale}

Automatic increase of storage size prevents situations where the disk runs out of free space and the host switches to read-only mode. The storage size increases upon reaching the specified threshold percentage of the total capacity. There are two thresholds:

* Scheduled expansion threshold: When reached, the storage size increases during the next [maintenance window](maintenance.md#maintenance-window).
* Immediate expansion threshold: When reached, the storage size increases immediately.

You can use either one or both thresholds. If you set both, make sure the immediate increase threshold is higher than the scheduled one.

Upon reaching the specified threshold, the storage expands differently depending on the disk type:

* For network HDDs and SSDs, by the higher of the two values: 20 GB or 20% of the current disk size.
* For non-replicated SSDs, by 93 GB.
* For local SSDs, in a cluster based on:

    * **Intel Cascade Lake**, by 100 GB.
    * **Intel Ice Lake** or **AMD Zen 4**, by 368 GB.


If the threshold is reached again, the storage will be automatically expanded until it reaches the specified maximum. After that, you can set a new maximum storage size manually.

You can configure automatic storage expansion when [creating](../operations/cluster-create.md) or [updating a cluster](../operations/storage-space.md#disk-size-autoscale). If you set the scheduled increase threshold, you also need to configure the maintenance window schedule.

{% note warning %}

You cannot reduce the storage size following an automatic expansion.

{% endnote %}


## Use cases {#examples}

* [Configuring Kafka Connect to work with a Managed Service for Apache Kafka® cluster](../tutorials/kafka-connect.md)
* [Working with the managed schema registry](../tutorials/managed-schema-registry.md)
* [Using Confluent Schema Registry with Managed Service for Apache Kafka®](../tutorials/confluent-schema-registry.md)