[Yandex Cloud documentation](../../index.md) > [Yandex Managed Service for Apache Kafka®](../index.md) > [Concepts](index.md) > Apache Kafka® settings

# Apache Kafka® settings

For Managed Service for Apache Kafka® clusters, you can configure Apache Kafka® settings. Some settings are configured at the [cluster level](#cluster-settings), while others, at the [topic level](#topic-settings).

The label next to the setting name shows which interface can be used to configure this setting: the management console, CLI, API, or Terraform. The <code><b><small>All interfaces</small></b></code> label means that all of the above interfaces are supported.

Depending on the selected interface, the same setting will be represented differently. For example, **Compression type** in the management console is the same as:

* `--compression-type` in the CLI
* `compression_type` in the gRPC API and Terraform
* `compressionType` in the REST API

## Cluster-level settings {#cluster-settings}

The following settings are available:

* **Auto create topics enable** <code><b><small>All interfaces</small></b></code>{#settings-auto-create-topics}

    Manages automatic topic creation.

    This option is disabled by default (`false`).

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_auto.create.topics.enable).

* **Compression type** <code><b><small>All interfaces</small></b></code> {#settings-compression-type}    

    Codecs used for message compression:

    | Management console, Terraform, and API | CLI            | Description                                                                                         |
    | ---------------------------------- | -------------- | ------------------------------------------------------------------------------------------------ |
    | `COMPRESSION_TYPE_UNCOMPRESSED`    | `uncompressed` | Compression is disabled.                                                                                 |
    | `COMPRESSION_TYPE_ZSTD`            | `zstd`         | [zstd](https://facebook.github.io/zstd/)                                                   |
    | `COMPRESSION_TYPE_LZ4`             | `lz4`          | [lz4](https://lz4.github.io/lz4/)                                                          |
    | `COMPRESSION_TYPE_SNAPPY`          | `snappy`       | [snappy](https://github.com/google/snappy)                                                 |
    | `COMPRESSION_TYPE_GZIP`            | `gzip`         | [gzip](https://www.gzip.org)                                                               |
    | `COMPRESSION_TYPE_PRODUCER`        | `producer`     | The codec is set by the [producer](producers-consumers.md).|

    By default, the compression codec is set by the producer (`COMPRESSION_TYPE_PRODUCER`).

    This is a global cluster-level setting. You can override it at the [topic level](#settings-topic-compression-type). In the management console, this setting appears as **Compression type**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_compression.type).

* **Default replication factor** <code><b><small>All interfaces</small></b></code> {#settings-default-replication-factor}

    Number of [data replicas](brokers.md) per topic within a cluster.

    This setting only applies to [automatically created topics](#settings-auto-create-topics).

    The minimum and default value is `1`. The maximum value is equal to the number of broker hosts in the cluster.

    See also the description of the [Replication factor](#settings-topic-replication-factor) topic-level setting.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_default.replication.factor).

* **Log flush interval messages** <code><b><small>All interfaces</small></b></code> {#settings-log-flush-interval-messages}

    Number of topic messages that can accumulate in memory before being flushed to disk. For example, if set to `1`, each message will be flushed to disk once received. If set to `5`, messages will be flushed to disk in groups of five.

    The minimum value is `1`, the maximum and default one is `9223372036854775807`.

    This is a global cluster-level setting. You can override it at the [topic level](#settings-topic-flush-messages). In the management console, this setting appears as **Flush messages**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_log.flush.interval.messages).

* **Log flush interval ms** <code><b><small>All interfaces</small></b></code> {#settings-log-flush-interval-ms}

    Maximum time in milliseconds a message can be stored in memory before being flushed to disk. If no value is specified, the [Log flush scheduler interval ms](#settings-log-flush-scheduler-interval-ms) value is used.

    The maximum value is `9223372036854775807`.

    This is a global cluster-level setting. You can override it at the [topic level](#settings-topic-flush-ms).

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_log.flush.interval.ms).

* **Log flush scheduler interval ms** <code><b><small>All interfaces</small></b></code> {#settings-log-flush-scheduler-interval-ms}

    Time period (in milliseconds) after which the log flusher checks for logs to flush to disk.

    The maximum and default value is `9223372036854775807`.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_log.flush.scheduler.interval.ms).

* **Log preallocate** <code><b><small>All interfaces</small></b></code> {#settings-log-preallocate}

    This setting determines whether to pre-allocate space for log segment files.

    By default, the space for log segment files is allocated as the segments get full (`false`).

    This is a global cluster-level setting. You can override it at the [topic level](#settings-topic-preallocate).

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_log.preallocate).

* **Log retention bytes** <code><b><small>All interfaces</small></b></code> {#settings-log-retention-bytes}    

    Maximum size a partition can grow to. When a partition reaches this size, Apache Kafka® starts deleting old log segments. This setting applies if the `Delete` [log cleanup policy](#settings-topic-cleanup-policy) is effective.

    The minimum and default value is `-1`, which means the log size is unlimited. The maximum value is `9223372036854775807`.

    Use this setting if you need to manage the log size due to limited disk space.

    This is a global cluster-level setting. You can override it at the [topic level](#settings-topic-retention-bytes). In the management console, this setting appears as **Retention, bytes**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_log.retention.bytes).

    See also the [Log retention ms](#settings-log-retention-ms) setting.

* **Log retention hours** <code><b><small>All interfaces</small></b></code> {#settings-log-retention-hours}

    Time (in hours) for Apache Kafka® to keep a log segment file. This setting applies if the `Delete` [log cleanup policy](#settings-topic-cleanup-policy) is effective; once the specified timeout expires, the segment file is deleted.

    The default value is `168`.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_log.retention.hours).

* **Log retention minutes** <code><b><small>All interfaces</small></b></code> {#settings-log-retention-minutes}

    Time (in minutes) for Apache Kafka® to keep a log segment file. This setting applies if the `Delete` [log cleanup policy](#settings-topic-cleanup-policy) is effective; once the specified timeout expires, the segment file is deleted.

    The maximum value is `2147483647`. If no value is specified, the [Log retention hours](#settings-log-retention-hours) value is used.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_log.retention.minutes).

* **Log retention ms** <code><b><small>All interfaces</small></b></code> {#settings-log-retention-ms}    

    Time (in milliseconds) for Apache Kafka® to keep a log segment file. This setting applies if the `Delete` [log cleanup policy](#settings-topic-cleanup-policy) is effective; once the specified timeout expires, the segment file is deleted.

    The minimum value is `-1`, which means logs are stored without any time limit. The maximum value is `9223372036854775807`. If no value is specified, the [Log retention minutes](#settings-log-retention-minutes) value is used.

    {% note warning %}

    If both **Log retention bytes** and **Log retention ms** are set to `-1`, the log grows indefinitely. This can quickly exhaust the cluster storage space.

    {% endnote %}

    This is a global cluster-level setting. You can override it at the [topic level](#settings-topic-log-retention-ms). In the management console, this setting appears as **Retention, ms**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_log.retention.ms).

    See also the [Log retention bytes](#settings-log-retention-bytes) setting.

* **Log segment bytes** <code><b><small>All interfaces</small></b></code> {#settings-log-segment-bytes}

    This setting determines the segment size for log files, in bytes.

    Segment files are saved and cleared file-by-file. A smaller segment size results in more log files but offers more flexible storage management. 

    The minimum value is `14`; the maximum value is `2147483647`. The default value is `1073741824` (1 GB).

    This is a global cluster-level setting. You can override it at the [topic level](#settings-topic-segment-bytes).

    For a full description of this setting, see [Maximum log segment size](storage.md#maximum-log-segment-size) and [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_log.segment.bytes).

* **Message max bytes** <code><b><small>All interfaces</small></b></code> {#settings-message-max-bytes}

    Maximum size of a message sent by the producer.

    The minimum value is `0` (no limit); the default one is `1048588`. The setting value relates to the [Replica fetch max bytes](#settings-replica-fetch-max-bytes) value as follows:
    `replica.fetch.max.bytes >= message.max.bytes + 12`

    This is a global cluster-level setting.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_message.max.bytes).

* **Num partitions** <code><b><small>All interfaces</small></b></code> {#settings-num-partitions}

    Number of log partitions per topic in the cluster.

    This setting only applies to [automatically created topics](#settings-auto-create-topics).

    The minimum and default value is `1`.

    See also the description of the [Num partitions](#settings-topic-num-partitions) topic-level setting.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_num.partitions).

* **Offsets retention minutes** <code><b><small>All interfaces</small></b></code> {#settings-offsets-retention-minutes}

    Offset retention period for inactive [consumer groups](producers-consumers.md#consumer-groups) (in minutes).

    The minimum value is `1` and the default is `10080`.

    This is a global cluster-level setting.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_offsets.retention.minutes).

* **Replica fetch max bytes** <code><b><small>All interfaces</small></b></code> {#settings-replica-fetch-max-bytes}

    Inter-broker message buffer size (in bytes).

    The minimum value is `0` (no limit); the default one is `1048576`. The setting value relates to the **Message max bytes** value as follows:
    `message.max.bytes <= replica.fetch.max.bytes - 12` 

    This is a global cluster-level setting.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_replica.fetch.max.bytes).

* **Sasl enabled mechanisms** <code><b><small>All interfaces</small></b></code> {#settings-sasl-enabled-mechanisms}

    Encryption options for cluster connections.

    You can select one or more values:

    | Management console, Terraform, and API | CLI             |
    | ---------------------------------- | --------------- |
    | `SASL_MECHANISM_SCRAM_SHA_256`     | `SCRAM-SHA-256` |
    | `SASL_MECHANISM_SCRAM_SHA_512`     | `SCRAM-SHA-512` |

    If the setting is not defined by the user, `SCRAM-SHA-512` is only used.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_sasl.enabled.mechanisms).

* **Socket receive buffer bytes** <code><b><small>Management console</small></b></code> <code><b><small>Terraform</small></b></code> <code><b><small>API</small></b></code>{#settings-socket-receive-buffer-bytes}

    Socket receive buffer size (in bytes).

    The minimum and default value is `-1`, which means the OS settings apply. The maximum value is `2147483647`.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_socket.receive.buffer.bytes).

* **Socket send buffer bytes** <code><b><small>Management console</small></b></code> <code><b><small>Terraform</small></b></code> <code><b><small>API</small></b></code>{#settings-socket-send-buffer-bytes}

    Socket send buffer size (in bytes).

    The minimum and default value is `-1`, which means the OS settings apply. The maximum value is `2147483647`.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_socket.send.buffer.bytes).

* **Ssl cipher suites** <code><b><small>All interfaces</small></b></code> {#settings-ssl-cipher-suites}

    List of cluster encryption algorithms.

    There is no default value, which means all supported encryption algorithms are enabled.

    This is a global cluster-level setting.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_ssl.cipher.suites).

## Settings for individual topics {#topic-settings}

The following settings are available:

* **Cleanup policy** <code><b><small>All interfaces</small></b></code> {#settings-topic-cleanup-policy}

    Retention policy for old log messages:

    * `Delete` (`CLEANUP_POLICY_DELETE` for Terraform and API): Delete log segments when either their retention period or log size limit is reached.
    * `Compact` (`CLEANUP_POLICY_COMPACT` for Terraform and API): Compact the log.
    * `CompactAndDelete` (`CLEANUP_POLICY_COMPACT_AND_DELETE` for Terraform and API): Both compact the log and delete log segments.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/topic-configs/#topicconfigs_cleanup.policy). In the management console, this setting appears as **Cleanup policy**.

* **Compression type** <code><b><small>All interfaces</small></b></code> {#settings-topic-compression-type}

    Codec used for message compression.

    For more information, see the description of the [Compression type](#settings-compression-type) cluster-level setting.

    In the management console, this setting appears as **Compression type**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/topic-configs/#topicconfigs_compression.type).

* **Delete delay, ms** <code><b><small>All interfaces</small></b></code> {#settings-topic-file-delete-delay}

    Time to wait before deleting a file from the file system.

    In the management console, this setting appears as **Delete delay, ms**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/topic-configs/#topicconfigs_file.delete.delay.ms).

* **Delete retention** <code><b><small>All interfaces</small></b></code> {#settings-delete-retention}

    Time (in milliseconds) to retain delete tombstone markers for topics with a compacted log. This setting only applies if the [log cleanup policy](#settings-topic-cleanup-policy) is set either to `Compact` or `CompactAndDelete`.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/topic-configs/#topicconfigs_delete.retention.ms).

* **Flush messages** <code><b><small>All interfaces</small></b></code> {#settings-topic-flush-messages}

    Number of topic messages that can accumulate in memory before being flushed to disk.

    For more information, see the description of the [Log flush interval messages](#settings-log-flush-interval-messages) cluster-level setting.

    In the management console, this setting appears as **Flush messages**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/topic-configs/#topicconfigs_flush.messages).

* **Flush, ms** <code><b><small>All interfaces</small></b></code> {#settings-topic-flush-ms}

    Maximum time in milliseconds a message can be stored in memory before being flushed to disk.    

    For more information, see the description of the [Log flush interval ms](#settings-log-flush-interval-ms) cluster-level setting.

    In the management console, this setting appears as **Flush, ms**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/topic-configs/#topicconfigs_flush.ms).

* **Message max bytes** <code><b><small>All interfaces</small></b></code> {#settings-topic-message-max-bytes}    

    Maximum size of a message sent by the producer.

    The minimum value is `0` (no limit); default: `1048588`. The value of this setting must not exceed the **Replica fetch max bytes** value by more than 12 bytes.

    In the management console, this setting appears as **Maximum batch size**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/topic-configs/#topicconfigs_max.message.bytes).

* **Min compaction lag, ms** <code><b><small>All interfaces</small></b></code> {#settings-topic-max-compaction-lag-ms}    

    Minimum time a message remains uncompacted in the log.

    In the management console, this setting appears as **Min compaction lag, ms**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/topic-configs/#topicconfigs_min.compaction.lag.ms).

* **Minimum number of in-sync replicas** <code><b><small>All interfaces</small></b></code> {#settings-topic-min-insync-replicas}

    Minimum number of replicas that have to acknowledge a write to consider a message successfully written to a topic. Use this setting if the producer waits too long for write acknowledgments from all broker hosts in the cluster.

    The minimum value depends on the number of [broker hosts](brokers.md):

    * For clusters with one broker host: `1`.
    * For clusters with two or more broker hosts: `2`.

    In the management console, this setting appears as **Minimum number of in-sync replicas**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/topic-configs/#topicconfigs_min.insync.replicas).

* **Num partitions** <code><b><small>All interfaces</small></b></code> {#settings-topic-num-partitions}    

    Number of log partitions per topic.

    The minimum and default value is `1`.

    For more information, see the description of the [Num partitions](#settings-num-partitions) cluster-level setting.

    In the management console, this setting appears as **Number of partitions**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_num.partitions).

* **Pre-allocation of the segment file** <code><b><small>All interfaces</small></b></code> {#settings-topic-preallocate}    

    This setting determines whether to pre-allocate space for log segment files. By default, the space for log segment files is allocated as the segments get full (`false`).

    For more information, see the description of the [Log preallocate](#settings-log-preallocate) cluster-level setting.

    In the management console, this setting appears as **Pre-allocation of the segment file**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/topic-configs/#topicconfigs_preallocate).

* **Replication factor** <code><b><small>All interfaces</small></b></code> {#settings-topic-replication-factor}

    Number of [data replicas](brokers.md) per topic.

    The minimum and default value is `1`. The maximum value is equal to the number of broker hosts in the cluster.

    For more information, see the description of the [Default replication factor](#settings-default-replication-factor) cluster-level setting.

    In the management console, this setting appears as **Replication factor**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/kafka-streams-configs/#streamsconfigs_replication.factor).

* **Retention, bytes** <code><b><small>All interfaces</small></b></code> {#settings-topic-retention-bytes}

    Maximum size a partition can grow to.

    For more information, see the description of the [Log retention bytes](#settings-log-retention-bytes) cluster-level setting.

    In the management console, this setting appears as **Retention, bytes**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/topic-configs/#topicconfigs_retention.bytes).

* **Retention, ms** <code><b><small>All interfaces</small></b></code> {#settings-topic-log-retention-ms}

    Time (in milliseconds) for Apache Kafka® to keep a log segment file. 

    For more information, see the description of the [Log retention ms](#settings-log-retention-ms) cluster-level setting.

    In the management console, this setting appears as **Retention, ms**.

    For a full description of this setting, see [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/topic-configs/#topicconfigs_retention.ms).

* **Segment bytes** <code><b><small>All interfaces</small></b></code> {#settings-topic-segment-bytes}

    This setting determines the segment size for log files, in bytes.

    Segment files are saved and cleared file-by-file. A smaller segment size results in more log files but offers more flexible storage management.

    The minimum value is `14` and the default is `1073741824` (1 GB).

    For more information, see the description of the [Log segment bytes](#settings-log-segment-bytes) cluster-level setting.

    For a full description of this setting, see [Maximum log segment size](storage.md#maximum-log-segment-size) and [this Apache Kafka® guide](https://kafka.apache.org/42/configuration/topic-configs/#topicconfigs_segment.bytes).