Configuring Kafka Parameters
Purpose
Modify Kafka parameters to prevent Kafka from becoming a bottleneck.
Procedure
On the Storm web page, search for and modify the following parameters:
Parameter |
Recommended Value |
Description |
|---|---|---|
log.dir |
Set this parameter based on the site requirements. |
This parameter specifies the Kafka data storage directories (you can set multiple directories). |
num.partitions |
Set this parameter based on the site requirements. |
This parameter specifies the number of automatically created topic partitions. Generally, the value is 1 to 2 times the number of physical drives on the Kafka service node. |
num.network.threads |
128 |
This parameter specifies the number of threads used by the broker to process network requests. The maximum value is 128. |
num.io.threads |
8 |
Number of threads used by the Kafka broker to process drive I/Os. |