Modifying Flink Parameters
Purpose
Modify Flink parameters to ensure that the maximum resources can be used by Flink.
Procedure
Modify the flink-conf.yaml file in the conf directory under the root directory of Flink.
Parameter |
Recommended Value |
Description |
|---|---|---|
vcores |
Adjust the value based on the actual scenarios. |
Indicates the number of YARN container vCores. One TaskManager is a container. This parameter specifies the number of vCores that can be used by a TaskManager. A JobManager is also a container. However, a JobManager is always allocated with one vCore, which is not affected by this parameter. This parameter has a great impact on performance. You can increase the value of this parameter as much as possible. The JobManager occupies one vCore. After the number of vCores is excluded, each node can use the cores of the node. Therefore, you can calculate the number of vCores based on the number of TaskManagers. |
network.numberofBuffers |
Adjust the value based on the actual scenarios. |
Indicates the number of taskmanager network sending buffers. If an error occurs during job running, indicating that the available buffer in the system is insufficient, you can increase the value of this parameter. |
taskmanager.compute.numa |
This parameter does not exist in the default Flink configuration file. You need to manually add this parameter. This parameter is used to enable NUMA core binding of TaskManager. This parameter must be used together with Yarn parameters. For details, see Parameter Configuration. |