Tuning Strategy

To tune performance, you need to identify problems, locate performance bottlenecks, and select a tuning method based on the bottleneck level.

The tuning analysis process is as follows:

• Server bottleneck: Focus on analyzing hardware metrics, including CPU, memory, drive, and BIOS configurations. For compute-intensive tasks (such as K-means clustering), CPU utilization is the primary metric to monitor.
• Network bottleneck: Optimizing network configurations is critical. Specifically, binding interrupts to specific cores can significantly enhance task processing performance.
• When tuning SQL performance, you are advised to start with baseline configurations for the client and WebUI. Use theoretical formulas to derive optimal Executor parameters. On the Kunpeng platform, leverage its affinity features to fully exploit hardware advantages for superior SQL performance.
• In the HiBench scenario, you can set the partition count and parallelism to 3 to 5 times the total number of cluster cores. This implements fine-grained data partitioning, and reduces the data payload per task, boosting overall job throughput.
• During the tuning, strive to maintain high utilization of both CPU and memory. If it is impossible to run both CPU and memory at full capacity simultaneously, prioritize high CPU utilization, then determine whether additional memory is required based on GC logs. For memory-intensive scenarios, the final tuning result may result in memory utilization approaching full capacity while maintaining a certain CPU margin.