Reliability Test Cases

The Compatibility Testing tool has integrated reliability test cases.

The following test cases describe the specific test procedures for each reliability test item in the Compatibility Testing tool.

Table 1 describes the CPU stress test case.

**Table 1** CPU stress test case
Item	Description
Case Name	Reliability_Pressure_Cpu
Objective	Verify the CPU utilization fluctuation during the stable running of the target software.
Prerequisites	The environment preparations are complete for the compatibility test, that is, the test environment must remain idle (no load). The target software can be started, and the load can be increased so that CPU utilization exceeds the default idle level after startup. The load may be generated using a performance testing tool selected based on user-specific service characteristics (configurable in the configuration script of the Compatibility Testing tool) or through manual service simulation. The sar tool is available. Using CentOS 7.6 as an example, the tool comes from the *sysstat..rpm** package. It is recommended to mount the ISO file and configure a yum repository for installation.
Procedure	Start the target software by running the software startup command or script in the configuration file of the Compatibility Testing tool. If the command or script is not configured, wait until the software is manually started. Keep services running under a certain pressure for 60 minutes. Record the CPU %idle value (X_i) during running. Then, run the following command: # sar -u 60 60 Stop the target software by running the software shutdown command or script in the configuration file of the Compatibility Testing tool. If the command or script is not configured, wait until the software is manually stopped. Calculate the CPU utilization fluctuation before and after the test (X_max–X_min).
Expected Result	The CPU utilization fluctuation is smaller than 5%.

Table 2 describes the memory stress test case.

**Table 2** Memory stress test case
Item	Description
Case Name	Reliability_Pressure_Memory
Objective	Verify the memory utilization fluctuation during the stable running of the target software.
Prerequisites	The environment preparations are complete for the compatibility test, that is, the test environment must remain idle (no load). The target software can be started, and the load can be increased so that CPU utilization exceeds the default idle level after startup. The load may be generated using a performance testing tool selected based on user-specific service characteristics (configurable in the configuration script of the Compatibility Testing tool) or through manual service simulation. The sar tool is available. Using CentOS 7.6 as an example, the tool comes from the *sysstat..rpm** package. It is recommended to mount the ISO file and configure a yum repository for installation.
Procedure	Start the target software by running the software startup command or script in the configuration file of the Compatibility Testing tool. If the command or script is not configured, wait until the software is manually started. Keep services running under a certain pressure for 60 minutes. Record the memory %memused value (X_i) during running. Then, run the following command: # sar -r 60 60 Stop the target software by running the software shutdown command or script in the configuration file of the Compatibility Testing tool. If the command or script is not configured, wait until the software is manually stopped. Calculate the memory utilization fluctuation before and after the test (X_max–X_min).
Expected Result	The memory utilization fluctuation is smaller than 5%.

Table 3 describes the disk stress test case.

**Table 3** Disk stress test case
Item	Description
Case Name	Reliability_Pressure_Disk
Objective	Verify the disk utilization fluctuation during the stable running of the target software.
Prerequisites	The environment preparations are complete for the compatibility test, that is, the test environment must remain idle (no load). The target software can be started, and the load can be increased so that CPU utilization exceeds the default idle level after startup. The load may be generated using a performance testing tool selected based on user-specific service characteristics (configurable in the configuration script of the Compatibility Testing tool) or through manual service simulation. The sar tool is available. Using CentOS 7.6 as an example, the tool comes from the *sysstat..rpm** package. It is recommended to mount the ISO file and configure a yum repository for installation.
Procedure	Start the target software by running the software startup command or script in the configuration file of the Compatibility Testing tool. If the command or script is not configured, wait until the software is manually started. Keep services running under a certain pressure for 60 minutes. Record the disk %util value (X_i) during running. Then, run the following command: # sar -d -p 60 60 Stop the target software by running the software shutdown command or script in the configuration file of the Compatibility Testing tool. If the command or script is not configured, wait until the software is manually stopped. Calculate the disk utilization fluctuation before and after the test (X_max–X_min). Note: Filter out disk data not related to the execution of the target software.
Expected Result	The disk utilization fluctuation is smaller than 5%.

Table 4 describes the network stress test case.

**Table 4** Network stress test case
Item	Description
Case Name	Reliability_Pressure_Network
Objective	Verify the network utilization fluctuation during the stable running of the target software.
Prerequisites	The environment preparations are complete for the compatibility test, that is, the test environment must remain idle (no load). The target software can be started, and the load can be increased so that CPU utilization exceeds the default idle level after startup. The load may be generated using a performance testing tool selected based on user-specific service characteristics (configurable in the configuration script of the Compatibility Testing tool) or through manual service simulation. The sar tool is available. Using CentOS 7.6 as an example, the tool comes from the *sysstat..rpm** package. It is recommended to mount the ISO file and configure a yum repository for installation.
Procedure	Start the target software by running the software startup command or script in the configuration file of the Compatibility Testing tool. If the command or script is not configured, wait until the software is manually started. Keep services running under a certain pressure for 60 minutes. Record the network %rx value (X_i) and %tx value (Y_i) during running. Then, run the following command: # sar -n DEV 60 60 Stop the target software by running the software shutdown command or script in the configuration file of the Compatibility Testing tool. If the command or script is not configured, wait until the software is manually stopped. Calculate the network utilization RX fluctuation and TX fluctuation before and after the test, that is, (X_max–X_min)/X_mean and (Y_max–Y_min)/Y_mean respectively. Note: Filter out NIC data not related to the execution of the target software.
Expected Result	Both the network utilization RX fluctuation and TX fluctuation are smaller than 5%. A conditional pass may occur, which could be caused by a misjudgment during the compatibility testing process, OS or server-related issues, or inherent characteristics of the application leading to system resource fluctuations exceeding 5% during runtime. In such cases, only the measured fluctuation data shall be provided to the user for reference, enabling comparative analysis based on actual service characteristics or x86 runtime behavior.

Table 5 describes the exception test case.

**Table 5** Exception test case
Item	Description
Case Name	Reliability_Exception_Kill
Objective	Verify whether the target software can start as scheduled and run stably after being stopped unexpectedly, and no error is reported during the process.
Prerequisites	The environment preparations are complete for the compatibility test.
Procedure	Run the target software startup command or script in the configuration file of the Compatibility Testing tool. If the command or script is not configured, wait until the software is manually started. Run the kill -9 command to forcibly stop the background process of the target software. After the background process is forcibly stopped, restart the target software and check whether the corresponding process exists in the system process list. Check whether any new error is recorded in the /var/log/messages file.
Expected Result	After the target software is forcibly stopped, its background process does not exist in the system process list. After the target software is started, its background process is detected in the system process list. No errors are reported when the target software is being stopped and started.

Parent topic: Test Cases