Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Creating an HPC Application Analysis Task

Function

The tool collects Performance Monitor Unit (PMU) events of the system and the key metrics of OpenMP and MPI applications to help accurately obtain the serial and parallel time of the parallel region and barrier-to-barrier, calibrated L2 microarchitecture metrics, instruction distribution, L3 usage, and memory bandwidth.

During OpenMP data collection, the kernel parameters /proc/sys/kernel/kptr_restrict and /proc/sys/kernel/perf_event_paranoid are enabled to collect call graph data and PMU events. After the collection is complete, the two kernel parameters are restored to their original values.

Prerequisites

  • No nodes are in the Offline state.
  • HPC application analysis is only applicable to physical servers powered by Kunpeng 920 processors, because the top-down and DRAM bandwidth data in HPC application tasks requires OS kernel 4.19 or later or patched openEuler 4.14 or later.
  • On an Ubuntu OS, data may fail to be collected in an HPC application analysis task. For details, see Failed to Collect HPC Application Analysis Task Data on Ubuntu. Ubuntu 20.04.1 does not support HPC application analysis.
  • To use MPICH, add the path of libmpi.so.12 to LD_LIBRARY_PATH, which is generally the lib folder in the MPICH installation path.
  • The C, C++, and Fortran programming languages must be enabled for MPI (Open MPI/MPICH). For example, for Open MPI that uses the GCC, run --enable-mpi-compatibility CC=gcc CXX=g++ FC=gfortran to enable C, C++, and Fortran.
  • Choose Open MPI 4.1.6 (preferred) or a later version and MPICH 4.3.0 (preferred) or a later version.

Procedure

  1. Click next to System Profiler.

    Choose General Analysis from the drop-down list. The page for creating a task is displayed.

  2. Set task parameters by referring to "Task Management" and Table 1.
    See Figure 1.
    Figure 1 Creating an HPC application analysis task
    Table 1 Parameters for creating an HPC application analysis task

    Parameter

    Description

    Task Name

    Name of the task. The name must meet the following requirements:

    1. Contain only letters, digits, and underscores (_).
    2. Contain 1 to 64 characters.

    Select Nodes

    Select the nodes to be analyzed. If there is only one node, this node is selected by default. A maximum of 10 nodes can be selected at a time. This parameter is mandatory when Collection Mode is set to OpenMP.

    Analysis Object

    Select Application. Currently, only Application is supported.

    Mode

    Select Launch application. Currently, only Launch application is supported.

    This parameter is mandatory when Analysis Object is set to Application.

    Application Path

    Enter the absolute path of the application to be analyzed. For example, to analyze the loop_test application stored in the /home/test directory, enter /home/test/loop_test.

    NOTE:
    • By default, applications in the /opt/ or /home/ directory are analyzed. The administrator can click in the upper right corner on the home page and choose Tool Settings > System Profiler > System Settings, and enter application paths (separated by semicolons) in the Application Path text box. Only administrators can modify this parameter. Common users can only view this parameter.
    • You are advised to set the application path to a path such as /home or /opt. Do not set the application path to a system directory such as /, /dev, /sys, or /boot. Otherwise, system exceptions may occur.
    • The OS running user (devkitworker1) of the System Profiler must have the read and execute permissions on the applications to be analyzed.
    • In the multi-node scenario, you can enable Configure Node Parameters to configure this parameter separately for each node.

    (Optional) Application Parameters

    Set application parameters based on the actual scenario.

    NOTE:

    You can enable Configure Node Parameters to configure this parameter separately for each node.

    (Optional) Application User

    Information about the OS user who runs the application. The application runs under the preset devkitworker1 user by default. If the application only runs under a specific user, enable the option, configure the corresponding user name and password, and then run the application.

    This parameter is available when Collection Mode is set to OpenMP. By default, this parameter is disabled.

    Name

    Name of the OS user who runs the application.

    This parameter is mandatory when Application User is enabled.

    Password

    User password.

    This parameter is mandatory when Application User is enabled.

    Analysis Type

    Select HPC Application.

    Sampling Duration (s)

    Sampling duration. The default value is 300. When Sampling Mode is set to Detail, the value ranges from 1 to 600. When Sampling Mode is set to Summary, the value ranges from 1 to 1800. If the value exceeds the range, you will be prompted and an autocomplete is performed upon the blur event.

    If Collection Mode is set to MPI/MPI+OpenMP, the sampling duration defaults to Limited application running time. You can also select Unlimited application running time, for which the sampling will not stop until the application stops running.

    NOTE:
    • A longer sampling duration requires more memory resources and a longer analysis duration. If you select Unlimited for Sampling Duration, ensure that the application stops before system resources are used up.
    • The task may fail if the sampling duration is too short and the application running has not been completed.

    Sampling Delay (s)

    The sampling starts after the specified time. This parameter is used to ignore the program startup analysis, warm up the sampling program, and eliminate sampling delay caused by factors such as environment detection. The default value is 0. The value range is 0 to 900. If the value exceeds the range, you will be prompted and an autocomplete is performed upon the blur event.

    (Optional) Configure Node Parameters

    Set node parameters as required. If you do not need to set parameters for each node separately, all nodes are executed based on the current parameter settings.

    This parameter is available when Collection Mode is set to OpenMP and when there are multiple nodes.

    Collection Mode

    Collection mode. The options are:

    • OpenMP
    • MPI/MPI+OpenMP

    (Optional) MPI Only

    Only MPI data is collected while OpenMP data is not.

    This option is disabled by default. This parameter is available in Advanced Configurations when Collection Mode is set to MPI/MPI+OpenMP and Analysis Mode is set to Statistical analysis.

    mpirun Running Node

    Node where the mpirun process is running.

    This parameter is mandatory when Collection Mode is set to MPI/MPI+OpenMP.

    Shared Directory

    Shared directory for analysis. This parameter is mandatory when Collection Mode is set to MPI/MPI+OpenMP.

    The shared directory can be in /opt/, /home/, or /mysharedir/. Contact the administrator to configure the directory.

    NOTE:

    Properly configure the permissions on the shared directory to prevent network security risks caused by excessive permissions.

    Cluster Check Account

    Running user. Possible values are mpirun user (default) and root user.

    NOTE:

    Before checking the cluster as the root user, ensure that password-free login has been configured for the root user in the cluster.

    After a node in the cluster is restarted, you need to check the cluster as the root user. Otherwise, the perf-related collection configuration does not support mpirun execution as a common user.

    Root User Password

    Enter the password of the root user.

    This parameter is mandatory when Collection Mode is set to MPI/MPI+OpenMP and Cluster Check Account is set to root user.

    (Optional) mpirun Parameter

    Used for setting the MPI environment variable -x.

    This parameter is available when Collection Mode is set to MPI/MPI+OpenMP.

    NOTE:

    Specify the task node (for example, --hostfile) in the parameter. If only the -np parameter is entered, the task will be randomly sent to a node. As a result, the cluster and MPI operating environment are inconsistent, and the task fails.

    mpirun Path

    MPI path.

    This parameter is mandatory when Collection Mode is set to MPI/MPI+OpenMP.

    (Optional) rankfile

    Upload the rank configuration file. The file content is in the following format: rank <N>=<host_name>slot=<slot_list>

    This parameter is available when Collection Mode is set to MPI/MPI+OpenMP.

    Analysis Mode

    Possible values are Statistical analysis (default) and Refined analysis.

    NOTE:

    SSH password-free login must be configured for nodes running MPI tasks. If it is not configured, enable SSH password-free login for the user who runs the mpirun process.

    (Optional) Collect Critical Path

    Indicates whether to collect critical paths. By default, critical paths are not collected.

    This parameter is available when Collection Mode is set to MPI/MPI+OpenMP and Analysis Mode is set to Statistical analysis.

    (Optional) Basic Information Collection

    Indicates whether to collect RDMA or shared storage data. By default, this option is not selected.

    This parameter is available when Collection Mode is set to MPI/MPI+OpenMP and Analysis Mode is set to Refined analysis.

    Basic Information Sampling Interval

    Interval for collecting RDMA or shared storage data. The value ranges from 1 to 15 seconds, and is 1 second by default.

    This parameter is available when Collection Mode is set to MPI/MPI+OpenMP, Analysis Mode is set to Refined analysis, and Basic Information Collection is selected.

    Sampling Mode

    Data sampling mode. The default value is Detail. This parameter is mandatory when Analysis Mode is set to Statistical analysis.

    • Summary: collects basic metrics. This mode has low sampling and analysis overheads.
    • Detail: collects hot functions and detailed metrics. This mode has high sampling and analysis overheads.

    (Optional) Collect More Call Stack Statistics

    Indicates whether to collect call stacks and top-down events. This parameter is disabled by default. It is available in Advanced Configurations when Analysis Mode is set to Refined analysis.
  3. Click OK.

    You can click the icons next to the task name to perform the following operations:

    • : cancels the analysis task. After an analysis task is canceled, the collected information will be deleted. You can also use this button to start an analysis task. Starting analysis will interrupt the current collection and use the collected data for analysis. The task may fail because the application is stopped.
    • : restarts the analysis task. You can modify task parameter settings and restart an analysis task. This button is available when a task is canceled or fails.
    • : deletes the analysis task. After a task is deleted, all data of this task will be deleted. Exercise caution when performing this operation.
    • : performs the analysis again. The analysis task is renamed and restarted.
    • : changes the task or report name. The report naming rule is the same as that of a task.
Parent topic: HPC Application Analysis