Specifying Algorithms

Allreduce Algorithms

• When using algorithms 13 and 14, add the following options to ensure that the number of processes is evenly distributed on sockets:

  --map-by-socket --rank-by-core

• When algorithm 5 or 6 is selected, the K-nomial algorithm is used within nodes. You can adjust the K value of the K-nomial tree structure to find the optimal parameter value that maximizes performance. The K value range is (0, Number of processes]. The example commands are as follows:

  -x UCG_PLANC_UCX_ALLREDUCE_FANOUT_INTRA_DEGREE=3

  -x UCG_PLANC_UCX_ALLREDUCE_FANIN_INTRA_DEGREE=8

• When algorithm 7 or 8 is selected, the K-nomial algorithm is used within nodes and across nodes. You can adjust the K value of the K-nomial tree structure to find the optimal parameter value that maximizes performance. The K value range is (0, Number of processes]. An example command is as follows:

  -x UCG_PLANC_UCX_ALLREDUCE_FANOUT_INTER_DEGREE=7 -x UCG_PLANC_UCX_ALLREDUCE_FANIN_INTER_DEGREE=7 -x UCG_PLANC_UCX_ALLREDUCE_FANOUT_INTRA_DEGREE=3 -x UCG_PLANC_UCX_ALLREDUCE_FANIN_INTRA_DEGREE=8

To improve performance, you can also add the following option to adjust the transmission mode:

-x UCX_TLS=sm,rc_x

The following is an example command used when MPI_Allreduce and MPI_Iallreduce are called (on Kunpeng processors):

mpirun -np 16 -N 2 --hostfile hf8 -x UCX_TLS=sm,rc_x -x UCG_PLANC_UCX_ALLREDUCE_ATTR=I:nS:200R:0- test_case

Bcast Algorithms

• When algorithm 3 is selected, the K-nomial algorithm is used for inter-node communication. You can adjust the K value to find the optimal parameter value that maximizes performance. The K value range is (0, Number of processes]. An example command is as follows:

  -x UCG_PLANC_UCX_BCAST_NA_KNTREE_INTER_DEGREE=7

• When algorithm 4 is selected, the K-nomial algorithm is used for intra-node and inter-node communication. You can adjust the K value to find the optimal parameter value that maximizes performance. The K value range is (0, Number of processes]. The example commands are as follows:

  -x UCG_PLANC_UCX_BCAST_NA_KNTREE_INTER_DEGREE=7

  -x UCG_PLANC_UCX_BCAST_NA_KNTREE_INTRA_DEGREE=3

• When algorithm 9 is selected, you can modify the following parameter to adjust the number of blocks divided by the ESBT algorithm and find the optimal parameter value that maximizes performance. The default value is 0, indicating auto. An example command is as follows:

  -x UCG_PLANC_UCX_BCAST_ESBT_BLOCKS=3

  To improve performance, you can add the following option (shared memory and reliable connection based on any transmission protocol) to adjust the transmission mode:

-x UCX_TLS=sm,rc_x

The following is an example of the command used when MPI_Bcast and MPI_Ibcast are called (Kunpeng processor):

mpirun -np 16 -N 2 --hostfile hf8 -x UCX_TLS=sm,rc_x -x UCG_PLANC_UCX_BCAST_ATTR=I:nS:200R:0- test_case

Barrier Algorithms

As listed in Algorithms, Barrier algorithms are a subset of Allreduce algorithms. For details about how to specify Barrier algorithms, see the description of specifying Allreduce algorithms.

Alltoallv Algorithms

When algorithm 1 is selected, you can change the value of the following parameter to adjust the number of communication requests each time based on the site requirements. An example command is as follows:

-x UCG_PLANC_UCX_ALLTOALLV_LADD_THROTTLED_FACTOR=2

When algorithm 5 is selected, you need to enable the following parameter to enable shared memory. An example command is as follows:

-x UCG_USE_SHM_POOL=y

To improve performance, you can also add the following option to adjust the transmission mode:

-x UCX_TLS=sm,rc_x

The following is an example of the command used when MPI_Alltoallv and MPI_Ialltoallv are called (on Kunpeng processors):

mpirun -np 16 -N 2 --hostfile hf8 -x UCX_TLS=sm,rc_x -x UCG_PLANC_UCX_ALLTOALLV_ATTR=I:nS:200R:0- test_case

Scatterv Algorithms

• When algorithm 1 is selected, you can adjust the values of the following parameters to find the optimal parameter values that maximize performance. The example commands are as follows:

  -x UCG_PLANC_UCX_SCATTERV_MIN_SEND_BATCH=7

  -x UCG_PLANC_UCX_SCATTERV_MAX_SEND_BATCH=8

• When algorithm 2 is selected, the K-nomial algorithm is used. You can adjust the K value to find the optimal parameter value that maximizes performance. An example command is as follows:

  -x UCG_PLANC_UCX_SCATTERV_KNTREE_DEGREE=3

• When algorithm 3 is selected, the Node-aware K-nomial tree algorithm is used. As the K-nomial algorithm is used for inter-node and intra-node communication, you can adjust the K value to find the optimal parameter value that maximizes performance. The example commands are as follows:

  -x UCG_PLANC_UCX_SCATTERV_NA_KNTREE_INTER_DEGREE=4

  -x UCG_PLANC_UCX_SCATTERV_NA_KNTREE_INTRA_DEGREE=3

• Algorithm 3 of the Scatterv operation has better performance when the data volume is small, but poorer performance when the data volume is large.

• The Scatterv operation cannot automatically select the optimal algorithm based on the data volume. Therefore, you are advised to use algorithm 3 when the data volume is small.

To improve performance, you can also add the following option to adjust the transmission mode:

-x UCX_TLS=sm,rc_x

The following is an example command used when MPI_Scatterv and MPI_Iscatterv are called (on Kunpeng processors):

mpirun -np 16 -N 2 --hostfile hf8 -x UCX_TLS=sm,rc_x -x UCG_PLANC_UCX_SCATTERV_ATTR=I:nS:200R:0- test_case

Allgatherv Algorithms

To improve performance, you can add the following option to adjust the transmission mode:

-x UCX_TLS=sm,rc_x

The following is an example of the command used when MPI_Allgatherv and MPI_Iallgatherv are called (Kunpeng processor):

mpirun -np 16 -N 2 --hostfile hf8 -x UCX_TLS=sm,rc_x -x UCG_PLANC_UCX_ALLGATHERV_ATTR=I:nS:200R:0- test_case

Gather Algorithms

• When algorithm 3 is selected, the K-nomial algorithm is used. You can adjust the K value to find the optimal parameter value that maximizes performance. Example command:

  -x UCG_PLANC_UCX_GATHER_KNTREE_DEGREE=3

• When algorithm 4 is selected, the Node-aware K-nomial tree algorithm is used. As the K-nomial algorithm is used for inter-node and intra-node communication, you can adjust the K value to find the optimal parameter value that maximizes performance. Example commands:

  -x UCG_PLANC_UCX_GATHER_NA_KNTREE_INTER_DEGREE=4

  -x UCG_PLANC_UCX_GATHER_NA_KNTREE_INTRA_DEGREE=3

Gatherv Algorithms

• When algorithm 3 is selected, the K-nomial algorithm is used. You can adjust the K value to find the optimal parameter value that maximizes performance. Example command:

  -x UCG_PLANC_UCX_GATHERV_KNTREE_DEGREE=3

• When algorithm 4 is selected, the Node-aware K-nomial tree algorithm is used. As the K-nomial algorithm is used for inter-node and intra-node communication, you can adjust the K value to find the optimal parameter value that maximizes performance. Example commands:

  -x UCG_PLANC_UCX_GATHERV_NA_KNTREE_INTER_DEGREE=4

  -x UCG_PLANC_UCX_GATHERV_NA_KNTREE_INTRA_DEGREE=3