Tuning Strategy

Based on the discussion about the factors that influence MPI precision in the preceding section, you are advised to use either of the following methods for tuning:

• Unify reduction algorithm

  Use the following parameters on Hyper MPI and Intel MPI so that the reduction algorithm is performed according to the Recursive doubling algorithm. That is, the reduction is performed based on the process rank.

  • Add UCX_BUILTIN_ALLREDUCE_ALGORITHM=1 to Hyper MPI.
  • Add -genv I_MPI_ADJUST_ALLREDUCE=1 to Intel MPI.
• Rewrite reduction algorithm

  Re-implement the Allreduce algorithm. Sort the values of each process before reduction.