Accuracy of Allreduce Algorithm 8 Causing Inconsistent Execution Results
Symptom
The Allreduce algorithm 8 is executed for multiple times, but the floating-point calculation results are inconsistent.
Possible Cause
The reduction order of Allreduce algorithm 8 cannot be fixed, which affects the accuracy of floating-point calculation.
Recovery Procedure
In scenarios that require high accuracy, use Allreduce algorithm 1 to ensure consistent accuracy.
Parent topic: FAQ