Data Distribution
Processes in ScaLAPACK are arranged on a 2D (more common) or 1D process grid.
On a 2D grid, if a process grid consists of P process rows and Q process columns, coordinates of a process on the grid may be represented as (p, q), where 0 ≤ p ≤ P, and 0 ≤ q ≤ Q. The following figure shows the process distribution of six nodes.
In the preceding figure, processes are mapped into a process grid in row-major order. Similarly, processes can also be mapped into a process grid in column-major order. By default, row-major order is used for mapping.
After a process grid is determined, a matrix is cyclically distributed to the process grid according to a preset block size mb x nb. The following figure shows the numbering of the processes where the blocks reside after the distribution.
When P or Q is set to 1, the process grid is set to 1D, and coordinates of a process in the grid may be set to p or q according to the process number, where 0 ≤ p ≤ P or 0 ≤ q ≤ Q.
Matrices of a same size are allocated to the 1D process grid. The following figure shows the matrix division and related process numbering of the column blocks.
You can also set Q to 1, so that data blocks on the process grid are divided by row.