Multi-Network Plane Instructions

If the operating environment of the mpirun command contains multiple network planes, that is, the UCX module uses network ports in different network segments, add -x UCX_RC_VERBS_ROCE_LOCAL_SUBNET=y and -x UCX_UD_VERBS_ROCE_LOCAL_SUBNET=y to the boot options of the mpirun command. Otherwise, the task delivery will fail due to network disconnection.

Example:

mpirun -np 16 -N 16 -x PATH -x LD_LIBRARY_PATH -x UCX_RC_VERBS_ROCE_LOCAL_SUBNET=y -x UCX_UD_VERBS_ROCE_LOCAL_SUBNET=y -x UCX_NET_DEVICES=mlx5_0:1,mlx5_1:1 osu_init