Recommended Configuration of UCX Environment Variables
TCP Scenario
- UCX_TCP_TX_SEG_SIZE: size of the buffer on the transmit (TX) side. The default value is 8 KB.
- UCX_TCP_TX_MAX_BUFS: maximum number of buffers in the buffer pool on the TX side. The default value is -1, indicating infinite number of buffers.
- UCX_TCP_TX_BUFS_GROW: number of buffers that are added every time the buffer pool grows on the TX side. The default value is 8.
- UCX_TCP_RX_SEG_SIZE: size of the buffer on the receive (RX) side. The default value is 64 KB.
- UCX_TCP_RX_MAX_BUFS: maximum number of buffers in the buffer pool on the RX side. The default value is -1, indicating infinite number of buffers.
- UCX_TCP_RX_BUFS_GROW: number of buffers that are added every time the buffer pool grows on the RX side. The default value is 8.
- UCX_TCP_MAX_NUM_EPS: maximum number of endpoints (sockets) that can be created. The default value is 256.
- UCX_TCP_SNDBUF: size of the transmit buffer of the kernel socket. The default value is auto.
- UCX_TCP_RCVBUF: size of the receive buffer of the kernel socket. The default value is auto.
RC Verbs Scenario
- UCX_RC_VERBS_MAX_NUM_EPS: maximum number of RC endpoints that can be created. The default value is 256.
- UCX_RC_VERBS_TX_QUEUE_LEN: length of the TX queue in a queue pair (QP). The default value is 256.
- UCX_RC_VERBS_SEG_SIZE: size of the buffer. The default value is 8256.
- UCX_RC_VERBS_TX_MAX_BUFS: maximum number of buffers in the buffer pool on the TX side. The default value is -1, indicating infinite number of buffers.
- UCX_RC_VERBS_TX_BUFS_GROW: number of buffers that are added every time the buffer pool grows on the TX side. The default value is 1024.
- UCX_RC_VERBS_TX_CQ_LEN: maximum length of a completion queue (CQ) in a QP on the TX side, which limits the total number of sent requests. The default value is 4096.
- UCX_RC_VERBS_RX_QUEUE_LEN: length of the RX queue in a QP. The default value is 4095.
- UCX_RC_VERBS_RX_MAX_BUFS: maximum number of buffers in the buffer pool on the RX side. The default value is -1, indicating infinite number of buffers.
- UCX_RC_VERBS_RX_BUFS_GROW: number of buffers that are added every time the buffer pool grows on the RX side. The default value is 0 (automatically set by the UCX).
RC MLX5 Scenario
- UCX_RC_MLX5_TX_QUEUE_LEN: length of the TX queue in a QP. The default value is 256.
- UCX_RC_MLX5_SEG_SIZE: size of the buffer. The default value is 8256.
- UCX_RC_MLX5_TX_MAX_BUFS: maximum number of buffers in the buffer pool on the TX side. The default value is -1, indicating infinite number of buffers.
- UCX_RC_MLX5_TX_BUFS_GROW: number of buffers that are added every time the buffer pool grows on the TX side. The default value is 1024.
- UCX_RC_MLX5_TX_CQ_LEN: maximum length of a CQ in a QP on the TX side, which limits the total number of sent requests. The default value is 4096.
- UCX_RC_MLX5_RX_QUEUE_LEN: length of the RX queue in a QP. The default value is 4095.
- UCX_RC_MLX5_RX_MAX_BUFS: maximum number of buffers in the buffer pool on the RX side. The default value is -1, indicating infinite number of buffers.
- UCX_RC_MLX5_RX_BUFS_GROW: number of buffers that are added every time the buffer pool grows on the RX side. The default value is 0 (automatically set by the UCX).
Parent topic: Reference