--- title: （可选）安装knem组件 description: "knem是MPI节点内通信的一个增强组件，在部分高性能计算应用中，对于MPI通信中大包场景，安装使能该组件后会有一些性能提升。" url: https://www.hikunpeng.com/document/detail/zh/kunpenghpcs/hpckit/instg/KunpengHPCKit_install_021.html sourcePath: /source/zh/kunpenghpcs/hpckit/instg/KunpengHPCKit_install_021.html indexId: 16baeea2e5e5702e423341c90a90df45e3a032ebcfdccd160be952ff1d49ee7a66 --- # （可选）安装knem组件 knem是MPI节点内通信的一个增强组件，在部分高性能计算应用中，对于MPI通信中大包场景，安装使能该组件后会有一些性能提升。 #### 检查knem使能情况 安装knem组件前，可以先检查knem使能情况，如果可以使能成功则无需再安装knem。 1. 执行命令cat /dev/knem，回显如下表示可以使能成功。 ``` knem 1.1.4.90 (git20250207.4b4ebc6) Driver ABI=0xd Flags: forcing 0x0, ignoring 0x0 DMAEngine: NoKernelSupport Debug: NotBuilt Requests submitted : 16320 Requests processed (total) : 16320 processed (using DMA) : 0 processed (offloaded to thread) : 0 processed (with pinned local pages) : 0 Requests rejected (invalid flags) : 0 rejected (not enough memory) : 0 rejected (invalid ioctl argument) : 0 rejected (unexisting region cookie) : 0 rejected (failed to pin local pages): 0 Requests failed during memcpy from/to user : 0 failed during DMA copy : 0 DMA copy cleanup timeout : 0 ``` 如果knem未正常使能，可以尝试安装knem后，再次使用上述方法确认使能情况。 #### 安装knem knem需要在每个计算节点安装。 1. 下载rpm包（不同os对应的rpm包不同）。 - openEuler 20.03 LTS AArch64 ``` wget --no-check-certificate https://repo.oepkgs.net/openeuler/rpm/openEuler-20.03-LTS-SP1/extras/aarch64/Packages/k/knem-1.1.4.90-0.aarch64.rpm(https://repo.oepkgs.net/openeuler/rpm/openEuler-20.03-LTS-SP1/extras/aarch64/Packages/k/knem-1.1.4.90-0.aarch64.rpm) ``` - openEuler 22.03 LTS AArch64 ``` wget --no-check-certificate https://repo.oepkgs.net/openeuler/rpm/openEuler-22.03-LTS/extras/aarch64/Packages/k/knem-1.1.4.90-0.aarch64.rpm(https://repo.oepkgs.net/openeuler/rpm/openEuler-22.03-LTS/extras/aarch64/Packages/k/knem-1.1.4.90-0.aarch64.rpm) ``` - Kylin Linux Advanced Server V10 （HPC）AArch64 ``` wget --no-check-certificate https://repo.oepkgs.net/openeuler/rpm/openEuler-22.03-LTS/extras/aarch64/Packages/k/knem-1.1.4.90-0.ky10h.aarch64.rpm(https://repo.oepkgs.net/openeuler/rpm/openEuler-22.03-LTS/extras/aarch64/Packages/k/knem-1.1.4.90-0.ky10h.aarch64.rpm) ``` - Kylin Linux Advanced Server V10 SP2 AArch64 ``` wget --no-check-certificate https://repo.oepkgs.net/openeuler/rpm/openEuler-20.03-LTS-SP1/extras/aarch64/Packages/k/knem-1.1.4.90-0.ky10.aarch64.rpm(https://repo.oepkgs.net/openeuler/rpm/openEuler-20.03-LTS-SP1/extras/aarch64/Packages/k/knem-1.1.4.90-0.ky10.aarch64.rpm) ``` - 麒麟信安服务器操作系统V3.5.2 AArch64 ``` wget --no-check-certificate https://repo.oepkgs.net/openeuler/rpm/openEuler-22.03-LTS/extras/aarch64/Packages/k/knem-1.1.4.90-0.aarch64.rpm(https://repo.oepkgs.net/openeuler/rpm/openEuler-22.03-LTS/extras/aarch64/Packages/k/knem-1.1.4.90-0.aarch64.rpm) ``` 2. 安装rpm包。 ``` rpm -ivh rpm包名称 ``` 如果回显提示有系统自带rpm包，新的rpm包无法被安装，则需要手动卸载系统自带rpm包，再重新安装。 3. 检查knem内核是否加载，如果回显出现“knem”说明加载成功。 lsmod | grep knem 1 knem 57344 4 4. 执行命令cat /dev/knem，回显如下表示安装成功。 ``` knem 1.1.4.90 (git20250207.4b4ebc6) Driver ABI=0xd Flags: forcing 0x0, ignoring 0x0 DMAEngine: NoKernelSupport Debug: NotBuilt Requests submitted : 16320 Requests processed (total) : 16320 processed (using DMA) : 0 processed (offloaded to thread) : 0 processed (with pinned local pages) : 0 Requests rejected (invalid flags) : 0 rejected (not enough memory) : 0 rejected (invalid ioctl argument) : 0 rejected (unexisting region cookie) : 0 rejected (failed to pin local pages): 0 Requests failed during memcpy from/to user : 0 failed during DMA copy : 0 DMA copy cleanup timeout : 0 ``` 5. 执行以下命令，运行knem的自带测试用例knem_pingpong，默认安装在/opt/knem-1.1.4.90 。 ``` cd /path/to/install ``` ``` /path/to/install / bin/knem_pingpong ``` /path/to/install表示knem安装目录。 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 1024: 25.705 us 39.84 MB/s 37.99 MiB/s 2048: 25.869 us 79.17 MB/s 75.50 MiB/s 4096: 29.301 us 139.79 MB/s 133.32 MiB/s 8192: 30.902 us 265.09 MB/s 252.81 MiB/s 16384: 35.213 us 465.28 MB/s 443.73 MiB/s 32768: 45.170 us 725.44 MB/s 691.83 MiB/s 65536: 69.500 us 942.96 MB/s 899.28 MiB/s 131072: 62.795 us 2087.30 MB/s 1990.61 MiB/s 262144: 91.160 us 2875.64 MB/s 2742.43 MiB/s 524288: 158.172 us 3314.67 MB/s 3161.12 MiB/s 1048576: 216.383 us 4845.93 MB/s 4621.44 MiB/s 2097152: 310.549 us 6753.05 MB/s 6440.21 MiB/s 4194304: 329.018 us 12747.96 MB/s 12157.40 MiB/s 8388608: 615.082 us 13638.19 MB/s 13006.40 MiB/s 16777216: 1232.287 us 13614.70 MB/s 12983.99 MiB/s