我现在使用鲲鹏920(Kunpeng-920)+麒麟(ky10.aarch64)+deepseek-r1:8b部署,推理很卡,还比不上本地的I7处理器+T500小显卡,想了解yi'xia啥原因?
服务器CPU信息如下:
[root@localhost home]# uname -a
Linux localhost.localdomain 4.19.90-52.22.v2207.ky10.aarch64 #1 SMP Tue Mar 14 11:52:45 CST 2023 aarch64 aarch64 aarch64 GNU/Linux
[root@localhost home]# lscpu
架构: aarch64
CPU 运行模式: 64-bit
字节序: Little Endian
CPU: 96
在线 CPU 列表: 0-95
每个核的线程数: 1
每个座的核数: 48
座: 2
NUMA 节点: 4
厂商 ID: HiSilicon
型号: 0
型号名称: Kunpeng-920
步进: 0x1
CPU 最大 MHz: 2600.0000
CPU 最小 MHz: 200.0000
BogoMIPS: 200.00
L1d 缓存: 6 MiB
L1i 缓存: 6 MiB
L2 缓存: 48 MiB
L3 缓存: 96 MiB
NUMA 节点0 CPU: 0-23
NUMA 节点1 CPU: 24-47
NUMA 节点2 CPU: 48-71
NUMA 节点3 CPU: 72-95
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
标记: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop asimddp asimdfhm
本机电脑配置: Core i7-1165G7 +NVIDIA T500(2G)
我现在使用鲲鹏920(Kunpeng-920)+麒麟(ky10.aarch64)+deepseek-r1:8b部署,推理很卡,还比不上本地的I7处理器+T500小显卡,想了解yi'xia啥原因?
服务器CPU信息如下:
[root@localhost home]# uname -a
Linux localhost.localdomain 4.19.90-52.22.v2207.ky10.aarch64 #1 SMP Tue Mar 14 11:52:45 CST 2023 aarch64 aarch64 aarch64 GNU/Linux
[root@localhost home]# lscpu
架构: aarch64
CPU 运行模式: 64-bit
字节序: Little Endian
CPU: 96
在线 CPU 列表: 0-95
每个核的线程数: 1
每个座的核数: 48
座: 2
NUMA 节点: 4
厂商 ID: HiSilicon
型号: 0
型号名称: Kunpeng-920
步进: 0x1
CPU 最大 MHz: 2600.0000
CPU 最小 MHz: 200.0000
BogoMIPS: 200.00
L1d 缓存: 6 MiB
L1i 缓存: 6 MiB
L2 缓存: 48 MiB
L3 缓存: 96 MiB
NUMA 节点0 CPU: 0-23
NUMA 节点1 CPU: 24-47
NUMA 节点2 CPU: 48-71
NUMA 节点3 CPU: 72-95
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
标记: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop asimddp asimdfhm
本机电脑配置: Core i7-1165G7 +NVIDIA T500(2G)