Kunpeng Math Library Acceleration

Installing the Kunpeng Math Library

Some machine learning algorithms use math libraries such as BLAS and LAPACK to implement math operations in algorithms. Kunpeng optimizes such math libraries so that the algorithm performance can be improved. Perform the following steps to install the Kunpeng math library on each controller and compute node in the cluster:

Compile the JAR files and SO files related to netlib and jniloader based on the AArch64 architecture and upload them to the /opt/netlib/ directory.
- Obtain netlib files at https://github.com/fommil/netlib-java.
- Obtain jniloader files at https://github.com/ibmsoe/jniloader.
- Before compilation, ensure that the BLAS math library is not installed.
Install netlib dependencies.
1. gcc
  The version must be 4.8.5 or later. If it has been installed, skip this step.
```
yum install gcc
```
2. gfortran
```
yum install gcc-gfortran
```
3. libblas
```
yum install blas-devel
```
  Perform the verification:
```
ll /usr/lib64/ | grep libblas
```
4. liblapack
```
yum install lapack-devel
```
  Perform the verification:
```
ll /usr/lib64/ | grep liblapackll /usr/lib64/ | grep liblapack
```
Obtain and install the Kunpeng BLAS and LAPACK math libraries, and generate the KML_LAPACK dynamic library with complete functions by following instructions in Kunpeng Math Library Developer Guide.

Create a soft link to the installed math libraries.

Example:

sudo update-alternatives --install /usr/lib64/libblas.so libblas.so /usr/local/kml_blas/lib/single/libkblas.so 1000
sudo update-alternatives --install /usr/lib64/libblas.so.3 libblas.so.3 /usr/local/kml_blas/lib/single/libkblas.so 1000
sudo update-alternatives --install /usr/lib64/liblapack.so liblapack.so /usr/local/kml_lapack/lib/libklapackfull.so 1000
sudo update-alternatives --install /usr/lib64/liblapack.so.3 liblapack.so.3 /usr/local/kml_lapack/lib/libklapackfull.so 1000

Perform the verification:

sudo update-alternatives --display libblas.so
sudo update-alternatives --display libblas.so.3
sudo update-alternatives --display liblapack.so
sudo update-alternatives --display liblapack.so.3

Using the Kunpeng Math Library

When running Spark, add the following configurations:

--conf "spark.driver.extraLibraryPath=/opt/netlib" \
--conf "spark.executor.extraLibraryPath=/opt/netlib" \
--conf "spark.driver.extraClassPath=/opt/netlib/*:lib/sophon-ml-acc_2.11-1.2.0.jar:lib/sophon-mlcore_2.11-1.2.0.jar:lib/sophon-ml-kernel-2.11-1.2.0-aarch_64.jar" \
--conf "spark.executor.extraClassPath=/opt/netlib/*" \

Parent topic: Algorithm Performance Tuning Guide