Kunpeng Math Library Acceleration

Installing the Kunpeng Math Library

Some machine learning algorithms (for example, PCA and SVD) use math libraries such as BLAS and LAPACK to implement math operations in algorithms. Kunpeng optimizes such math libraries so that the algorithm performance can be improved. Perform the following steps to install the Kunpeng Math Library on each management and data node in the cluster:

1. Compile the JAR files and SO files related to netlib and jniloader based on the AArch64 architecture and upload them to the /opt/netlib/ directory.

   

   

   • Obtain netlib files at https://github.com/fommil/netlib-java.
   • Obtain jniloader files at https://github.com/ibmsoe/jniloader.
   • Before compilation, ensure that the BLAS math library is not installed.
2. Install netlib dependencies.
   1. Install GCC.
      The version must be 4.8.5 or later. If it has been installed, skip this step.

      1	yum install gcc

      

   2. Install GFortran.

      1	yum install gcc-gfortran

      

   3. Install libblas.

      1	yum install blas-devel

      View the installation result.

      1	ll /usr/lib64/ | grep libblas

      

   4. Install liblapack.

      1	yum install lapack-devel

      View the installation result.

      1	ll /usr/lib64/ | grep liblapack

      

3. Obtain and install the Kunpeng BLAS and LAPACK math libraries, and generate the KML_LAPACK dynamic library with complete functions by following instructions in Kunpeng Math Library Developer Guide.
4. Create a soft link to the installed BLAS and LAPACK math libraries.

   Example:

   1 2 3 4

   sudo update-alternatives --install /usr/lib64/libblas.so libblas.so /usr/local/kml_blas/lib/single/libkblas.so 1000 sudo update-alternatives --install /usr/lib64/libblas.so.3 libblas.so.3 /usr/local/kml_blas/lib/single/libkblas.so 1000 sudo update-alternatives --install /usr/lib64/liblapack.so liblapack.so /usr/local/kml_lapack/lib/libklapack-full.so 1000 sudo update-alternatives --install /usr/lib64/liblapack.so.3 liblapack.so.3 /usr/local/kml_lapack/lib/libklapack-full.so 1000

   Verification:

   1 2 3 4

   sudo update-alternatives --display libblas.so sudo update-alternatives --display libblas.so.3 sudo update-alternatives --display liblapack.so sudo update-alternatives --display liblapack.so.3

   

Using the Kunpeng Math Library

1
2
3
4

--conf "spark.driver.extraLibraryPath=/opt/netlib" \
--conf "spark.executor.extraLibraryPath=/opt/netlib" \
--conf "spark.driver.extraClassPath=/opt/netlib/*:lib/boostkit-ml-acc_2.11-2.1.0-spark2.3.2.jar:lib/boostkit-ml-core_2.11-2.1.0-spark2.3.2.jar:lib/boostkit-ml-kernel_2.11-2.1.0-spark2.3.2-aarch64.jar" \
--conf "spark.executor.extraClassPath=/opt/netlib/*" \