Documentation
Resource
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Batch Installing Tools

The DevKit Pipeline is a tool for batch installation. You can install tools related to gated check-in, compilation, testing, and virus scanning to worker nodes as required.

On the master node, you can use the DevKit Pipeline to batch install all tools to worker nodes.

Procedure

  1. Log in to the master node as the root user and obtain the software package devkit-pipeline-v1.1.tar.gz provided in Obtaining Software Packages. Upload the package to the Kunpeng server. The following uses the installation directory /home/pipeline as an example.
  2. Extract the software package.
    cd /home/pipeline
tar -zxvf devkit-pipeline-v1.1.tar.gz

    The extracted linux folder includes the following files:

    • deploy_tool: batch installation tool
    • download_tool: one-click download tool
    • machine.yaml: configuration file
    • script_generator generates Jenkins and GitLab pipeline running templates.

    For details about the tools, see download_tool, deploy_tool, and script_generator.

  3. Go to the linux folder and edit the machine.yaml file. In this file, configure the IP addresses of the worker nodes where the components are installed and other information.
    cd devkit-pipeline-v1.1/linux/
vim machine.yaml

    The source repository provides the following machine.yaml configuration file template:

    user: root
pkey: /root/.ssh/id_rsa
scanner:
  - 192.168.0.1
  - 192.168.0.2
  - 192.168.0.3
c_builder_gcc:
  - 192.168.0.2
c_builder_bisheng_compiler:
  - 192.168.0.2
java_builder_jdk8:
  - 192.168.0.1
java_builder_jdk17:
  - 192.168.0.1
compatibility:
  - 192.168.0.1
tester:
  - 192.168.0.3
devkit:
  - 192.168.0.3
clamav:
  - 192.168.0.3
    • user: user name for password-free access from the current node to other nodes.
    • pkey: private key path corresponding to the public key for password-free access.
    • scanner: DevKit CLI scanning node.
    • c_builder_gcc: GCC for openEuler build node.
    • c_builder_bisheng_compiler: BiSheng compiler build node.
    • java_builder_jdk8: BiSheng JDK 8 build node.
    • java_builder_jdk17: BiSheng JDK 17 build node.

      Select either BiSheng JDK 8 or BiSheng JDK 17. Do not install both in the same environment. If both are installed, only one will take effect.

    • compatibility: compatibility test execution node.
    • devkit: DevKit Web worker node.
    • clamav: ClamAV virus scan execution node.
    • tester: Java performance test execution node.

    The following are role names: scanner, java_builder_jdk8, java_builder_jdk17, c_builder_gcc, c_builder_bisheng_compiler, compatibility, devkit, clamav and tester. You need to enter the corresponding IP addresses in the YAML list format based on the template. Enter the IP addresses of the servers based on the roles to be installed. You can remove roles that are not required. All roles can be deployed on the same server.

  4. Configure password-free access from the master node to each worker node. This operation must be performed for each worker node and user.
    1. Use the RSA protocol to generate a public-private key pair.
      ssh-keygen -t rsa

      Generate a public-private key pair: ssh-keygen [-b bits] [-t rsa] [-f output_keyfile]

      • -b specifies the byte length.
      • -t specifies the type of the key to be generated.
      • -f specifies the file name for the public-private key pair.

      After the command is executed, a message prompts you to enter a password for generating the public-private key pair. If you enter the password, you need to manually input it each time the batch installation tool runs. Alternatively, you can add a password parameter to the machine.yaml configuration file and set it to the same password.

    2. Configure password-free access from the master node to each worker node.
      ssh-copy-id -i /root/.ssh/id_rsa.pub USER@REMOTE_HOST
      • -i specifies the public key path for configuring password-free login.
      • USER: user name of a worker node
      • REMOTE_HOST: IP address of the worker node

      After the command is executed, a message prompts you to enter the password of the worker node. The public key is then copied to the authorized_keys file of the worker node to implement password-free login. If the IP address is set to that of the local host or you want to implement local-to-local installation, you need to configure password-free login for local-to-local access.

  5. (Optional) Log in to the worker node as the root user and configure the sudo permission and password-free sudo on the worker node. This is required only when a common user installs the compatibility test execution node, DevKit CLI scanning node, ClamAV virus scanning execution node, or DevKit Web worker node.

    Run the visudo command to modify the /etc/sudoers.

    visudo -f /etc/sudoers
    1. To configure the sudo permission as a common user (USER), add a directive after "root ALL=(ALL) ALL" in the /etc/sudoers as follows:
      root    ALL=(ALL)       ALL 
USER    ALL=(ALL)       NOPASSWD: ALL
    2. To configure password-free sudo for USER, add a directive after "# %wheel ALL=(ALL) NOPASSWD: ALL" in the /etc/sudoers file as follows:
      # %wheel        ALL=(ALL)       NOPASSWD: ALL 
USER            ALL=(ALL)       NOPASSWD: ALL
  6. (Optional) Use the root user to log in to the worker node where A-FOT is installed and use yum to install the perf utility. A-FOT is the c_builder_gcc role.
    yum install perf -y

    If no network is available, configure a local repository and then run the yum install perf -y command to install perf.

    For details about how to configure a local repository, see Configuring the OS Yum, APT, or Zypper Source.

  7. (Optional) Download the tools.

    If your server does not have Internet access, you can download the tools in advance and then use deploy_tool to batch install the tools. If your server has Internet access, skip this step. Directly use deploy_tool to batch install the tools.

    Select either the Linux download tool or the Windows download tool as needed.

    • Linux: download_tool

      download_tool is used for one-click download. If the node where the batch installation tool is installed does not have Internet access, you can use download_tool to download the required installation packages on a host with Internet access and then upload the packages to this node so that the batch installation tool can load and use the packages in the default path.

      • Use download_tool to download the tools based on the valid roles configured in the machine.yaml file in 3.
        ./download_tool -f ./machine.yaml

        After the download is complete, save all downloaded tool installation packages to the devkitdependencies folder and upload the packages to the node where the batch installation tool is located.

      • Download the ISO image using the one-click download tool.
        ./download_tool -iso auto

        download_tool command parameters:

        • -f specifies the path to the YAML file.
        • -iso specifies the ISO version to be downloaded. If you enter auto, the tool automatically detects the OS version and downloads the corresponding ISO file.
    • Windows: download_tool-for-windows.exe
      1. Obtain the download_tool-for-windows.exe tool. For details, see Obtaining Software Packages.
      2. Create a machine.yaml file in the path where download_tool-for-windows.exe is stored and fill in the file. For details, see 3.
      3. Open the cmd window in the path where download_tool-for-windows.exe is stored and download the component packages. The tool automatically compresses the downloaded packages into a devkitdependencies.tar.gz package.
        download_tool-for-windows.exe
      4. Upload devkitdependencies.tar.gz to the linux folder.
  8. Install the tools.
    deploy_tool can be used to install GCC for openEuler, BiSheng Compiler, BiSheng JDK 8, BiSheng JDK 17, Compatibility Testing tool, DevKit Web tool, DevKit CLI tool, Java Performance Testing tool, and ClamAV based on the valid roles configured in the machine.yaml file.
    ./deploy_tool -f ./machine.yaml
    • -f specifies the path to the YAML file.
    • The batch installation tool uses standard output as the log output destination, without generating a separate log file.
    • Before using the batch installation tool, install the tar command on the target nodes to install the tools.
    • Do not deploy BiSheng JDK 8 and BiSheng JDK 17 in the same environment. If they are deployed in the same environment, only one JDK takes effect.
    • Before using the Compatibility Testing tool, configure parameters as described in Use Cases.
  9. The Java Performance Testing tool relies on Apache JMeter. Verify that the Java program to be profiled is running on a worker node on which the jcmd command is available. For details, see Installing Apache JMeter and Verifying jcmd Availability on the Machine for Java Program Profiling.
  10. After the installation is complete, run the su command on each worker node to reload environment variables. Replace USER with the tool installation user.
    su - USER
  11. (Optional) Perform non-invasive dependency switching.

    If certain applications or commands need to use the environment variables of the BiSheng Compiler installed by the batch installation tool, run the following command to switch the dependencies.

    source ${HOME}/.local/wrap-bin/devkit_NonInvasiveSwitching.sh

    When the BiSheng Compiler is installed using the batch installation tool, script files such as wrap-bin/devkit_NonInvasiveSwitching.sh are automatically generated in the ~/.local folder.

    You can run the gcc -v command to view environment information before and after the switching. The command output changes from gcc to clang.

Verifying the Installation

After the installation is complete, you can view the installation result on the worker nodes where the tools are installed.

Open the bashrc file and view environment variables to determine whether the tool environment is successfully installed.

cat ~/.bashrc
  • GCC for openEuler

    View the bashrc file. If the following four commands are added, the GCC for openEuler environment is successfully set.

    export GCC_HOME=${HOME}/.local/gcc-10.3.1-2023.12-aarch64-linux
export PATH=${GCC_HOME}/bin:${PATH}
export INCLUDE=${GCC_HOME}/include:${INCLUDE}
export LD_LIBRARY_PATH=${GCC_HOME}/lib64:${LD_LIBRARY_PATH}

    Query the GCC version used in the current environment.

    gcc --version

    If the following information is displayed, GCC for openEuler is successfully installed.

    gcc (gcc for openEuler 2.3.6) 10.3.1
  • BiSheng Compiler

    View the bashrc file. If the following three commands are added, the BiSheng Compiler environment is successfully set.

    export BISHENG_COMPILER_HOME=${HOME}/.local/BiShengCompiler-3.2.0-aarch64-linux
export PATH=${BISHENG_COMPILER_HOME}/bin:${PATH}
export LD_LIBRARY_PATH=${BISHENG_COMPILER_HOME}/lib:${BISHENG_COMPILER_HOME}/lib/aarch64-unknown-linux-gnu:${LD_LIBRARY_PATH}

    Query the Clang version used in the current environment.

    clang --version

    If the following information is displayed, the BiSheng Compiler is successfully installed.

    BiSheng Enterprise 3.2.0.B008 clang version 15.0.4 (1e11b36754d8)
  • BiSheng JDK 8

    View the bashrc file. If the following four commands are added, the BiSheng JDK 8 environment is successfully set.

    export JAVA_HOME=${HOME}/.local/bisheng-jdk1.8.0_402
export PATH=${JAVA_HOME}/bin:${PATH}
export CLASSPATH=.:${JAVA_HOME}/lib/dt.jar:${JAVA_HOME}/lib/tools.jar
export JRE_HOME=${JAVA_HOME}/jre

    Query the Java version used in the current environment.

    java -version

    If the following information is displayed, BiSheng JDK 8 is successfully installed.

    openjdk version "1.8.0_402"
OpenJDK Runtime Environment BiSheng (build 1.8.0_402-b11)
  • BiSheng JDK 17

    View the bashrc file. If the following two commands are added, the BiSheng JDK 17 environment is successfully set.

    export JAVA_HOME=${HOME}/.local/bisheng-jdk-17.0.10
export PATH=${JAVA_HOME}/bin:${PATH}

    Query the Java version used in the current environment.

    java -version

    If the following information is displayed, BiSheng JDK 17 is successfully installed.

    openjdk version "17.0.10" 2024-01-16
OpenJDK Runtime Environment BiSheng (build 17.0.10+11)
  • A-FOT

    View the bashrc file. If the following two commands are added, the A-FOT environment is successfully set.

    export A_FOT_HOME=${HOME}/.local/a-fot
export PATH=${A_FOT_HOME}:${PATH}

    Query the location of A-FOT.

    which a-fot

    If the following information is displayed, A-FOT is successfully installed.

    /USER/.local/a-fot/a-fot

    After re-logging in to the server, you can optimize the application by running a-fot --config_file a-fot.ini after configuring the configuration file (or by flexibly configuring the command as described in A-FOT Commands).

  • Compatibility testing tool

    The batch installation tool extracts the compatibility_testing.tar.gz package in the ${HOME} directory. Run the following command to check whether necessary files exist:

    ls ${HOME}/.local/compatibility_testing/bin/compatibility_test
  • DevKit Web

    The batch installation tool installs the DevKit Web backend in the /opt directory, installs all plugins, and initializes the administrator password. The initial password is devkit123. Open the browser on the local PC, access https://Installation_too_IP_address:Port (for example, https://x.x.x.x:8086), and enter the administrator user name (devadmin) and password (devkit123).

  • DevKit CLI

    View the bashrc file. If the following command is added, the DevKit CLI environment is successfully set.

    export PATH=${HOME}/.local/DevKit-CLI-24.0.T50-Linux-Kunpeng:${PATH}

    Query the location of the DevKit CLI.

    which devkit

    If the following information is displayed, DevKit CLI is successfully installed.

    /USER/.local/DevKit-CLI-24.0.T50-Linux-Kunpeng/devkit
  • Java Performance Testing tool

    The batch installation tool extracts the devkit_tester.tar.gz package in the ${HOME} directory. Run the following command to check whether necessary files exist:

    ls ${HOME}/.local/devkit_tester
  • ClamAV

    Query the ClamAV tool version used in the current environment.

    clamscan -V

    If the following information is displayed, ClamAV is successfully installed.

    ClamAV 0.103.9/26818/Mon Feb 20 16:21:31 2023
This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details