Rate This Document
Findability
Accuracy
Completeness
Readability

Environment Requirements

Before installing OmniAdvisor 2.0, prepare the hardware and software environments to facilitate subsequent installation operations.

Hardware Requirements

Table 1 lists the hardware requirements for each node in the cluster.

Table 1 Hardware requirements

Item

Management/Compute/Storage Node

Processor

Kunpeng 920 5250

Memory size

384 GB (12 x 32 GB)

Memory frequency

2666 MHz

NIC

  • Service network: 10GE
  • Management network: 1GE

Drive

  • System drive: 1 x RAID 0 (1 x 1.2 TB SAS HDD)
  • Data drive: 12 x RAID 0 (12 x 8 TB SATA HDD)

RAID controller card

LSI SAS3508

OS and Software Requirements

Table 2 lists the OS and software requirements for each node in the cluster.

Table 2 OS and software requirements

Item

Version

Description

Management Node (Server)

Compute/Storage Node

OS

  • CentOS 7.9
  • openEuler 22.03 LTS SP1

Install the correct version.

JDK

BiSheng JDK 1.8.0_342

For details about how to install the BiSheng JDK, see BiSheng JDK 8 Installation Guide.

Hadoop

3.2.0

See Hadoop Deployment Guide (CentOS 7.6 & openEuler 20.03).

Spark

3.3.1

See Spark Deployment Guide (CentOS 7.6 & openEuler 20.03).

-

ZooKeeper

3.6.2 or later

See ZooKeeper Deployment Guide (CentOS 7.6 & openEuler 20.03).

-

Python

3.11.4

Install the correct version.

PostgreSQL

12.0 or later

See PostgreSQL Installation Guide.

-

Anaconda

4.12.0 or later

Python package management tool. Determine whether to install it based on your requirements.

Its installation varies with the OS and server architecture. No specific download link can be provided.

-

√: indicates that the item is required on the node.

-: indicates that the item is not required on the node.

Note: To ensure the security of the production environment and reduce the risk of attacks, periodically patch software vulnerabilities.

Table 3 describes the OmniAdvisor 2.0 software packages and how to obtain them.

Table 3 OmniAdvisor 2.0 software packages

Item

Package Name

Release Type

Description

How to Obtain

OmniRuntime package

BoostKit-omniruntime_1.9.0.zip

Closed source

OmniRuntime package. Extract the package to obtain the OmniAdvisor software package BoostKit-omniadvisor_2.0.0.zip.

This feature samples task parameters and recommends optimal configurations through AI iterative tuning, expert rule–based tuning, migration generalization tuning, and operator acceleration, enabling end-to-end parameter tuning for Spark tasks.

Kunpeng community: Contact Huawei technical support.

NOTE:

Before using the software package, read and agree to Kunpeng BoostKit User License Agreement 2.0.

OmniAdvisor2.0 system architecture

BoostKit-omniruntime-omniadvisor-2.0.0.zip

Open source

Performs functions such as Spark task interception, task submission, and database interaction.

Link

OmniAdvisor2.0 tuning algorithm

BoostKit-omniadvisor_core_2.0.0.zip

Closed source

Implements multiple tuning algorithms.

Extract the OmniRuntime package (BoostKit-omniruntime_1.9.0.zip).

Verifying the Software Package Integrity

After downloading a software package from the Kunpeng community, verify the software package to ensure that it is consistent with the original one on the website.

Verify a software package as follows:

  1. Obtain the digital certificate and software.
  2. Click the link to obtain the verification tool and method.
  3. Verify the package integrity by following the procedure described in the OpenPGP Signature Verification Guide obtained from the URL.