Rate This Document
Findability
Accuracy
Completeness
Readability

Environment Requirements

Before installing OmniMV, prepare the hardware and software environments to facilitate subsequent installation operations.

Hardware Requirements

Table 1 lists the hardware requirements for each node in the cluster.

Table 1 Hardware requirements

Item

Management/Compute/Storage Node

Processor

Kunpeng 920 5250 processor/New Kunpeng 920 processor model

Memory size

384 GB (12 x 32 GB)

Memory frequency

2666 MHz

Network

  • Service network: 10GE
  • Management network: 1GE

Drive

  • System drive: 1 x RAID 0 (1 x 1.2 TB SAS HDD)
  • Data drive: 12 x RAID 0 (12 x 8 TB SATA HDD)

RAID controller card

LSI SAS3508

OS and Software Requirements

Table 2 lists the OS and software requirements for each node in the cluster.

Table 2 OS and software requirements

Item

Version

Description

Management Node (Server)

Compute/Storage Node

OS

openEuler 22.03 LTS SP1

None

JDK

BiSheng JDK 1.8.0_342

For details about how to install the BiSheng JDK, see BiSheng JDK 8 Installation Guide.

Hadoop

3.2.0

See Hadoop Deployment Guide (CentOS 7.6 & openEuler 20.03).

Spark

3.1.1/3.4.3

See Spark Deployment Guide (CentOS 7.6 & openEuler 20.03).

-

ZooKeeper

3.6.2 or later

See ZooKeeper Deployment Guide (CentOS 7.6 & openEuler 20.03).

-

Python

3.10.2 or later

None

  • √: indicates that the item is required on the node.
  • -: indicates that the item is not required on the node.
  • If the preceding third-party software has vulnerabilities, fix the vulnerabilities based on official instructions.
  • The preceding component versions may be different from those in the Deployment Guide. The Deployment Guide is for reference only.

Obtaining Software

Table 3 describes the OmniMV software packages and how to obtain them.

Table 3 Software required for OmniMV

Software Name

Package Name

Release Type

Description

Download URL

OmniMV software installation package

BoostKit-omnimv_1.2.0.zip

Open source

-

Link

OmniMV plugin JAR package

boostkit-omnimv-spark-3.1.1-1.2.0-aarch64.jar

Open source

Spark extension package for using the OmniMV feature.

Link

boostkit-omnimv-spark-3.4.3-1.2.0-aarch64.jar

Open source

Spark extension package for using the OmniMV feature.

Link

OmniMV log parser JAR package

boostkit-omnimv-logparser-spark-3.1.1-1.2.0-aarch64.jar

Open source

Function package for OmniMV to parse Spark logs.

Link

boostkit-omnimv-logparser-spark-3.4.3-1.2.0-aarch64.jar

Open source

Function package for OmniMV to parse Spark logs.

Link

hive-testbench

hive-testbench-hdp3.zip

Open source

TestBench source package used for generating TPC-DS/TPC-H data.

Link

Verifying the Software Package Integrity

After downloading a software package from the Kunpeng community, verify the software package to ensure that it is consistent with the original one on the website.

Verify a software package as follows:

  1. Obtain the digital certificate and software.
  2. Obtain the verification tool and guide.
  3. Verify the package integrity by following the procedure described in the OpenPGP Signature Verification Guide obtained from the URL.