Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Environment Requirements

Before installing OmniShuffle, prepare the hardware and software environments to facilitate subsequent installation operations.

Hardware requirements

See Table 1 for the hardware requirements for each node in the cluster.

Table 1 Hardware requirements in RSS mode

Item

Management Node

Compute Node

Storage Node

Server

TaiShan server

TaiShan server

TaiShan server

Processor

Kunpeng 920 7260

Kunpeng 920 7260

Kunpeng 920 7260

Memory size

512 GB

512 GB

512 GB

Memory frequency

2666 MHz

2666 MHz

2666 MHz

NIC
  • Service network: 10GE
  • Management network: GE
  • Service network: 20GE
  • Management network: GE
  • Service network: 20GE
  • Management network: GE

Drive
  • System drive: 1 x RAID 0 (1 x 1.2 TB SAS HDD)
  • Data drive: 12 x RAID 0 (12 x 8 TB SATA HDD)
  • System drive: 1 x RAID 0 (1 x 1.2 TB SAS HDD)
  • Data drive: 12 x RAID 0 (12 x 8 TB SATA HDD)
  • System drive: 1 x RAID 0 (1 x 1.2 TB SAS HDD)
  • Data drive: 2 x 4 TB NVMe SSD

RAID controller card

LSI SAS3508

LSI SAS3508

LSI SAS3508

OS and Software Requirements

Table 2 lists the OS and software requirements for each node in the cluster.

Table 2 OS and software requirements

Item

Version

Description

Management Node

Compute/Storage Node

OS

openEuler 22.03 LTS SP1

openEuler 20.03 LTS SP1

-

√

√

NIC driver

Mellanox 5.4-3.6.8.1-LTS

-

√

√

JDK

BiSheng JDK 1.8 (BiSheng JDK 1.8.0_342 recommended)

openEuler 22.03 LTS SP1 is incompatible with BiSheng JDK 1.8.0_262, which needs to be replaced with BiSheng JDK 1.8.0_342. For details about how to install the BiSheng JDK, see BiSheng JDK 8 Installation Guide.

√

√

Hadoop

3.2.0

See Hadoop Deployment Guide (CentOS 7.6 & openEuler 20.03).

√

√

Hive

3.1.0

See Hive Deployment Guide (CentOS 7.6 & openEuler 20.03).

√

-

Spark

3.1.1/3.3.1

See Spark Deployment Guide (CentOS 7.6 & openEuler 20.03).

√

-

ZooKeeper

3.6.2 or later

See ZooKeeper Deployment Guide (CentOS 7.6 & openEuler 20.03).

√

-
  • √: indicates that the item is required on the node.
  • -: indicates that the item is not required on the node.

Obtaining Software

Table 3 OmniShuffle software package

Software Name

Package Name

Release Type

Description

Download URL

OmniRuntime package

BoostKit-omniruntime_1.4.0.zip

Closed source

OmniRuntime package. Decompress the package to obtain the OmniShuffle software package BoostKit-omnishuffle_1.6.0.tar.gz.

Contact Huawei technical support.

NOTE:
NOTE:

Before using the software package, read and agree to Kunpeng BoostKit User License Agreement 2.0.

Verifying the Software Package Integrity

After downloading a software package from the Kunpeng community, verify the software package to ensure that it is consistent with the original one on the website.

Verify the software packages as follows:

  1. Obtain the digital certificate and software.
  2. Obtain the verification tool and method from the following link:
  3. Verify the package integrity by following the procedure described in the OpenPGP Signature Verification Guide obtained from the URL.
Parent topic: Installation Overview