文档
我要评分
获取效率
正确性
完整性
易理解
提交关闭

OmniData

Hardware Requirements

Table 1 lists the hardware requirements for each node in the cluster.

Table 1 Hardware requirements

Item

Description

Processor

Kunpeng 920 5250

Memory size

384 GB (12 x 32 GB)

Memory frequency

2666 MHz

NIC
  • Ceph environment: 25GE for the service network and GE for the management network
  • HDFS environment: 10GE for the service network and GE for the management network

Drive
  • System drive: 1 x RAID 0 (1 x 1.2 TB SAS HDD)
  • Data drives (on the management node): 12 x RAID 0 (12 x 4 TB SATA HDD)
  • Data drives (compute node):
    • Ceph environment: 1 x 3.2 TB NVMe
    • HDFS environment: 12 x RAID 0 (12 x 4 TB SATA HDD)
  • Data drives (storage node):
    • Ceph environment: 12 x RAID 0 (12 x 4 TB SATA HDD) 1 x 3.2 TB NVMe
    • HDFS environment: 12 x RAID 0 (12 x 4 TB SATA HDD)

RAID controller card

LSI SAS3508

OS and Software Requirements

Table 2 lists the OS and software requirements for each node in the cluster.

Table 2 OS and software requirements

Item

Version

Description

Management Node

(Server)

Compute Node

(Agent)

Storage Node

(HDFS or Ceph)

OS
  • CentOS 7.6
  • openEuler 20.03 LTS SP1
  • openEuler 22.03 LTS SP1

-

√

√

√

JDK

BiSheng JDK 1.8 (BiSheng JDK 1.8.0_342 recommended)

openEuler 22.03 LTS SP1 is incompatible with BiSheng JDK 1.8.0_262, which needs to be replaced with BiSheng JDK 1.8.0_342.

√

√

√

Hadoop

3.2.0

See Hadoop Deployment Guide (CentOS 7.6 & openEuler 20.03).

√

√

√

ZooKeeper

3.6.2

See ZooKeeper Deployment Guide (CentOS 7.6 & openEuler 20.03).

-

√

-

Hive

3.1.0

See Hive Deployment Guide (CentOS 7.6 & openEuler 20.03).

√

-

-

Ceph

14.2.8

-

-

-

√

Spark

3.1.1

See Spark Deployment Guide (CentOS 7.6 & openEuler 20.03).

√

-

-

OpenSSL

1.1.1 or later

Link

-

√

√

Python

3.10.2 or later

-

-

√

√

Obtaining Software

Table 3 describes the OmniData software packages and how to obtain them.

Table 3 OmniData software packages

Software Name

Package Name

Release Type

Description

Download URL

OmniRuntime package

BoostKit-omniruntime_1.0.0.zip

Closed source

OmniRuntime package. Decompress the package to obtain the OmniData software package BoostKit-omnidata_1.5.0.zip.

Contact Huawei technical support.

OmniData plugin for Spark

boostkit-omnidata-spark-sql_2.12-3.1.1-1.5.0-aarch64.zip

Open source

Spark extension package for using the OmniData function.

Link

Third-party dependency package for the OmniData Server

boostkit-omnidata-server-1.5.0-aarch64-lib.zip

Open source

Third-party dependency package for OmniData.

Obtain this package by referring to README.md.

Hive UDF plugin

boostkit-omnidata-hive-udf-loader-1.6.1-1.4.0-aarch64.zip

Open source

Hive UDF plugin package. The Hive UDF plugin is used to execute UDFs compiled based on the Hive UDF framework.

Obtain boostkit-omnidata-hive-udf-loader-1.6.1.zip and rename it to boostkit-omnidata-hive-udf-loader-1.6.1-1.4.0-aarch64.zip. For details, see README.md.

HAF software package

BoostKit-haf_1.3.0.zip

Closed source

HAF software package.

Contact Huawei technical support.

Dependencies required to access Ceph

hdfs-ceph-3.2.0.jar

Open source

Dependency package required by Hadoop and OmniData to access Ceph.

Link

librgw_jni.so

Open source

Binary file required by Hadoop and OmniData to access Ceph.

Link

Verifying Software Package Integrity

After downloading a software package from the Kunpeng community, verify the software package to ensure that it is consistent with the original one on the website.

Verify a software package as follows:

  1. Obtain the digital certificate and software.
  2. Obtain the verification tool and method from the following link:
  3. Verify the package integrity by following the procedure described in the OpenPGP Signature Verification Guide obtained from the URL.
Parent topic: Environment Requirements