Environment Requirements for OmniOperator
Hardware Requirements
Table 1 lists the hardware requirements for each node in the cluster.
Item |
Management Node |
Compute/Storage Node |
|---|---|---|
Processor |
Kunpeng 920 5220 |
Kunpeng 920 5220 |
Memory size |
384 GB (12 x 32 GB) |
384 GB (12 x 32 GB) |
Memory frequency |
2666 MHz |
2666 MHz |
NIC |
|
|
Drive |
|
|
RAID controller card |
LSI SAS3508 |
LSI SAS3508 |
OS and Software Requirements
Table 2 lists the OS and software requirements for each node in the cluster.
Item |
Version |
Description |
Management Node (Server) |
Compute/Storage Node |
|---|---|---|---|---|
OS |
|
- |
√ |
√ |
JDK |
- |
√ |
√ |
|
Hadoop |
3.2.0 |
√ |
√ |
|
Hive |
3.1.0 |
√ |
- |
|
Spark |
3.1.1 |
√ |
√ |
|
Python |
3.10.2 or later |
- |
√ |
√ |
If the preceding third-party software has vulnerabilities, fix the vulnerabilities based on official instructions.
Obtaining Software
Table 3 lists the software packages required for OmniOperator and how to obtain them.
Software Name |
Package Name |
Release Type |
Description |
Download URL |
|---|---|---|---|---|
OmniRuntime package |
BoostKit-omniruntime_1.0.0.zip |
Closed source |
Decompress the OmniRuntime package to obtain the OmniOperator binary package and OmniMV binary package. |
Contact Huawei technical support. |
SparkExtension |
boostkit-omniop-spark-3.1.1-1.2.0-aarch64.zip |
Open source |
Spark extension package for the OmniOperator computing base. |
|
LLVM |
LLVM.zip |
- |
- |
|
jemalloc |
jemalloc.zip |
- |
- |
|
HMPP |
BoostKit-hmpp_1.7.0.zip |
This library depends on the Kunpeng 920 hardware environment. |
The latest HMPP version is 2.0.0. Download HMPP 1.7.0 from historical versions. |
|
ORC |
ORC.zip |
- |
- |
|
Protobuf |
Protobuf.zip |
- |
- |
Verifying Software Package Integrity
After downloading a software package from the Kunpeng community, verify the software package to ensure that it is consistent with the original one on the website.
Verify a software package as follows:
- Obtain the digital certificate and software.
- Obtain the verification tool and method from the following link:
- Verify the package integrity by following the procedure described in the OpenPGP Signature Verification Guide obtained from the URL.