Installing HiveExtension
The OmniOperator feature supports the Hive engine. You need to install the Hive engine on the cluster management node, and configure the HiveExtension dependency of openEuler on the management node and all compute nodes.
- Install the Hive engine. For details, see Table 2.
- Download the HiveExtension plugin package and decompress it.
Download boostkit-omniop-hive-3.1.0-1.7.0-aarch64.zip from Table 3, decompress it to obtain boostkit-omniop-hive-3.1.0-1.7.0-aarch64.jar, and upload the JAR file to the management node.
- Install the HiveExtension dependency of openEuler.
Configure the local yum source for each OS image and run the following commands to install the dependencies:
yum install lz4-devel zstd-devel snappy-devel protobuf-c-devel protobuf-lite-devel boost-devel cyrus-sasl-devel jsoncpp-devel openssl-devel libatomic -y
- Configure HiveExtension.
- Obtain the Dependency_library packages (Dependency_library.zip and Dependency_library.z01) from Obtaining Software Packages and upload them to the /opt/omni-operator directory. Then decompress the subpackages applicable to your OS and copy them to the /opt/omni-operator/lib directory.
1 2 3 4 5
cd /opt/omni-operator rm -rf Dependency_library_*.zip zip -F Dependency_library.zip --out Dependency_library_complete.zip unzip Dependency_library_complete.zip cp /opt/omni-operator/Dependency_library_openeuler/* /opt/omni-operator/lib
- Copy boostkit-omniop-hive-3.1.0-1.7.0-aarch64.jar to the /opt/omni-operator/lib directory and change the permission on the files in the directory to 550.
chmod -R 550 /opt/omni-operator/lib/*
- Create a HiveExtension configuration file directory.
mkdir -p /opt/omni-operator/hive/conf
- Set the following temporary environment variables on the management node:
export OMNI_CONF=/opt/omni-operator/hive export HIVE_AUX_JARS_PATH=/opt/omni-operator/lib export LD_LIBRARY_PATH=/opt/omni-operator/lib:$LD_LIBRARY_PATH
- Obtain the Dependency_library packages (Dependency_library.zip and Dependency_library.z01) from Obtaining Software Packages and upload them to the /opt/omni-operator directory. Then decompress the subpackages applicable to your OS and copy them to the /opt/omni-operator/lib directory.
- You can customize the /opt/omni-operator/hive directory of the OMNI_CONF variable. If SparkExtension is deployed in the environment as well, this directory must be different from the default /opt/omni-operator directory.
- Each time a new shell session is started, you need to reset the environment variables in 4.
Parent topic: Using on Hive