Installing HiveExtension

The OmniOperator feature supports the Hive engine. You need to install the Hive engine on the cluster management node, and configure the HiveExtension dependency of openEuler on the management node and all compute nodes.

1. Install the Hive engine. For details, see Table 2.
2. Download the HiveExtension plugin package and extract it.
   1. Download boostkit-omniop-hive-3.1.0-2.0.0-aarch64.zip from Table 3 and extract it to obtain the boostkit-omniop-spark-3.1.1-2.0.0-aarch64-openeuler.zip (for NEON) and boostkit-omniop-spark-3.1.1-2.0.0-aarch64-openeuler-sve.zip (for SVE) packages. Select either based on whether the model supports NEON or SVE instructions.
   2. Extract boostkit-omniop-spark-3.1.1-2.0.0-aarch64-openeuler.zip (for NEON) or boostkit-omniop-spark-3.1.1-2.0.0-aarch64-openeuler-sve.zip (for SVE) to obtain the boostkit-omniop-hive-3.1.0-2.0.0-aarch64.jar file. Then upload the JAR file to the cluster management node.
3. Install the HiveExtension dependency of openEuler.

   Configure the local yum repository for each OS image and run the following commands to install the dependencies:

   yum install lz4-devel zstd-devel snappy-devel protobuf-c-devel protobuf-lite-devel boost-devel cyrus-sasl-devel jsoncpp-devel openssl-devel libatomic -y

4. Configure HiveExtension.
   1. Create an /opt/omni-operator/ directory on the management and compute nodes as the root directory for installing OmniOperator. Then go to the directory.

      mkdir /opt/omni-operator
      cd /opt/omni-operator

   2. Obtain the Dependency_library_openeuler22.03.zip package from Obtaining Software Packages and upload it to the /opt/omni-operator directory. Then extract the package content applicable to your OS and copy it to the /opt/omni-operator/lib directory.

      1 2	unzip Dependency_library_openeuler22.03.zip cp /opt/omni-operator/Dependency_library_openeuler22.03/* /opt/omni-operator/lib

   3. Copy boostkit-omniop-hive-3.1.0-2.0.0-aarch64.jar to the /opt/omni-operator/lib directory and change the permission on the files in the directory to 550.

      1	chmod -R 550 /opt/omni-operator/*

   4. Create a HiveExtension configuration file directory.

      mkdir -p /opt/omni-operator/hive/conf

   5. Set the following temporary environment variables on the management node:

      export OMNI_CONF=/opt/omni-operator/hive
      export HIVE_AUX_JARS_PATH=/opt/omni-operator/lib
      export LD_LIBRARY_PATH=/opt/omni-operator/lib:$LD_LIBRARY_PATH

      

      • You can customize the /opt/omni-operator/hive directory of the OMNI_CONF variable. If SparkExtension is deployed in the environment as well, this directory must be different from the default /opt/omni-operator directory.
      • Each time a new shell session is started, you need to reset the environment variables in 4.