(Optional) Installing the UDF Plugin

Prerequisites

Operations in this section are required only when OmniOperator UDFs are used. The OmniOperator UDF plugin supports only Hive simple UDFs, which are used to execute UDFs written based on the Hive UDF framework. HiveSimpleUDF is one of the UDF types in Hive.

Configuring the OmniOperator UDF plugin requires related JAR packages and configuration files, including udf.zip, conf.zip, and udf.properties. The udf.zip package contains all UDF class files, the conf.zip file includes the configuration file on which OmniOperator UDFs depend, and udf.properties is the OmniOperator UDF configuration file. The following uses the udfName1 and udfName2 functions as an example to describe the content format of the udf.properties file.

1
2

udfName1 com.huawei.udf.UdfName1
udfName2 com.huawei.udf.UdfName2

Installing the UDF Plugin Row-by-Row Processing

1. Create an /opt/omni-operator/hive-udf directory on the management and compute nodes.

   1	mkdir /opt/omni-operator/hive-udf

2. Upload the udf.zip and conf.zip packages to the /opt/omni-operator/hive-udf directory on management and compute nodes.
3. Decompress the related packages.

   1 2 3 4 5

   cd /opt/omni-operator/hive-udf unzip udf.zip rm -f udf.zip unzip conf.zip rm -f conf.zip

4. In the /opt/omni-operator/conf/omni.conf file, update the configuration.
   1. Open /opt/omni-operator/conf/omni.conf.

      1	vi /opt/omni-operator/conf/omni.conf

   2. Press i to enter the insert mode and add the following UDF configuration.

      1 2 3 4 5 6 7

      # <----UDF properties----> # false indicates expression row-by-row processing and true indicates expression bath processing. enableBatchExprEvaluate=false # UDF trustlist file path hiveUdfPropertyFilePath=./hive-udf/udf.properties # Directory of the Hive UDF JAR package hiveUdfDir=./hive-udf/udf

      

      The directory must start with a period (.). When OmniOperator is running, it reads the value of the OMNI_HOME environment variable and uses this value to replace the period (.).

   3. Press Esc, type :wq!, and press Enter to save the file and exit.
5. Use vi to open the ~/.bashrc file and add LD_LIBRARY_PATH to the file to update environment variables.
   1. Open the ~/.bashrc file.

      1	vi ~/.bashrc

   2. Press i to enter the insert mode and add LD_LIBRARY_PATH to update environment variables.

      1	export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${JAVA_HOME}/jre/lib/aarch64/server

   3. Press Esc, type :wq!, and press Enter to save the file and exit.
   4. Update the environment variables.

      1	source ~/.bashrc

You can customize the example names of the udf.zip and conf.zip packages based on your service requirements.

Installing the UDF Plugin Batch Processing

After installing the row-by-row processing on the management node, perform the following steps to update the configuration in the omni.conf file:

1. Open /opt/omni-operator/conf/omni.conf.

   1	vi /opt/omni-operator/conf/omni.conf

2. Press i to enter the insert mode and update the UDF row-by-row and batch processing configurations.

   1	enableBatchExprEvaluate=true

3. Press Esc, type :wq!, and press Enter to save the file and exit.