Prerequisites
- Operations in this section are required only when OmniOperator user-defined functions (UDFs) are used. The OmniOperator UDF plugin must be Simple UDF, which is used to execute UDFs compiled based on the Hive UDF framework.
- OmniOperator UDFs support expression row-by-row processing and batch processing. The two methods can be switched using the configuration file.
Configuring the OmniOperator UDF plugin requires related JAR packages and configuration files, including udf.zip, conf.zip, and udf.properties. The udf.zip package contains all UDF class files, the conf.zip file includes the configuration file on which OmniOperator UDFs depend, and udf.properties is the OmniOperator UDF configuration file. The following uses the udfName1 and udfName2 functions as an example to describe the content format of the udf.properties file.
- Upload the previously mentioned packages to the /opt/omni-operator/hive-udf directory on management and compute nodes.
- Select all nodes and click Configure Service.
- Select Custom and click Next.
- Select File Transfer and click Add a Task.
- Set parameters for tasks 1 to 3 and click Save.
Table 1 Task 1 parameters Parameter
Description
Task Name
Copy the udf.zip software package.
Task Description
Copy the udf.zip software package to all nodes.
Transfer Direction
From Local to Remote
Local Path
Directory where the local installation package is stored
Remote Path
/opt/omni-operator/hive-udf
Table 2 Task 2 parameters Parameter
Description
Task Name
Copy the conf.zip software package.
Task Description
Copy the conf.zip software package to all nodes.
Transfer Direction
From Local to Remote
Local Path
Directory where the local installation package is stored
Remote Path
/opt/omni-operator/hive-udf
Table 3 Task 3 parameters Parameter
Description
Task Name
Copy the udf.properties file.
Task Description
Copy the udf.properties software package to all nodes.
Transfer Direction
From Local to Remote
Local Path
Directory where the local installation package is stored
Remote Path
/opt/omni-operator/hive-udf
- Select all nodes and click Configure Service.
- Decompress udf.zip and conf.zip and set environment variables.
- Select Command Execution and click Add a Task.
- Set parameters for tasks 4 and 5 and click Save.
Table 4 Task 4 parameters Parameter
Description
Task Name
Decompress the udf.zip and conf.zip software packages.
Task Description
Decompress the udf.zip and conf.zip software packages.
Run Directory
/opt/omni-operator/hive-udf
Run Command
unzip -o udf.zip;rm -f udf.zip;unzip -o conf.zip;rm -f conf.zip;
Table 5 Task 5 parameters Parameter
Description
Task Name
Configure UDF environment variables for OmniOperator.
Task Description
Set OmniOperator UDF environment variables on all nodes.
Run Directory
/opt/omni-operator
Run Command
sed -i '$a export LD_LIBRARY_PATH=${JAVA_HOME}/jre/lib/aarch64/server/:$LD_LIBRARY_PATH' ~/.bashrc;source ~/.bashrc;sed -i '$a export OMNI_HOME=/opt/omni-operator' ~/.bashrc;source ~/.bashrc;
- Select Command Execution and click Add a Task.
- Export the service flow and save it to your local PC. You can import service flows to quickly install OmniOperator.
- In the upper right corner of the displayed page, click Export.
The window for selecting a path for saving the template is displayed.
- Name the template OmniOperator UDF Plugin Deployment and Configuration Workflow and click Save.
- In the upper right corner of the displayed page, click Export.
- Execute the service flow.
Click Finish in the lower right corner and select Run Service. The procedure is complete when the execution is successful.
You can customize the example name of the udf.zip package based on your service requirements.