Deploying the Spark UDF Plugin
- Operations in this section are required only when OmniOperator UDFs are used.
- The Spark engine has been deployed by following instructions provided in Prerequisites to Deploying the UDF Plugin Batch Processing.
- Add the following environment variable to the ~/.bashrc file on all nodes:
export OMNI_HOME=/opt/omni-operator
- Place the JAR package on which UDFs depend to the /user/hive-udf directory of HDFS.
- The /user/hive-udf directory can be customized.
- The JAR package on which UDF depends needs to be provided by yourself.
- Register Hive UDF on the management nodes of the cluster.
Parent topic: Using OmniOperator on the Spark Engine