Executing Spark UDF Services

• Operations in this section are required only when OmniOperator UDFs are used. Currently, the OmniOperator UDF plugin must be Simple UDF. If OmniOperator UDF is not available, the plugin will be rolled back to the native Spark.
• For details about how to execute SQL statements, view the execution plans, and compare results, see Executing Spark Services.

For details, see Executing Spark Services. You only need to add the following parameters to the command of starting the Spark-SQL CLI of SparkExtension.

• Add /opt/omni-operator/lib/boostkit-omniop-udf-1.3.0-aarch64.jar to the end of the spark.driver.extraClassPath parameter:

  --conf spark.driver.extraClassPath=/opt/omni-operator/lib/boostkit-omniop-spark-3.1.1-1.3.0-aarch64.jar:/opt/omni-operator/lib/boostkit-omniop-bindings-1.3.0-aarch64.jar:/opt/omni-operator/lib/boostkit-omniop-udf-1.3.0-aarch64.jar:/opt/omni-operator/lib/dependencies/*

• Add /opt/omni-operator/lib/boostkit-omniop-udf-1.3.0-aarch64.jar to the end of the spark.executor.extraClassPath parameter:

  --conf spark.executor.extraClassPath=/opt/omni-operator/lib/boostkit-omniop-spark-3.1.1-1.3.0-aarch64.jar:/opt/omni-operator/lib/boostkit-omniop-bindings-1.3.0-aarch64.jar:/opt/omni-operator/lib/boostkit-omniop-udf-1.3.0-aarch64.jar:/opt/omni-operator/lib/dependencies/*

• Add the --jars /opt/omni-operator/lib/boostkit-omniop-udf-1.3.0-aarch64.jar parameter:

  --jars /opt/omni-operator/lib/boostkit-omniop-udf-1.3.0-aarch64.jar