(可选)执行Spark UDF业务
仅在使用OmniOperator算子加速 UDF功能的情况下,才需要执行Spark UDF业务,执行前需先执行Spark引擎业务。当前的OmniOperator算子加速 UDF插件仅支持Simple UDF,若OmniOperator算子加速 UDF不支持,会回退到原生Spark。
执行步骤请参见执行Spark引擎业务,只需在SparkExtension的Spark-SQL命令行窗口启动命令基础上补充如下命令参数即可。
- 在spark.driver.extraClassPath参数后追加/opt/omni-operator/lib/boostkit-omniop-udf-1.3.0-aarch64.jar,追加后结果如下。
1
--conf spark.driver.extraClassPath=/opt/omni-operator/lib/boostkit-omniop-spark-3.1.1-1.3.0-aarch64.jar:/opt/omni-operator/lib/boostkit-omniop-bindings-1.3.0-aarch64.jar:/opt/omni-operator/lib/boostkit-omniop-udf-1.3.0-aarch64.jar:/opt/omni-operator/lib/dependencies/*
- 在spark.executor.extraClassPath参数后追加/opt/omni-operator/lib/boostkit-omniop-udf-1.3.0-aarch64.jar,追加后结果如下。
1
--conf spark.executor.extraClassPath=/opt/omni-operator/lib/boostkit-omniop-spark-3.1.1-1.3.0-aarch64.jar:/opt/omni-operator/lib/boostkit-omniop-bindings-1.3.0-aarch64.jar:/opt/omni-operator/lib/boostkit-omniop-udf-1.3.0-aarch64.jar:/opt/omni-operator/lib/dependencies/*
- 添加--jars /opt/omni-operator/lib/boostkit-omniop-udf-1.3.0-aarch64.jar参数。
1
--jars /opt/omni-operator/lib/boostkit-omniop-udf-1.3.0-aarch64.jar
父主题: 在Spark引擎上的应用