算法库更新版本后,可以按照以下流程进行版本升级。
1 2 3 4 | rm -f /home/test/boostkit/lib/boostkit-* cp /opt/Spark-graph-algo-lib-1.3.0-spark2.3.2/graph-accelerator/target/boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar /home/test/boostkit/lib cp /opt/Spark-graph-algo-lib-1.3.0-spark2.3.2/graph-accelerator/target/boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar /home/test/boostkit/lib cp /opt/boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar /home/test/boostkit/lib |
1 2 3 4 5 6 7 8 9 10 11 12 | #!/bin/bash spark-submit \ --class com.bigdata.graph.WCEMain \ --master yarn \ --deploy-mode client \ --driver-cores 36 \ --driver-memory 50g \ --jars "lib/fastutil-8.3.1.jar,lib/boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar,lib/boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar,lib/boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar" \ --driver-class-path "lib/graph-test.jar:lib/fastutil-8.3.1.jar:lib/snakeyaml-1.17.jar:lib/boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar:lib/boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar:lib/boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar" \ --conf "spark.executor.extraClassPath=fastutil-8.3.1.jar:boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar:boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar:boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar" \ ./graph-test.jar |
1 2 3 4 5 6 7 8 9 10 11 12 13 | #!/bin/bash spark-submit \ --class com.bigdata.graph.WCEMain \ --master yarn \ --deploy-mode cluster \ --driver-cores 36 \ --driver-memory 50g \ --jars "lib/fastutil-8.3.1.jar,lib/boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar,lib/boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar,lib/boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar" \ --driver-class-path "graph-test.jar:fastutil-8.3.1.jar:snakeyaml-1.17.jar:boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar:boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar:boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar" \ --conf "spark.yarn.cluster.driver.extraClassPath=graph-test.jar:snakeyaml-1.17.jar:boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar:boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar:boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar" \ --conf "spark.executor.extraClassPath=fastutil-8.3.1.jar:boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar:boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar:boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar" \ ./graph-test.jar |
脚本中的语句含义如表2所示。