鲲鹏社区首页
中文
注册
我要评分
文档获取效率
文档正确性
内容完整性
文档易理解
在线提单
论坛求助

软件升级

算法库更新版本后,可以按照以下流程进行版本升级。

  1. 按照获取代码的说明获取最新的软件包。
  2. 以大数据组件的授权用户登录服务器,将原来客户端的“/home/test/boostkit/lib/”目录下的算法库软件包,替换成最新版本。
    1
    2
    3
    4
    rm -f /home/test/boostkit/lib/boostkit-*
    cp /opt/Spark-graph-algo-lib-1.3.0-spark2.3.2/graph-accelerator/target/boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar /home/test/boostkit/lib
    cp /opt/Spark-graph-algo-lib-1.3.0-spark2.3.2/graph-accelerator/target/boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar /home/test/boostkit/lib
    cp /opt/boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar /home/test/boostkit/lib
    
  3. 将提交任务shell脚本中算法库软件包名替换成最新版本的包名,使用yarn-client模式启动Spark作业,替换后的shell脚本示例如下。
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    #!/bin/bash
    
    spark-submit \
    --class com.bigdata.graph.WCEMain \
    --master yarn \
    --deploy-mode client \
    --driver-cores 36 \
    --driver-memory 50g \
    --jars "lib/fastutil-8.3.1.jar,lib/boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar,lib/boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar,lib/boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar" \
    --driver-class-path "lib/graph-test.jar:lib/fastutil-8.3.1.jar:lib/snakeyaml-1.17.jar:lib/boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar:lib/boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar:lib/boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar" \
    --conf "spark.executor.extraClassPath=fastutil-8.3.1.jar:boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar:boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar:boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar" \
    ./graph-test.jar
    
    将提交任务shell脚本中算法库软件包名替换成最新版本的包名,使用yarn-cluster模式启动Spark作业,替换后的shell脚本示例如下。
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    #!/bin/bash
    
    spark-submit \
    --class com.bigdata.graph.WCEMain \
    --master yarn \
    --deploy-mode cluster \
    --driver-cores 36 \
    --driver-memory 50g \
    --jars "lib/fastutil-8.3.1.jar,lib/boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar,lib/boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar,lib/boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar" \
    --driver-class-path "graph-test.jar:fastutil-8.3.1.jar:snakeyaml-1.17.jar:boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar:boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar:boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar" \
    --conf "spark.yarn.cluster.driver.extraClassPath=graph-test.jar:snakeyaml-1.17.jar:boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar:boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar:boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar" \
    --conf "spark.executor.extraClassPath=fastutil-8.3.1.jar:boostkit-graph-acc_2.11-1.3.0-spark2.3.2.jar:boostkit-graph-kernel-2.11-1.3.0-spark2.3.2-aarch64.jar:boostkit-graph-core_2.11-1.3.0-spark2.3.2.jar" \
    ./graph-test.jar 
    

    脚本中的语句含义如表2所示。