Deploying Tez

The installation package used for deploying Tez can be obtained from Compiling Tez (recommended) or downloaded from the Huawei mirrored repository. In this section, the installation package is obtained from Compiling Tez.

1. Log in to the server1 node, copy the compiled installation package to the /usr/local directory, and rename it apache-tez-0.10.0-bin.tar.gz.

   1	cp tez-0.10.0.tar.gz /usr/local/apache-tez-0.10.0-bin.tar.gzcd /usr/local

2. Create an /apache-tez-0.10.0-bin directory and decompress the installation package to the /apache-tez-0.10.0-bin directory.

   1	mkdir apache-tez-0.10.0-bintar -zxvf apache-tez-0.10.0-bin.tar.gz -C apache-tez-0.10.0-bin

3. Go to the /usr/local/apache-tez-0.10.0-bin/lib directory.

   1	cd /usr/local/apache-tez-0.10.0-bin/lib

4. The snappy-java-1.0.5.jar package compiled from Tez is incompatible with the Kunpeng server. Check for snappy-java-1.0.5.jar.
   • If it is not found, download the version that is compatible with the Kunpeng server.

     1	llwget https://repo.huaweicloud.com/kunpeng/maven/org/xerial/snappy/snappy-java/1.0.5/snappy-java-1.0.5.jar

   • If it is found, delete it and download the version that is compatible with the Kunpeng server.

     1	llrm -rf snappy-java-1.0.5.jarwget https://repo.huaweicloud.com/kunpeng/maven/org/xerial/snappy/snappy-java/1.0.5/snappy-java-1.0.5.jar

5. The leveldbjni-all-1.8.jar package compiled from Tez is incompatible with the Kunpeng server. Check for leveldbjni-all-1.8.jar.
   • If it is not found, download the version that is compatible with the Kunpeng server.

     1	llwget https://mirror.iscas.ac.cn/kunpeng/maven/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar

   • If it is found, delete it and download the version that is compatible with the Kunpeng server.

     1	llrm -rf leveldbjni-all-1.8.jarwget https://mirror.iscas.ac.cn/kunpeng/maven/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.jar

6. Return to the /usr/local directory and create a soft link.

   1	cd /usr/localln -s apache-tez-0.10.0-bin /usr/local/tez

7. Create a tez-site.xml configuration file.
   1. Create a /usr/local/tez/conf directory and go to the directory.

      1	mkdir -p /usr/local/tez/confcd /usr/local/tez/conf

   2. Create a tez-site.xml file.

      1	vi tez-site.xml

   3. Press i to enter the insert mode and add the following information to the file:

      1

      <?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><configuration><property> <name>tez.lib.uris</name> <value>${fs.defaultFS}/apps/tez/tez.tar.gz</value></property><property> <name>tez.use.cluster.hadoop-libs</name> <value>false</value></property><property> <name>tez.history.logging.service.class</name> <value>org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService</value></property><property> <name>tez.allow.disabled.timeline-domains</name> <value>true</value></property><property> <name>tez.tez-ui.history-url.base</name> <value>http://server1:8822/tez-ui</value></property></configuration>

   4. Press Esc, type :wq!, and press Enter to save the file and exit.
8. Upload the Tez installation package to HDFS.
   1. Go to the /usr/local/tez directory and compress the current files and directories to the tez.tar.gz file.

      1	cd /usr/local/teztar -czvf ../tez.tar.gz .

   2. Go back to the upper-level directory.

      1

      cd ..

   3. Create a directory on HDFS.

      1	hdfs dfs -mkdir -p /apps/tez/

   4. Delete the old tez.tar.gz file to prevent residual files on HDFS.

      1	hdfs dfs -rmr /apps/tez/tez.tar.gz

   5. Upload the compressed package to the /apps/tez directory.

      1	hdfs dfs -put ./tez.tar.gz /apps/tez/rm -rf ./tez.tar.gz

9. Modify the Hadoop configuration file hadoop-env.sh and synchronize the operation to all nodes.
   1. The Hadoop configuration directory is /usr/local/hadoop/etc/hadoop. Go to this directory.

      1	cd /usr/local/hadoop/etc/hadoop

   2. Open hadoop-env.sh.

      1	vi hadoop-env.sh

   3. Press i to enter the insert mode and add the following content to the end of the file:

      1	export TEZ_CONF_DIR=/usr/local/tez/conf/tez-site.xmlexport TEZ_JARS=/usr/local/tezexport HADOOP_CLASSPATH=${HADOOP_CLASSPATH}:${TEZ_CONF_DIR}:${TEZ_JARS}/*:${TEZ_JARS}/lib/*

   4. Press Esc, type :wq!, and press Enter to save the file and exit.