Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Solution Implementation

Software Version

Item

Version

Hadoop

apache hadoop-2.7.2

Code Implementation

To implement the solution, you need to modify the source code file FSDownload.java in Hadoop.

For details about the original code of FSDownload.java, see the following link.

  1. Add the aarch64PyMix(File dst) method to the hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java file. This method handles directory renaming for Arm architecture compatibility after decompression.
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
    private void aarch64PyMix(File dst) {  
  String dstPath = dst.getAbsolutePath();  
  String fileFullName = dst.getName();  
  String fileName = fileFullName.substring(0,fileFullName.length() - 4);  
  String orgDirName = dstPath + File.separator + fileName;  
  String aarch64DirName = dstPath + File.separator + fileName + "_aarch64";
  File orgFile = new File(orgDirName);
  File aarch64File = new File(aarch64DirName);
  if (orgFile.exists() && aarch64File.exists() && "aarch64".equals(System.getProperty("os.arch"))) {
     orgFile.renameTo(new File(orgDirName + "_bak"));
     aarch64File.renameTo(new File(orgDirName));
  }
}
  2. In the hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/FSDownload.java file, add code for invoking the aarch64PyMix method after the ZIP package is decompressed under the unpack method's ARCHIVE case, as shown in the following information in bold:
    ...
case ARCHIVE:
...  
  } else if (lowerDst.endsWith(".zip")) {
    FileUtil.unZip(localrsrc, dst);
    aarch64PyMix(dst);
  } else if (lowerDst.endsWith(".tar.gz")) ||
...