Failed to Start Hadoop Using Yarn

Symptom

The following information is displayed when using Yarn to start Hadoop 3.2.2 or later:

2024-09-19 15:41:42,256 INFO org.apache.hadoop.service.AbstractService: Service NodeManager failed in state INITED
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to initialize container executor
...
Caused by: ExitCodeException exitCode=24: File /home/sparkadmin/hadoop-3.2.0/etc/hadoop/container-executor.cfg must be owned by root, but is owned by 1002

Key Process and Cause Analysis

LinuxContainerExecutor uses container-executor to start containers. For system security purposes, the owner of the container-executor.cfg file and its parent paths must be the root user. The path configured in container-executor is the default path and container-executor must be recompiled.

Conclusion and Solution

1. Obtain the Hadoop source package of the required version, for example, Hadoop 3.2.0 RC1.
2. Decompress the source package and go to the hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager directory.
3. Perform the compilation.

   cmake src -DHADOOP_CONF_DIR=/etc/hadoop
   make

4. Go to the target/native/target/usr/local/bin directory and extract the container-executor file and change the file owner to root. Replace the corresponding binary file in the system environment.