Prerequisites

Before using the OmniAdvisor 2.0 feature, activate or start associated services. Ensure that the management node meets the requirements described in this section.

1. Activate Anaconda.

   conda activate OmniAdvisor2.0

2. In the sbin directory of Spark, start the Spark history server.

   sh $SPARK_HOME/sbin/start-history-server.sh

3. Verify that the history server has been started.

   In the address box of the browser, enter http://<server1>:18080 (the default port is 18080) to view the UI.

   

4. Optional: Change the history server port.

   The default port of the Spark history server is 18080. If the port is occupied, you can modify the configuration to change the port to another port that is not occupied. The following describes how to change the port to 18181.

   1. Open the $SPARK_HOME/conf/spark-defaults.conf configuration file.

      vi $SPARK_HOME/conf/spark-defaults.conf

   2. Press i to enter the insert mode and add spark.history.ui.port or change its value to 18181.

      spark.history.ui.port 18181

   3. Press Esc, type :wq!, and press Enter to save the file and exit.
   4. Restart the history server.

      In the sbin directory of Spark, stop and start the history server.

      # Stop the history server.
      sh $SPARK_HOME/sbin/stop-history-server.sh
      # Start the history server.
      sh $SPARK_HOME/sbin/start-history-server.sh