Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Running and Verifying Spark

After porting Spark to the Kunpeng platform, run Spark to verify its functionality.

The following uses Spark 2.3.0 as an example to describe how to run and verify Spark after the porting. These steps are applicable to other versions of Spark. You need to change the version number in the commands accordingly.

  1. Extract the Spark software package spark-2.3.0-bin-2.6.5.tgz obtained in 4.
    1
2
3
4
    		 
    mv spark-2.3.0-bin-2.6.5.tgz /usr/local
cd /usr/local
tar -zxvf spark-2.3.0-bin-2.6.5.tgz
cd spark-2.3.0-bin-2.6.5
  2. Start the Spark local environment.
    1
    		 
    bin/spark-shell

    If the message Spark context Web UI available at http://server1:4040 is displayed, the Spark startup is successful.

  3. Check whether Spark is running properly.
    1. Check whether any Spark process is running. 
      1
      		 
      ps -ef | grep spark

      If a Spark process is displayed, it indicates that Spark is running.

    2. Access the Spark Web UI.

      Open the browser, enter http://Spark Server IP address:4040 in the address box, and press Enter.

      If the Spark Web UI (including job information and execution status) is accessible, Spark is running properly.

    3. Run the sample program provided by Spark (estimating the value of π using Monte Carlo).

      Run the following command in the CLI:

      spark-submit --class org.apache.spark.examples.SparkPi --master local[*] /usr/local/spark-2.3.0-bin-2.6.5/examples/jars/spark-examples_*.jar 10

      Expected result: The command runs the example program for estimating π and outputs the estimation result. If the result is successfully output, Spark can execute tasks normally.