Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Compiling Yahoo Streaming Benchmark

  1. Download the Yahoo Streaming Benchmark source package.
    1
    		 
    wget https://github.com/dataArtisans/yahoo-streaming-benchmark/archive/master.zip
  2. Decompress the source package.
    1
    		 
    unzip master.zip
  3. Go to the source code directory yahoo-streaming-benchmark-master.
    1
    		 
    cd yahoo-streaming-benchmark-master
  4. Modify the stream-bench.sh file.
    1. Open the file. 
      1
      		 
      vi stream-bench.sh
    2. Press i to enter the insert mode and modify the file as follows: 
       1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
      		 
      #Fetch Kafka
KAFKA_FILE="$KAFKA_DIR.tgz"
fetch_untar_file "$KAFKA_FILE" "https://archive.apache.org/dist/kafka/$KAFKA_VERSION/$KAFKA_FILE"
#Fetch Storm
STORM_FILE="$STORM_DIR.tar.gz"
fetch_untar_file "$STORM_FILE" "https://archive.apache.org/dist/storm/$STORM_DIR/$STORM_FILE"
#Fetch Flink
FLINK_FILE="$FLINK_DIR-bin-hadoop27-scala_${SCALA_BIN_VERSION}.tgz"
fetch_untar_file "$FLINK_FILE" "https://archive.apache.org/dist/flink/flink-$FLINK_VERSION/$FLINK_FILE"
#Fetch Spark
SPARK_FILE="$SPARK_DIR.tgz"
fetch_untar_file "$SPARK_FILE" "https://archive.apache.org/dist/spark/spark-$SPARK_VERSION/$SPARK_FILE"
    3. Press Esc, type :wq!, and press Enter to save the file and exit.
  5. Create a code repository and track all code.
    1
2
    		 
    git init
git add *
  6. Run the compilation script.
    1
    		 
    sh stream-bench.sh SETUP
    Figure 1 Spark TAR file downloaded