Using the Foreground Spark Task Interception Function

After the enable_omniadvisor environment variable is enabled, you can directly submit a spark-submit or spark-sql task to trigger the task interception function of OmniAdvisor 2.0.

1. Enable the enable_omniadvisor environment variable.

   export enable_omniadvisor=true

   After the variable is successfully configured, the foreground interception function becomes available.

2. Submit a spark-sql task.

   spark-sql --master yarn --deploy-mode client --driver-cores 8 --driver-memory 20G --num-executors 36 --executor-cores 8 --executor-memory 29g -e "SHOW DATABASES"

   

   If "OmniAdvisor is enabled" is returned, OmniAdvisor is enabled.

3. View the actual execution records in the log file.

   cat $OMNIADVISOR_HOME/BoostKit-omniadvisor_2.0.0/logs/app.log