OmniAdvisor Configuration File

The configuration files of OmniAdvisor 2.0 are common_config.ini and native_config.ini.

common_config.ini

common_config.ini is the basic configuration file of OmniAdvisor 2.0. It is used to select the tuning method, set the number of retests, and configure database information and Spark history server parameters.

[common]
# Number of retests
tuning.retest.times=1
# Threshold for evaluating a configuration failure. If the number of configuration execution failures is greater than or equal to this threshold, the configuration becomes invalid.
config.fail.threshold=1
# Tuning policy
tuning.strategy=[["transfer", 1],["expert", 2],["iterative", 10]]
# Queue used for background retests. If this parameter is not set, the user queue is retained.
backend.retest.queue=

[database]
# Name of the backend PostgreSQL database
postgresql.database.name=
# User of the backend PostgreSQL database
postgresql.database.user=
# Name of the backend PostgreSQL database host
postgresql.database.host=
# Name of the backend PostgreSQL database port
postgresql.database.port=

[spark]
# URL of the Spark history server, which is used only in Rest mode
spark.history.rest.url=http://localhost:18080
# User name for the Spark history server URL (Set this parameter only when necessary.)
spark.history.username=
# Timeout interval for Spark to capture traces from the history server
spark.fetch.trace.timeout=30
# Interval for Spark to capture traces from the history server
spark.fetch.trace.interval=5
# Spark task timeout interval divided by the baseline time
spark.exec.timeout.ratio=10.0
# Indicates whether to separate stdout and stderr in the Spark output result
spark.output.merge.switch=False

[webpage]
# Time zone displayed on the admin page
admin.timezone=Asia/Shanghai

If the value of spark.fetch.trace.timeout is too small, the trace information of the Spark task may fail to be captured occasionally. See Figure 1.
This problem does not affect the subsequent tuning process. However, you can increase the value to reduce the capture failure probability.

When the network environment is stable, the default value 30 can ensure a relatively high success rate of capturing trace information.
Figure 1 Occasional failure of capturing Spark task trace information
If the value of spark.exec.timeout.ratio is too small, the probability of tuning configuration execution failures due to timeouts increases, especially when the baseline performance value is small.
The default value 10 can meet the tuning requirements in most cases.

If tuning failures frequently occur due to timeout, increase the value of this parameter.
The value of the tuning.retest.times parameter must be greater than 0.

native_config.ini

native_config.ini is the configuration file for the native tuning method. You can edit this file to specify the OmniOperator deployment path and version.

[native]
# Root directory for deploying OmniOperator
omnioperator_home = /opt/omni-operator
# OmniOperator version
omnioperator_version = 1.7.0
# Associated Spark version associated with the OmniOperator version, used to form content similar to boostkit-omniop-spark-3.3.1-1.7.0-aarch64.jar
omnioperator_spark_version = 3.3.1
# Root directory for deploying Hadoop
hadoop_home = /usr/local/hadoop

**Table 1** Fields in the database tables
Table Name	Field Name	Description
omniadvisor_exam_record	id	Auto-generated primary key
	tuning_record	ID of the tuning record associated with the test record
	start_time	Spark task start time
	end_time	Spark task end time
	status	Whether the Spark task is successfully executed
	runtime	Spark task execution duration
	application_id	application_id of the Spark task
	trace	Trace data generated during Spark task execution
omniadvisor_load	id	Auto-generated primary key
	name	Load name
	exec_attr	Load execution characteristics
	default_config	Default configuration
	best_config	Historical optimal configuration
	test_config	Configuration to be retested
	create_time	Load creation time
	tuning_needed	Whether tuning is required
	hash_value	Hash value calculated based on exec_attr and default_config
	backend_retest_enable	Whether the load supports background retests
omniadvisor_tuning_record	id	Auto-generated primary key
	load	Load ID associated with the tuning record
	config	Tuning configuration
	method	Tuning method
	method_extend	Subclasses of the tuning method
	rounds	Number of tuning rounds

Parent topic: References