Performing Spark on Yarn Application-Level Remote Attestation

Add a user for IMA measurement to the virtCCA confidential VM (cVM).
1
useradd attest
Configure the IMA measurement policy on the virtCCA cVM.
1. Open the policy file.
  1
  vi policy
2. Press i to enter the insert mode and add the following content to the file:
  1
  measure func=FILE_CHECK mask=MAY_READ fowner=1001
  1001 is the ID of the attest user in 1. Replace the ID specified by fowner in the policy file with the actual one. You can run cat /etc/passwd | grep attest to view the user ID.
  func=FILE_CHECK indicates that the IMA measurement object is a file.
  mask=MAY_READ indicates that the IMA measurement is triggered by a read operation.
  fowner=1001 indicates that the owner ID of the file to be measured is 1001.
  When the three parameter rules are combined, IMA measurement is triggered when the file whose owner is 1001 is read.
3. Press Esc, type :wq!, and press Enter to save the file and exit.
Make the IMA measurement policy take effect on the virtCCA cVM.
1
cat policy > /sys/kernel/security/ima/policy
Configure the environment variables of the IMA measurement user on the virtCCA cVM.
1. Open the /etc/profile file.
  1
  vi /etc/profile
2. Press i to enter the insert mode and add the following content to the file:
  1
  export IMA_USER=attest
  IMA_USER is the name of the user you added in 1.
3. Press Esc, type :wq!, and press Enter to save the file and exit.

Submit a Spark job on the ResourceManager node of Yarn and perform remote attestation. For example, run the following commands to start the job of calculating the Pi value in the Spark example:

spark-submit --master yarn  --class org.apache.spark.examples.SparkPi --num-executors 1  --executor-cores 1 --executor-memory 1G --conf spark.remote.attestation.enable=true  --conf spark.remote.attestation.agent.class=rats.RatsTLSRemoteAttestationAgent --conf spark.network.timeout=1200 --conf spark.yarn.max.executor.failures=1 --conf spark.yarn.maxAppAttempts=1  --conf spark.executor.extraClassPath=omnishield-1.0-SNAPSHOT.jar:rats-tls-demo-1.0.jar --driver-class-path omnishield-1.0-SNAPSHOT.jar:rats-tls-demo-1.0.jar  --jars "omnishield-1.0-SNAPSHOT.jar,rats-tls-demo-1.0.jar" /usr/local/spark/examples/jars/spark-examples_2.12-3.3.1.jar

rats-tls-demo-1.0.jar is the JAR package where the class specified by spark.remote.attestation.agent.class resides, which needs to be implemented by yourself. This step uses virtcca rats-tls as an example. For details, see Connecting OmniShield to virtcca rats-tls.
spark.remote.attestation.enable is a new configuration item in OmniShield. It is used to configure whether to perform remote attestation during Executor startup. true indicates that remote attestation is required when starting the Executor, whereas false indicates that remote attestation is not required.
spark.remote.attestation.agent.class is a new configuration item in OmniShield. It is used to configure the class name of remote attestation performed by the Executor. You need to implement the interconnection by yourself.
The Spark configuration items spark.yarn.max.executor.failures=1 and spark.yarn.maxAppAttempts=1 are used to set the number of retry times after Spark on Yarn Executor fails to be started. If remote attestation fails, the Spark Executor fails to be started. You can use the preceding two parameters to control the number of retry times.

Parent topic: Using the Feature