(Optional) Configuring the keytab and whitelist Files

Procedure

1. Log in to all servers in the OmniShuffle cluster and modify the /etc/hosts file on each server.

   Add the mapping between the IP addresses and host names of the KDC server to the hosts file.

   IPaddress1 server

2. Log in to the KDC server.
3. Add the OCK_HOME, OCK_VERSION, and OCK_BINARY_TYPE environment variables to ${HOME}/.bashrc of the ZooKeeper run user and the user who submits Spark tasks.
   1. Open the file.

      vi ~/.bashrc

   2. Press i to enter the insert mode and add the following content to the file:

      export OCK_HOME=/home/ockadmin/opt/ock
      export OCK_VERSION=24.0.0
      export OCK_BINARY_TYPE=linux-aarch64

   3. Press Esc, type :wq!, and press Enter to save the file and exit.
4. You need to create the digital certificate file, whitelist file, and KMC key file required by the KDC server. Copy the files to the specified location of the corresponding user on each node.
   • Files generated by the OCK user are stored in ${OCK_HOME}/security/.

     ├── authorization

     │ └── whitelist_en

     ├── kdc

     │ └── krb5-server_en.keytab

     ├── pmt

     │ ├── master

     │ │ └── ksfa

     │ └── standby

     │ └── ksfb

     └── tls

     ├── client.crt.pem

     ├── client.pem

     ├── server

     │ ├── ca.cert.pem

     │ ├── server.cert.pem

     │ ├── server.keypass.key

     │ └── server.private.key.pem

     └── server.crt.pem

     The configurations are as follows:

     User	File	Configuration Item	Example Path
     ockadmin	whitelist_en	ock.ucache.rpc.author.file.path	/home/ockadmin/opt/ock/security/authorization/whitelist_en
     ockadmin	krb5-server_en.keytab	ock.ucache.rpc.auth.kerb.server.keytab	/home/ockadmin/opt/ock/security/kdc/krb5-server_en.keytab
     ockadmin	krb5-server_en.keytab	ock.zookeeper.security.client.keytab	/home/ockadmin/opt/ock/security/kdc/krb5-server_en.keytab
     ockadmin	ksfa	ock.ucache.kmc.ksf.primary.path	/home/ockadmin/opt/ock/security/pmt/master/ksfa/keytab
     ockadmin	ksfb	ock.ucache.kmc.ksf.standby.path	/home/ockadmin/opt/ock/security/pmt/standby/ksfb
     ockadmin	client.crt.pem	ock.zookeeper.security.certs	/home/ockadmin/opt/ock/security/tls/client.crt.pem
     ockadmin	client.pem	ock.zookeeper.security.certs	/home/ockadmin/opt/ock/security/tls/client.pem
     ockadmin	ca.cert.pem	ock.ucache.rpc.tls.ca.cert.path	/home/ockadmin/opt/ock/security/tls/server/ca.cert.pem
     ockadmin	server.cert.pem	ock.ucache.rpc.tls.cert.path	/home/ockadmin/opt/ock/security/tls/server/server.cert.pem
     ockadmin	server.keypass.key	ock.ucache.rpc.tls.key.pass.path	/home/ockadmin/opt/ock/security/tls/server/server.keypass.key
     ockadmin	server.private.key.pem	ock.ucache.rpc.tls.key.path	/home/ockadmin/opt/ock/security/tls/server/server.private.key.pem
     ockadmin	server.crt.pem	ock.zookeeper.security.certs	/home/ockadmin/opt/ock/security/tls/server.crt.pem

   • Files generated by the user who submits the Spark tasks are stored in the ${SPARK_HOME}/huawei/ock/security directory of the user.

     ├── authorization

     │ └── whitelist_en

     ├── kdc

     │ ├── krb5-client_en.keytab

     │ └── krb5-server_en.keytab

     ├── pmt

     │ ├── master

     │ │ └── ksfa

     │ └── standby

     │ └── ksfb

     │

     └── tls

     ├── ca.cert.pem

     ├── client.crt.pem

     ├── client.pem

     ├── server.cert.pem

     ├── server.crt.pem

     ├── server.keypass.key

     └── server.private.key.pem

     The configurations are as follows:

     User	File	Configuration Item	Example Path
     Sparkadmin	whitelist_en	ock.ucache.rpc.author.driver.file.path	/home/Sparkadmin/huawei/ock/security/authorization/whitelist_en
     Sparkadmin	krb5-client_en.keytab	ock.ucache.rpc.auth.kerb.client.keytab	/home/Sparkadmin/huawei/ock/security/kdc/krb5-client_en.keytab
     Sparkadmin	krb5-client_en.keytab	ock.zookeeper.sdk.security.client.keytab	/home/Sparkadmin/huawei/ock/security/kdc/krb5-client_en.keytab
     Sparkadmin	krb5-server_en.keytab	ock.ucache.rpc.auth.driver.kerb.server.keytab	/home/Sparkadmin/huawei/ock/security/kdc/krb5-server_en.keytab
     Sparkadmin	ksfa	ock.ucache.sdk.kmc.ksf.primary.path	/home/Sparkadmin/huawei/ock/security/pmt/master/ksfa
     Sparkadmin	ksfb	ock.ucache.sdk.kmc.ksf.standby.path	/home/Sparkadmin/huawei/ock/security/pmt/standby/ksfb
     Sparkadmin	client.crt.pem	ock.zookeeper.sdk.security.certs	/home/Sparkadmin/huawei/ock/security/tls/client.crt.pem
     Sparkadmin	client.pem	ock.zookeeper.sdk.security.certs	/home/Sparkadmin/huawei/ock/security/tls/client.pem
     Sparkadmin	ca.cert.pem	ock.ucache.rpc.tls.sdk.ca.cert.path	/home/Sparkadmin/huawei/ock/security/tls/ca.cert.pem
     Sparkadmin	server.cert.pem	ock.ucache.rpc.tls.driver.cert.path	/home/Sparkadmin/huawei/ock/security/tls/server.cert.pem
     Sparkadmin	server.keypass.key	ock.ucache.rpc.tls.driver.key.pass.path	/home/Sparkadmin/huawei/ock/security/tls/server.keypass.key
     Sparkadmin	server.private.key.pem	ock.ucache.rpc.tls.driver.key.path	/home/Sparkadmin/huawei/ock/security/tls/server.private.key.pem
     Sparkadmin	server.crt.pem	ock.zookeeper.sdk.security.certs	home/Sparkadmin/huawei/ock/security/tls/server.crt.pem

     The files in the authorization directory are used for authorization, the files in the kdc directory are used for security authentication, the files in the tls directory are used for TLS secure transmission, and the files in the pmt directory are the encryption key used to encrypt certificate files. The files in the authorization, kdc, and tls directories are encrypted using the encryption key in the pmt directory. For details about how to encrypt files, see 4.2.

   • Files generated by the ZooKeeper user are stored in the ${ZOOKEEPER_HOME}/huawei/ock/security directory of the user.

     ├── kdc

     │ └── zookeeper_en.keytab

     ├── pmt

     │ ├── master

     │ │ └── ksfa

     │ └── standby

     │ └── ksfb

     The configurations are as follows:

     User	File	Configuration Item	Example Path
     Zookeeperadmin	zookeeper_en.keytab	keytab	/home/Zookeeperadmin/huawei/ock/security/kdc/zookeeper_en.keytab
     Zookeeperadmin	ksfa	kmc.ksf.primary.path	/home/Zookeeperadmin/huawei/ock/security/pmt/master/ksfa
     Zookeeperadmin	ksfb	kmc.ksf.standby.path	/home/Zookeeperadmin/huawei/ock/security/pmt/standby/ksfb

     For details about the ZooKeeper configuration, see 2.2.3.