Configuring the ZooKeeper TLS

This section does not describe how to use the certificate tool. See Configuring the Global Cache TLS for reference.

CCM-ZK Deployment

Obtain the password ciphertext on each ZooKeeper server node.

cat /opt/gcache/secure/Certs/identity.ks

The password ciphertext is in the following format:

AAAAAgAAAAAAAAAAAAAAAQAAAAmfTmJhF91SS6/7xEZldZErWUrkuRtyiFbjfM0gAAAAAAEAAAEAAAAAAAAAGr2WPWfiMhmqBd1w/bsAfJ2q+QBtJbC0EsBJ

Modify the ZooKeeper configuration file on each ZooKeeper server node.

vi /opt/apache-zookeeper-3.6.3-bin/conf/zoo.cfg

Add the following fields to the zoo.cfg file on each ZooKeeper server node:

secureClientPort=2281
ssl.protocol=TLSv1.2
ssl.enabledProtocols=TLSv1.2
ssl.ciphersuites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
ssl.keyStore.location= /opt/gcache/secure/Certs/keystore.jks
ssl.keyStore.password= 
    # Password ciphertext obtained in step 1.
ssl.trustStore.location= /opt/gcache/secure/Certs/truststore.jks
ssl.trustStore.password= 
    # Password ciphertext obtained in step 1.
ssl.switch=on 
    # on indicates that the ciphertext password is valid, and off indicates that the plaintext password is invalid.

Delete the following field:

clientPort=2181

Copy the KMC key to the ZooKeeper configuration file directory on each ZooKeeper server node.

mkdir -p /opt/apache-zookeeper-3.6.3-bin/conf/keystore/
chmod 750 /opt/apache-zookeeper-3.6.3-bin/conf/keystore/
cp /opt/gcache/secure/kmc/kmc.primary.ks /opt/apache-zookeeper-3.6.3-bin/conf/keystore/zk_kmc_primary.ks
cp /opt/gcache/secure/kmc/kmc.standby.ks /opt/apache-zookeeper-3.6.3-bin/conf/keystore/zk_kmc_standby.ks

Upload the ZooKeeper security hardening patch boostkit-zk-secure.tar.gz to the /opt/apache-zookeeper-3.6.3-bin directory and run the following command on each ZooKeeper server node to install the patch:

cd /opt/apache-zookeeper-3.6.3-bin
tar xvf boostkit-zk-secure.tar.gz
cp /opt/apache-zookeeper-3.6.3-bin/build/jar/one-track-4-kmc-21.0.2.jar /opt/apache-zookeeper-3.6.3-bin/lib/one-track-4-kmc-21.0.2.jar
cp /opt/apache-zookeeper-3.6.3-bin/build/jar/boostkit-globalcache-zk-21.0.0.jar /opt/apache-zookeeper-3.6.3-bin/lib/boostkit-globalcache-zk-21.0.0.jar

Modify the ZooKeeper startup script zkServer.sh on each ZooKeeper server node.

sed -ri 's|org.apache.zookeeper.server.quorum.QuorumPeerMain|com.huawei.kunpeng.zookeeper.KunpengQuorumPeerMain|g' /opt/apache-zookeeper-3.6.3-bin/bin/zkServer.sh

Grant the required permission on the new files to the globalcacheop user.

chown globalcacheop:globalcache /opt/apache-zookeeper-3.6.3-bin/conf/keystore/*
chown globalcacheop:globalcache /opt/apache-zookeeper-3.6.3-bin/lib/*

Restart ZooKeeper on each ZooKeeper server node.

         
              cd /opt/apache-zookeeper-3.6.3-bin/bin
sh zkServer.sh stop
sh zkServer.sh start

Modify the Global Cache configuration file on each node.

vi /opt/gcache/conf/gcache.conf

Modify the following configuration items as required by referring to the configurations in the security label in the gcache.conf file of the client and server software.

[communicate]
zk_server_list = ceph1:2281,ceph2:2281,ceph3:2281 # The port number must be the same as the value of secureClientPort in step 2.
[security]
tls_status = on
kmc_path = /opt/gcache/secure/kmc
cert_path = /opt/gcache/secure/Certs

BCM-ZK Deployment

Obtain the password ciphertext on each ZooKeeper server node.

cat /opt/gcache/secure/Certs/identity.ks

The password ciphertext is in the following format:

AAAAAgAAAAAAAAAAAAAAAQAAAAmfTmJhF91SS6/7xEZldZErWUrkuRtyiFbjfM0gAAAAAAEAAAEAAAAAAAAAGr2WPWfiMhmqBd1w/bsAfJ2q+QBtJbC0EsBJ

Modify the ZooKeeper configuration file on each ZooKeeper server node.

vi /opt/apache-zookeeper-3.6.3-bin-bcm/conf/zoo.cfg

Add the following fields to the zoo.cfg file on each ZooKeeper server node:

secureClientPort=2282
ssl.protocol=TLSv1.2
ssl.enabledProtocols=TLSv1.2
ssl.ciphersuites=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory
ssl.keyStore.location= /opt/gcache/secure/Certs/keystore.jks
ssl.keyStore.password= 
    # Password ciphertext obtained in step 1.
ssl.trustStore.location= /opt/gcache/secure/Certs/truststore.jks
ssl.trustStore.password= 
    # Password ciphertext obtained in step 1.
ssl.switch=on 
    # on indicates that the ciphertext password is valid, and off indicates that the plaintext password is invalid.

Delete the following field:

clientPort=2181

Copy the KMC key to the ZooKeeper configuration file directory on each ZooKeeper server node.

mkdir -p /opt/apache-zookeeper-3.6.3-bin-bcm/conf/keystore/
chmod 750 /opt/apache-zookeeper-3.6.3-bin-bcm/conf/keystore/
cp /opt/gcache/secure/kmc/kmc.primary.ks /opt/apache-zookeeper-3.6.3-bin-bcm/conf/keystore/zk_kmc_primary.ks
cp /opt/gcache/secure/kmc/kmc.standby.ks /opt/apache-zookeeper-3.6.3-bin-bcm/conf/keystore/zk_kmc_standby.ks

Upload the ZooKeeper security hardening patch boostkit-zk-secure.tar.gz to the /opt/apache-zookeeper-3.6.3-bin-bcm directory and run the following command on each ZooKeeper server node to install the patch:

cd /opt/apache-zookeeper-3.6.3-bin-bcm
tar xvf boostkit-zk-secure.tar.gz
cp /opt/apache-zookeeper-3.6.3-bin-bcm/build/jar/one-track-4-kmc-21.0.2.jar /opt/apache-zookeeper-3.6.3-bin-bcm/lib/one-track-4-kmc-21.0.2.jar
cp /opt/apache-zookeeper-3.6.3-bin-bcm/build/jar/boostkit-globalcache-zk-21.0.0.jar /opt/apache-zookeeper-3.6.3-bin-bcm/lib/boostkit-globalcache-zk-21.0.0.jar

Modify the ZooKeeper startup script zkServer.sh on each ZooKeeper server node.

sed -ri 's|org.apache.zookeeper.server.quorum.QuorumPeerMain|com.huawei.kunpeng.zookeeper.KunpengQuorumPeerMain|g' /opt/apache-zookeeper-3.6.3-bin-bcm/bin/zkServer.sh

Grant the required permission on the new files to the globalcache user.

chown globalcacheop:globalcache /opt/apache-zookeeper-3.6.3-bin-bcm/conf/keystore/*
chown globalcacheop:globalcache /opt/apache-zookeeper-3.6.3-bin-bcm/lib/*

Restart ZooKeeper on each ZooKeeper server node.

         
              cd /opt/apache-zookeeper-3.6.3-bin-bcm/bin
sh zkServer.sh stop
sh zkServer.sh start

Modify the BCM ZooKeeper cluster configuration file bcm.xml.
```
vi /opt/gcache/conf/bcm.xml
```
Change the port number in zk_server_list to 2282. For details about the bcm.xml file, see Description of bcm.xml.

After modifying zk_server_list in the bcm.xml file, you need to import the file again. For details, see 1.c and 1.d in Verifying Global Cache.

Parent topic: Configuring TLS