Configuring the Deployment Environment

Disabling SMMU

Before deploying Ceph, disable SMMU to prevent it from affecting Ceph performance.

1. Access the BIOS.

   For details, see "Accessing the BIOS" in the TaiShan Server BIOS Parameter Reference (Kunpeng 920 Processor).

2. Choose Advanced > MISC Config.

   

3. Set Support Smmu to Disabled and press F10 to save the settings and exit.

   

Configuring Hostnames

1. Configure static hostnames, for example, configure ceph 1 to ceph 3 for server nodes and client 1 to client 3 for client nodes.
   1. Configure hostnames for server nodes.

      Ceph 1:

      1	hostnamectl --static set-hostname ceph1

      

      Set hostnames for other server nodes in the same way.

   2. Set hostnames for client nodes.

      Client 1:

      1	hostnamectl --static set-hostname client1

      Set hostnames for other server nodes in the same way.

2. Modify the domain name resolution file.

   1	vi /etc/hosts

   On each server node and client node, add the following information to the /etc/hosts file:

   1 2 3 4 5 6

   192.168.3.166 ceph1 192.168.3.167 ceph2 192.168.3.168 ceph3 192.168.3.160 client1 192.168.3.161 client2 192.168.3.162 client3

Configuring Password-Free Login

Enable ceph 1 and client 1 to access all server and client nodes (including ceph 1 and client 1) without a password.

1. Generate a public key on ceph 1 and send the public key to each server node and client node.

   1 2 3

   ssh-keygen -t rsa for i in {1..3}; do ssh-copy-id ceph$i; done for i in {1..3}; do ssh-copy-id client$i; done

   

   After inputting the first command ssh-keygen -t rsa, press Enter to use the default configuration.

   

   

2. Generate a public key on client 1 and send the public key to each server node and client node.

   1 2 3

   ssh-keygen -t rsa for i in {1..3}; do ssh-copy-id ceph$i; done for i in {1..3}; do ssh-copy-id client$i; done

   

   After inputting the first command ssh-keygen -t rsa, press Enter to use the default configuration.

Disabling the Firewall

On each server node and client node, run the following commands in sequence to disable the firewall:

1
2
3

systemctl stop firewalld
systemctl disable firewalld
systemctl status firewalld

Setting the Permissive Mode

Set the permissive mode on each server node and client node.

• Set the permissive mode temporarily. The configuration becomes invalid after the system restarts.

  1	setenforce permissive

  

• Set the permissive mode permanently. The configuration takes effect upon the next restart.

  1	vi /etc/selinux/config

  Set SELINUX to permissive.

  

Configuring Repo Sources

This section provides online and offline installation modes for Repo sources. The online installation mode is recommended.

Method 1: online installation

1. Create the ceph.repo file on each server node and client node.

   1	vi /etc/yum.repos.d/ceph.repo

   Add the following information to the file:

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

   [Ceph] name=Ceph packages for $basearch baseurl=http://download.ceph.com/rpm-nautilus/el7/$basearch enabled=1 gpgcheck=1 type=rpm-md gpgkey=https://download.ceph.com/keys/release.asc priority=1 [Ceph-noarch] name=Ceph noarch packages baseurl=http://download.ceph.com/rpm-nautilus/el7/noarch enabled=1 gpgcheck=1 type=rpm-md gpgkey=https://download.ceph.com/keys/release.asc priority=1 [ceph-source] name=Ceph source packages baseurl=http://download.ceph.com/rpm-nautilus/el7/SRPMS enabled=1 gpgcheck=1 type=rpm-md gpgkey=https://download.ceph.com/keys/release.asc priority=1

2. Update the Yum source.

   1	yum clean all && yum makecache

   

3. Install the EPEL source. (Skip this step in openEuler.)

   1	yum -y install epel-release

   

4. Modify the proxy configuration of all nodes.

   1	vim /etc/environment

   Add the following information to support the installation of related dependencies:

   1 2 3 4

   export http_proxy=http://{Proxy-User-Name}:{Proxy-Password}@<Proxy-Server-IP-Address>:<Proxy-Port> export https_proxy= http://{Proxy-User-Name}:{Proxy-Password}@<Proxy-Server-IP-Address>:<Proxy-Port> export ftp_proxy= http://{Proxy-User-Name}:{Proxy-Password}@<Proxy-Server-IP-Address>:<Proxy-Port> export no_proxy=127.0.0.1,localhost

Method 2: offline installation

Perform the following operations on each node to configure Repo sources for all nodes in the cluster:

In this version, you need to manually create the source.zip package. For details, see Creating a Repo Source Package.

1. Upload the source.zip package to the /home directory and go to the directory.

   1

   cd /home

2. Decompress the package.

   1	unzip source.zip

   

3. Install createrepo.

   1	yum install -y createrepo/*.rpm

   

4. Create local sources.

   1 2	cd /home/local_source createrepo .

   

5. Go to the yum.repo.d directory.

   1	cd /home/yum.repo.d

6. Remove the backup of the .repo file provided by the system.

   1 2	mkdir bak mv *.repo bak

7. Create a .repo file.

   1	vi local.repo

   Add the following information to the file:

   1 2 3 4 5

   [local] name=local baseurl=file:///home/local_source enabled=1 gpgcheck=0

Compiling the python-enum34 Package

You need to compile the python-enum34 software package in openEuler only. Skip this section for CentOS.

Perform the following operations on all the nodes.

1. Download the source code and patch.

   1	git clone –b openEuler-20.03-LTS-SP1 https://gitee.com/src-openeuler/python-enum34.git

2. Move the source package to the /root/rpmbuild/SOURCES directory.

   1	mv enum34-1.1.6.tar.gz /root/rpmbuild/SOURCES

3. Modify the .spec file.

   1	vim python-enum34.spec

   Change %files -n python2-%{_name} in line 42 to %files -n python-%{_name}.

   

4. Compile the software package.

   1 2	mv python-enum34.spec /root/rpmbuild/SPECS/ rpmbuild -bb /root/rpmbuild/SPECS/python-enum34.spec

5. Move the compiled RPM package to the specified directory.

   1 2	mkdir -p /home/rpm/local cp -r /root/rpmbuild/RPMS/* /home/rpm/local/

6. Obtain the liboath software package and create a repo source.

   1 2 3 4 5 6

   cd /home/rpm/local wget http://mirrors.tools.huawei.com/openeuler/openEuler-21.03/everything/aarch64/Packages/liboath-2.6.5-2.oe1.aarch64.rpm wget http://mirrors.tools.huawei.com/openeuler/openEuler-21.03/everything/aarch64/Packages/liboath-devel-2.6.5-2.oe1.aarch64.rpm wget http://mirrors.tools.huawei.com/openeuler/openEuler-21.03/everything/aarch64/Packages/liboath-doc-2.6.5-2.oe1.noarch.rpm yum install -y createrepo cd /home/rpm/local & createrepo .

7. Configure the .repo file.

   1	vim /etc/yum.repos.d/local.repo

   Add the following information to the file:

   1 2 3 4 5 6

   [local-rpm] name=local-rpm baseurl=file:///home/rpm/local enabled=1 gpgcheck=0 priority=1

8. Clear the Yum cache.

   1	yum clean all && yum makecache