Installing ceph-ansible

Installing ceph-ansible

Both online and offline installation modes are supported. You need to perform the following operations on ceph 1 only.

Method 1: online installation

1. Install Ansible.
   • CentOS 7.6

     1	yum -y install ansible

   • openEuler 20.03

     1	pip3 install ansible==2.9

2. Install ceph-ansible.
   1. Ensure that Git has been installed in the environment. You can run the following command to install Git:

      1	yum -y install git

      

   2. Set http.sslVerify to false to skip the verification of the system certificate.

      1	git config --global http.sslVerify false

   3. Download the ceph-ansible package.

      1	git clone -b stable-4.0 https://github.com/ceph/ceph-ansible.git --recursive

      

Method 2: offline installation

1. Install Ansible.
   • CentOS 7.6

     1	yum -y install ansible

   • openEuler 20.03

     1	pip3 install ansible

2. Decompress the ceph-ansible package.

   1	unzip /home/ceph-ansible-stable-4.0.zip

   

Installing ceph-ansible Dependencies

Both online and offline installation modes are supported. You need to perform the following operations on ceph 1 only.

Method 1: online installation

1. Install Python 3 pip.

   1	yum install -y python3-pip

2. Go to the ceph-ansible directory.

   1	cd /home/ceph-ansible/

3. Check and install the required software versions.

   1	pip3 install -r requirements.txt

   

Method 2: offline installation

1. Install Python 3 pip.

   1	yum install -y python3-pip

2. Go to the ceph-ansible directory.

   1	cd /home/ceph-ansible/

3. Install the required dependencies.

   1	yum install -y python-netaddr

   

4. Check and install the required software versions.

   1	pip3 install -r requirements.txt

Solving Environment Dependency Problems

1. Solve the rpm_key dependency problem.
   1. Open the redhat_community_repository.yml file.

      1	vim /home/ceph-ansible/roles/ceph-common/tasks/installs/redhat_community_repository.yml

   2. Comment out the following code:

      

2. Solve the Grafana dependency problem.
   1. Open the configure_grafana.yml file.

      1	vim /home/ceph-ansible/roles/ceph-grafana/tasks/configure_grafana.yml

   2. Comment out the following code:

      

Creating the Service Node Configuration List

Create the hosts file in the ceph-ansible directory.

1

vi /home/ceph-ansible/hosts

Add the following information to the file:

This operation is used to define hosts in the cluster and the role of each host in the Ceph cluster. You can deploy applications on cluster nodes based on the requirements of the entire cluster.

Table 1 describes the parameters in the hosts file.

Modifying the ceph-ansible Configuration File

During Ansible deployment, the appropriate playbook must be transferred to the ansible-playbook command. Change the playbook name and modify the corresponding content to meet the cluster deployment requirements.

Use the Ansible variables provided by ceph-ansible to configure the Ceph cluster.

All options and default configurations are stored in the group_vars directory. Each Ceph process corresponds to a configuration file.

1
2
3
4
5
6
7
8

cd /home/ceph-ansible/group_vars/
cp mons.yml.sample mons.yml
cp mgrs.yml.sample mgrs.yml
cp mdss.yml.sample mdss.yml
cp rgws.yml.sample rgws.yml
cp osds.yml.sample osds.yml
cp clients.yml.sample clients.yml
cp all.yml.sample all.yml

all.yml.sample is a special configuration file that applies to all hosts in a cluster.

Adding parameters in the ceph.conf file

The ceph_conf_overrides variable in the all.yml file can be used to overwrite or add configuration in the ceph.conf file.

Open the all.yml file.

1

vim all.yml

1
2
3
4
5
6
7
8

ceph_conf_overrides: 
  global:
    osd_pool_default_pg_num: 64
    osd_pool_default_pgp_num: 64
    osd_pool_default_size: 2
mon_max_pg_per_osd: 3000
  mon:
    mon_allow_pool_create: true

Defining OSDs

You can configure OSD drives, write-ahead logging (WAL) drives, and database (DB) drives in the osds.yml file by using two methods.

Method 1: Running the ceph-volume lvm batch command

In the osds.yml file under the groups_vars directory, set devices, dedicated_devices, and bluestore_wal_devices as required.

• devices indicate the data drives in the system. This parameter can be used together with osds_per_device to set the number of OSDs created on each drive.
• dedicated_devices specifies DB drives.
• bluestore_wal_devices specifies WAL drives.
• The drives specified by dedicated_devices and bluestore_wal_devices must be different.
• dedicated_devices and bluestore_wal_devices can contain one or more drives.
• The system evenly divides the drives specified by dedicated_devices and bluestore_wal_devices based on the number of drives specified by devices and the value of osds_per_device.

The DB drive size of a single OSD must be greater than or equal to 50 GB. Otherwise, an error may be reported during the installation because the DB drive size is too small.

Method 2: Running the ceph-volume command to create OSDs from the LVs

In the osds.yml file under the groups_vars directory, specify all OSD devices in the lvm_volumes attribute as required. The following scenarios are involved:

• Specify only data drives.
• Specify data drives and WAL drives.
• Specify data drives, WAL drives, and DB drives.
• Specify data drives and DB drives.

  Before using this mode, create the corresponding LVs on all cluster nodes.

  1. Create the corresponding volume group (VG).

     1	/usr/sbin/vgcreate --yes -s 1G ceph-data /dev/sdd

     

     The command is used to create a VG named ceph-data. -s specifies the basic unit for creating an LV.

  2. Create an LV based on the VG.

     1	/usr/sbin/lvcreate -l 300 -n osd-data2 ceph-data

     

     The command is used to create on ceph-data an LV named osd-data2 whose size is 300 GB.

  3. Check the LV creation.

     1

     lvs

     [root@ceph1 ceph-ansible-stable-4.0.bak]# lvs
     LV        VG        Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
     home      centos    -wi-ao---- 839.05g
     root      centos    -wi-ao----  50.00g
     swap      centos    -wi-a-----   4.00g
     osd-data1 ceph-data -wi-ao---- 300.00g
     osd-data2 ceph-data -wi-ao---- 300.00g
     osd-db1   ceph-db   -wi-ao---- 100.00g
     osd-db2   ceph-db   -wi-ao---- 100.00g
     osd-wal1  ceph-wal  -wi-ao----  50.00g
     osd-wal2  ceph-wal  -wi-ao----  50.00g

The following figure shows how to create two OSDs, each of which contains DB and WAL drives.

Disabling Chrony

You need to disable Chrony in openEuler only. Skip this section for CentOS.

1. Open roles/ceph-infra/tasks/setup_ntp.yml.

   1	vim roles/ceph-infra/tasks/setup_ntp.yml

2. Comment out the Chrony content.