Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Deploying Ceph

Perform the following operations to install the Ceph software and deploy nodes to enable the smart write cache service in the Ceph cluster.

Installing the Ceph Software

For details, see "Configuring the Deployment Environment" in Ceph Block Storage Deployment Guide (CentOS 7.6, openEuler 20.03, and openEuler 22.03).

Deploying MON Nodes

For details, see "Deploying MON Nodes" in Ceph Block Storage Deployment Guide (CentOS 7.6, openEuler 20.03, and openEuler 22.03).

Deploying MGR Nodes

For details, see "Deploying MGR Nodes" in Ceph Block Storage Deployment Guide (CentOS 7.6, openEuler 20.03, and openEuler 22.03).

Deploying OSD Nodes

  1. Run the lsblk command to check whether the drive used as a data drive has partitions.
    1
    lsblk
    • If yes, go to 2. Clear the partition information and then start the deployment.
    • If no, go to 3.
  2. Clear the partition information (the drive letter /dev/sdb is used as an example).
    1
    ceph-volume lvm zap /dev/sdb --destroy
  3. Create a shell script on Ceph 1 and use the 12 bcache drives on each server as OSD data drives.
    1. Create create_osd.sh file.
      1
2
      cd /etc/ceph
vim /etc/ceph/create_osd.sh
    2. Press i to enter the insert mode and add the following content:
       1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
      #!/bin/bash 
for node in ceph1 ceph2 ceph3 
do 
         k=1 
         for i in `ssh ${node} "ls /sys/block | grep bcache | head -n 6"` 
         do 
                 ceph-deploy osd create ${node} --data /dev/${i}  --block-db /dev/nvme0n1p${k} 
                 ((j=${j}+1)) 
                 ((k=${k}+1)) 
                 sleep 3 
         done 
         k=1 
         for i in `ssh ${node} "ls /sys/block | grep bcache | tail -n 6"` 
         do 
                 ceph-deploy osd create ${node} --data /dev/${i}  --block-db /dev/nvme1n1p${k} 
                 ((j=${j}+1)) 
                 ((k=${k}+1)) 
                 sleep 3 
         done 
done
      • This script applies only to the current hardware configuration. For other hardware configurations, you need to modify the script.
      • In the ceph-deploy osd create command:
        • ${node} specifies the host name of the node.
        • --data specifies a data drive. The backend drive of bcache is used as a data drive.
        • --block-db specifies the DB partition.
      • The DB and WAL partitions are deployed on NVMe SSDs to improve write performance. If no NVMe SSD is configured or NVMe SSDs are used as data drives, you do not need to specify --block-wal. Instead, specify --data.
    3. Press Esc to exit the insert mode. Type :wq! and press Enter to save and exit the file.
  4. Run the script on ceph1.
    1
    bash create_osd.sh
  5. Check whether the OSD nodes are successfully created.
    ceph -s

    If the status of all the 36 OSD nodes is up, the creation is successful.