Environment Requirements
Hardware Requirements
Table 1 lists the hardware requirements.
Server |
Kunpeng server |
|---|---|
Processor |
Kunpeng 920 processor |
Number of Cores |
2 x 32-core |
CPU Frequency |
2600 MHz |
Memory Capacity |
8 x 16 GB |
Memory Frequency |
2933 MHz |
NIC |
IN200 NIC (4 x 25GE) |
Drive |
System drives: RAID 1 (2 x 960 GB SATA SSD) Data drives: JBOD enabled in RAID mode (12 x 4 TB SATA HDD) |
Cache drive: 1 x ES3000 V5 3.2 TB |
|
RAID Controller Card |
Avago SAS 3508 |
The installation of Ceph and its dependencies requires Internet access. Ensure that the server is connected to the Internet.
Software Requirements
Table 2 lists the software requirements.
Item |
Version |
|---|---|
OS |
CentOS Linux release 7.6.1810 |
openEuler 20.03 LTS SP1 (Kernel: 4.19.90-2012.4.0.0053.oe1) |
|
openEuler 20.03 LTS SP2 (Kernel: 5.10.0-153.12.0.92.oe2) |
|
openEuler 22.03 LTS SP4 (Kernel: 5.10.0-216.0.0.115.oe2203sp4.aarch64) |
|
Ceph |
14.2.8 Nautilus/16.2.7 Pacific |
ceph-deploy |
2.0.1/2.1.0 |
- This document uses Ceph 14.2.8 and 16.2.7 as an example. You can also refer to this document to install other versions.
- If the OS needs to be installed, the minimum installation mode is recommended.
Cluster Environment Planning
The cluster consists of Ceph clients and Ceph servers. The clients are client1 to client3, and the servers are ceph1 to ceph3. Figure 1 shows the networking.
Table 3 lists the server IP addresses in the Ceph cluster.
Cluster Node |
Management IP Address |
Public Network |
Cluster Network |
|---|---|---|---|
ceph1 |
192.168.2.166 |
192.168.3.166 |
192.168.4.166 |
ceph2 |
192.168.2.167 |
192.168.3.167 |
192.168.4.167 |
ceph3 |
192.168.2.168 |
192.168.3.168 |
192.168.4.168 |
Table 4 lists the client IP addresses in the Ceph cluster.
Client Node |
Management IP Address |
Public Network |
|---|---|---|
client1 |
192.168.2.160 |
192.168.3.160 |
client2 |
192.168.2.161 |
192.168.3.161 |
client3 |
192.168.2.162 |
192.168.3.162 |
- Management IP address: IP address used for remote SSH machine management and configuration.
- Cluster network IP address: IP address used for data synchronization between nodes in the cluster. The 25GE network port is recommended.
- Public network IP address: IP address of the storage node for other nodes to access. The 25GE network port is recommended.
- When clients are used as presses, ensure that the service port IP addresses of clients and the public network IP addresses of the cluster are in the same network segment. The 25GE network port is recommended.
Drive Partitioning
Ceph 14.2.10 uses BlueStore as the back-end storage engine. The Journal partition in the Jewel version is no longer used. Instead, the DB partition (metadata partition) and WAL partition are used. The two partitions store back-end metadata and log files generated by BlueStore. The metadata is used to improve the efficiency of the entire storage system, and the logs are used to maintain system stability.
In cluster deployment mode, each Ceph node is configured with twelve 4 TB data drives and one 3.2 TB NVMe drive. Each 4 TB data drive functions as the data partition of an Object Storage Daemon (OSD), and the NVMe drive functions as the DB and WAL partitions of the 12 OSDs. Generally, the WAL partition is sufficient if its capacity is greater than 10 GB. According to the official Ceph document, it is recommended that the size of each DB partition be at least 4% of the capacity of each data drive. The size of each DB partition can be flexibly configured based on the NVMe drive capacity. In this solution, the NVMe drive is divided into 27 partitions. Twelve 20 GB partitions are used as WAL, and twelve 45 GB partitions are used as DB. To improve the object storage performance, each NVMe drive has three 700 GB partitions as the storage pool for storing RGW metadata.
Table 5 lists the partitions of an NVMe drive.