Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Creating a Storage Pool

By default, Ceph storage pools use the three-replica mode. Data object storage pools sometimes use the erasure code (EC) mode to save storage space. The following describes how to create a storage pool in replication mode and in EC mode.

Perform operations in this section only on ceph1.

Creating a Storage Pool in Replication Mode

  1. Set CRUSH rules.
    1. Run the following command on ceph1 to check CRUSH classes: 
      1
      		 
      ceph osd crush class ls

      If the server has OSDs created based on SSDs and HDDs, two CRUSH classes are displayed.

      [
"hdd",
"ssd"
]
    2. Create CRUSH rules for the SSD class and HDD class. If no OSD is created based on an SSD, you do not need to create rule-ssd. 
      1
2
      		 
      ceph osd crush rule create-replicated rule-ssd default host ssd
ceph osd crush rule create-replicated rule-hdd default host hdd

    3. Check the CRUSH rules. 
      1
      		 
      ceph osd crush rule ls

      replicated_rule is the default CRUSH rule used by the cluster. If no CRUSH rule is specified, replicated_rule is used by default. replicated_rule is in the three-replica mode. All data in the storage pool is stored on all storage devices (SSDs and HDDs) based on a certain proportion. rule-ssd allows data to be stored only on SSDs, and rule-hdd allows data to be stored only on HDDs.

  2. Run the following commands on ceph1 to create a data pool and an index pool:
    1
2
3
4
5
    		 
    cd /root/ceph-mycluster/
ceph osd pool create default.rgw.buckets.data 32 32
ceph osd pool create default.rgw.buckets.index 8 8
ceph osd pool application enable default.rgw.buckets.data rgw
ceph osd pool application enable default.rgw.buckets.index rgw
    • The two numbers in the storage pool creation command (for example, ceph osd pool create default.rgw.buckets.data 32 32) correspond to the pg_num and pgp_num parameters of the created storage pool, indicating the placement group (PG) quantity and the quantity of PGs for placement purposes (PGP) respectively.
      • According to the official Ceph document, the recommended total number of storage pool PGs in a cluster is calculated as follows: (Number of OSDs x 100)/Data redundancy factor. For the replication mode, the data redundancy factor is the number of copies. For the EC mode, the data redundancy factor is the sum of the numbers of data blocks and parity blocks. For example, the data redundancy factor is 3 for the three-replica mode and 6 for the EC 4+2 mode.
      • Assume that the cluster has three servers and each server has 15 OSDs. The total number of OSDs is 45. In the three-replica mode, the PG quantity is 1500 (45 x 100/3). It is recommended that the PG quantity be an integral power of 2. The data volume of default.rgw.buckets.data is much larger than other storage pools and therefore more PGs need to be allocated to this storage pool. In this case, you can set the PG quantity of default.rgw.buckets.data to 1024 and the PG quantity of default.rgw.buckets.index to 128 or 256.
    • For more information about pools, see the description in the Ceph open source community.
  3. Modify the CRUSH rules of all storage pools.
    1
    		 
    ceph osd pool set default.rgw.buckets.data crush_rule rule-hdd
  4. Use cURL or a web browser to log in to the nodes for verification. Ensure that the IP address matches the port number. For example, the port number 10001 corresponds to the IP address 192.168.3.164. Enter 192.168.3.164:10001 in the address box of the browser. If the following information is displayed, RGW is created successfully.

Creating a Storage Pool in EC Mode

  1. Set CRUSH rules.
    1. Run the following command on ceph1 to check CRUSH classes: 
      1
      		 
      ceph osd crush class ls

      If the server has OSDs created based on SSDs and HDDs, two CRUSH classes are displayed.

      [
"hdd",
"ssd"
]
    2. Create CRUSH rules for the SSD class. (Assume that OSDs are deployed on SSDs.) 
      1
      		 
      ceph osd crush rule create-replicated rule-ssd default host ssd
    3. Check the CRUSH rules. 
      1
      		 
      ceph osd crush rule ls

      Expected command output:

      replicated_rule
rule-ssd

      replicated_rule is the default CRUSH rule used by the cluster. If no CRUSH rule is specified, replicated_rule is used by default. replicated_rule is in the three-replica mode. All data in the storage pool is stored on all storage devices (SSDs and HDDs) based on a certain proportion. rule-ssd allows data to be stored only on SSDs, and rule-hdd allows data to be stored only on HDDs.

  2. Create an EC profile.
    1
    		 
    ceph osd erasure-code-profile set myprofile k=4 m=2 crush-failure-domain=osd crush-device-class=ssd

    Take EC 4+2 as an example. The preceding command creates an EC profile named myprofile.

    • k specifies the number of data blocks.
    • m specifies the number of parity blocks.
    • crush-failure-domain=host indicates that the minimum fault domain is host.

      Generally, the minimum fault domain is set to host. If the number of hosts is less than k + m, the fault domain needs to be changed to osd. Otherwise, an error may occur due to insufficient number of hosts.

    • crush-device-class=ssd indicates that the CRUSH rule is created on SSDs.
  3. Run the following commands on ceph1 to create a data pool and an index pool:
    1
2
3
4
    		 
    ceph osd pool create default.rgw.buckets.data 2048 2048 erasure myprofile
ceph osd pool create default.rgw.buckets.index 256 256
ceph osd pool application enable default.rgw.buckets.data rgw
ceph osd pool application enable default.rgw.buckets.index rgw
    • The ceph-deploy osd pool create default.rgw.buckets.data 2048 2048 erasure myprofile command is used to create a pool in EC mode. For object storage, only default.rgw.buckets.data needs to be set to the EC mode. Other pools still use the default three-replica mode.
    • The two numbers in the storage pool creation command (for example, ceph osd pool create default.rgw.buckets.data 2048 2048 erasure myprofile) respectively correspond to the pg_num and pgp_num parameters of the storage pool.
      • According to the official Ceph document, the recommended total number of storage pool PGs in a cluster is calculated as follows: (Number of OSDs x 100)/Data redundancy factor. For the replication mode, the data redundancy factor is the number of copies. For the EC mode, the data redundancy factor is the sum of the numbers of data blocks and parity blocks. For example, the data redundancy factor is 3 for the three-replica mode and 6 for the EC 4+2 mode.
      • Assume that the cluster has three servers and each server has 36 OSDs in the EC 4+2 mode. The total number of OSDs is 108. According to the formula, the PG quantity is 1800. It is recommended that the PG quantity be an integral power of 2. The data volume of default.rgw.buckets.data is much larger than other storage pools and therefore more PGs need to be allocated to this storage pool.
  4. Modify the CRUSH rules of all storage pools.
    1
    		 
    ceph osd pool set default.rgw.buckets.data crush_rule rule-ssd