部署MON节点

在ceph1节点执行部署MON（Monitor）节点的所有操作。

1. 创建集群。

   1 2	cd /etc/ceph ceph-deploy new ceph1 ceph2 ceph3

   回显如下所示。

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

   [ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (2.0.1): /home/xiaoshuang/0808/kunpeng_perfstudiokit/venv/bin/ceph-deploy new ceph1 ceph2 ceph3 [ceph_deploy.cli][INFO ] ceph-deploy options: [ceph_deploy.cli][INFO ] verbose : False [ceph_deploy.cli][INFO ] quiet : False [ceph_deploy.cli][INFO ] username : None [ceph_deploy.cli][INFO ] overwrite_conf : False [ceph_deploy.cli][INFO ] ceph_conf : None [ceph_deploy.cli][INFO ] cluster : ceph [ceph_deploy.cli][INFO ] mon : ['ceph1', 'ceph2', 'ceph3'] [ceph_deploy.cli][INFO ] ssh_copykey : True [ceph_deploy.cli][INFO ] fsid : None [ceph_deploy.cli][INFO ] cluster_network : None [ceph_deploy.cli][INFO ] public_network : None [ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf object at 0xfffd10452890> [ceph_deploy.cli][INFO ] default_release : False [ceph_deploy.cli][INFO ] func : <function new at 0xfffd1044e830> [ceph_deploy.new][DEBUG ] Creating new cluster named ceph [ceph_deploy.new][INFO ] making sure passwordless SSH succeeds [ceph1][DEBUG ] connected to host: ceph1

2. 修改相关配置。
   修改用于将内部集群间的网络与外部访问的网络隔离。配置192.168.4.0用于内部存储集群之间的数据同步（仅在服务端节点间使用），192.168.3.0用于服务端节点与客户端节点的数据交互。

   1. 打开“/etc/ceph”目录下自动生成的“ceph.conf”文件。
      

      配置节点命令以及使用ceph-deploy配置OSD时，需在“/etc/ceph”目录下执行，否则会报错。

      1	vi /etc/ceph/ceph.conf

   2. 按“i”键进入编辑模式，对文件内容进行如下修改。

      1 2 3 4 5 6 7 8 9 10 11 12 13

      [global] fsid = f5a4f55c-d25b-4339-a1ab-0fceb4a2996f mon_initial_members = ceph1, ceph2, ceph3 mon_host = 192.168.3.166,192.168.3.167,192.168.3.168 auth_cluster_required = cephx auth_service_required = cephx auth_client_required = cephx public_network = 192.168.3.0/24 cluster_network = 192.168.4.0/24 [mon] mon_allow_pool_delete = true

      

      Ceph 14.2.8版本在使用bluestore引擎的时候默认会打开bluefs的buffer开关，可能导致系统下内存全部被buff/cache占用，使性能下降。可以采用以下两种方案解决：

      • 在集群压力不大的场景下可以将“bluefs_buffered_io”开关设置成“false”。
      • 可以通过定时执行echo 3 > /proc/sys/vm/drop_caches来强制回收“buffer/cache”中的内存。
   3. 按“Esc”键退出编辑模式，输入:wq!并按“Enter”键保存退出文件。
3. 初始化监视器并收集密钥。

   1	ceph-deploy mon create-initial

   执行完成后，脚本会自动生成ceph.client.admin.keyring，回显如下所示。

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

   [ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (2.0.1): /usr/bin/ceph-deploy mon create-initial [ceph_deploy.cli][INFO ] ceph-deploy options: [ceph_deploy.cli][INFO ] username : None [ceph_deploy.cli][INFO ] verbose : False [ceph_deploy.cli][INFO ] overwrite_conf : False [ceph_deploy.cli][INFO ] subcommand : create-initial [ceph_deploy.cli][INFO ] quiet : False [ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0xfffcbf407f00> [ceph_deploy.cli][INFO ] cluster : ceph [ceph_deploy.cli][INFO ] func : <function mon at 0xfffcbf3e75d0> [ceph_deploy.cli][INFO ] ceph_conf : None [ceph_deploy.cli][INFO ] keyrings : None [ceph_deploy.cli][INFO ] default_release : False [ceph_deploy.mon][DEBUG ] Deploying mon, cluster ceph hosts ceph1 ceph2 ceph3 [ceph_deploy.mon][DEBUG ] detecting platform for host ceph1 ... [ceph1][DEBUG ] connected to host: ceph1 [ceph1][DEBUG ] detect platform information from remote host [ceph1][DEBUG ] detect machine type [ceph1][DEBUG ] find the location of an executable [ceph_deploy.mon][INFO ] distro info: openEuler 20.03 LTS-SP1 [ceph1][DEBUG ] determining if provided host has same hostname in remote [ceph1][DEBUG ] get remote short hostname [ceph1][DEBUG ] deploying mon to ceph1 [ceph1][DEBUG ] get remote short hostname [ceph1][DEBUG ] remote hostname: ceph1 [ceph1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [ceph1][DEBUG ] create the mon path if it does not exist

4. 将ceph.client.admin.keyring拷贝到各个节点上。

   1	ceph-deploy --overwrite-conf admin ceph1 ceph2 ceph3 client1 client2 client3

   执行成功回显如下所示。

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

   [ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (2.0.1): /usr/bin/ceph-deploy --overwrite-conf admin ceph1 ceph2 ceph3 client1 [ceph_deploy.cli][INFO ] ceph-deploy options: [ceph_deploy.cli][INFO ] username : None [ceph_deploy.cli][INFO ] verbose : False [ceph_deploy.cli][INFO ] overwrite_conf : True [ceph_deploy.cli][INFO ] quiet : False [ceph_deploy.cli][INFO ] cd_conf : <ceph_deploy.conf.cephdeploy.Conf instance at 0xfffd01a83f50> [ceph_deploy.cli][INFO ] cluster : ceph [ceph_deploy.cli][INFO ] client : ['ceph1', 'ceph2', 'ceph3', 'client1'] [ceph_deploy.cli][INFO ] func : <function admin at 0xfffd01c495d0> [ceph_deploy.cli][INFO ] ceph_conf : None [ceph_deploy.cli][INFO ] default_release : False [ceph_deploy.admin][DEBUG ] Pushing admin keys and conf to ceph1 [ceph1][DEBUG ] connected to host: ceph1 [ceph1][DEBUG ] detect platform information from remote host [ceph1][DEBUG ] detect machine type [ceph1][DEBUG ] write cluster configuration to /etc/ceph/{cluster}.conf [ceph_deploy.admin][DEBUG ] Pushing admin keys and conf to ceph2 [ceph2][DEBUG ] connected to host: ceph2

5. 查看是否配置成功。

   1

   ceph -s

   配置成功回显如下所示。

   1 2 3 4 5

   cluster: id: f6b3c38c-7241-44b3-b433-52e276dd53c6 health: HEALTH_OK services: mon: 3 daemons, quorum ceph1,ceph2,ceph3 (age 25h)