Creating a Deployment Image

Install each RPM package in the deployment container to create an image with the minimum specifications.

1. Access the deployment container.

   1	podman exec -it openeuler2203sp4_release /bin/bash

2. Install the UCX packages.

   1 2	cd /home/local_rpm yum install ucx-1.14.1-1.aarch64.rpm ucx-cma-1.14.1-1.aarch64.rpm ucx-debuginfo-1.14.1-1.aarch64.rpm ucx-debugsource-1.14.1-1.aarch64.rpm ucx-devel-1.14.1-1.aarch64.rpm ucx-ib-1.14.1-1.aarch64.rpm ucx-rdmacm-1.14.1-1.aarch64.rpm ucx-static-1.14.1-1.aarch64.rpm -y

   

   Skip this step if only SPDK is required.

3. Install the Ceph dependency packages.

   1 2 3

   yum install python-asyncssh-help-2.7.0-2.noarch.rpm python3-asyncssh-2.7.0-2.noarch.rpm python3-natsort-8.4.0-3.noarch.rpm -y pip3 install kubernetes==18.20.0 pip install pycryptodome==3.19.1

   

   • To prevent deployment failures caused by earlier versions of the third-party Python library pycryptodome, you are advised to upgrade it as follows:

     1	pip install pycryptodome==3.19.1

   • To prevent the "No module named 'kubernetes.client.models.v1_event'" error during MGR deployment, you need to install a dependency package as follows:

     1	pip3 install kubernetes==18.20.0

4. If Ceph 17.2.8 and openEuler-22.03-LTS-SP3 are used, install the selinux-policy dependency before installing Ceph.
   

   • To obtain this dependency package, see 7.
   • When deploying Ceph 17.2.7, you need to install selinux-policy for openEuler-22.03-LTS-SP3 and earlier versions. If the OS version is openEuler-22.03-LTS-SP4, skip this step.

   1

   yum install noarch/selinux-policy-35.5-23.noarch.rpm noarch/selinux-policy-devel-35.5-23.noarch.rpm noarch/selinux-policy-help-35.5-23.noarch.rpm noarch/selinux-policy-minimum-35.5-23.noarch.rpm noarch/selinux-policy-mls-35.5-23.noarch.rpm noarch/selinux-policy-sandbox-35.5-23.noarch.rpm noarch/selinux-policy-targeted-35.5-23.noarch.rpm -y

5. Install Ceph.

   1 2 3

   yum install aarch64/ceph-17.2.7-0.aarch64.rpm aarch64/ceph-base-17.2.7-0.aarch64.rpm aarch64/ceph-common-17.2.7-0.aarch64.rpm aarch64/ceph-debugsource-17.2.7-0.aarch64.rpm aarch64/ceph-exporter-17.2.7-0.aarch64.rpm aarch64/ceph-fuse-17.2.7-0.aarch64.rpm aarch64/ceph-immutable-object-cache-17.2.7-0.aarch64.rpm aarch64/ceph-mds-17.2.7-0.aarch64.rpm aarch64/ceph-mgr-17.2.7-0.aarch64.rpm aarch64/ceph-mon-17.2.7-0.aarch64.rpm aarch64/ceph-osd-17.2.7-0.aarch64.rpm aarch64/ceph-radosgw-17.2.7-0.aarch64.rpm aarch64/ceph-selinux-17.2.7-0.aarch64.rpm aarch64/ceph-test-17.2.7-0.aarch64.rpm aarch64/cephfs-java-17.2.7-0.aarch64.rpm aarch64/cephfs-mirror-17.2.7-0.aarch64.rpm aarch64/libcephfs-devel-17.2.7-0.aarch64.rpm aarch64/libcephfs2-17.2.7-0.aarch64.rpm aarch64/libcephfs_jni-devel-17.2.7-0.aarch64.rpm aarch64/libcephfs_jni1-17.2.7-0.aarch64.rpm aarch64/libcephsqlite-17.2.7-0.aarch64.rpm aarch64/libcephsqlite-devel-17.2.7-0.aarch64.rpm aarch64/librados-devel-17.2.7-0.aarch64.rpm aarch64/librados2-17.2.7-0.aarch64.rpm aarch64/libradospp-devel-17.2.7-0.aarch64.rpm aarch64/libradosstriper-devel-17.2.7-0.aarch64.rpm aarch64/libradosstriper1-17.2.7-0.aarch64.rpm aarch64/librbd-devel-17.2.7-0.aarch64.rpm aarch64/librbd1-17.2.7-0.aarch64.rpm aarch64/librgw-devel-17.2.7-0.aarch64.rpm aarch64/librgw2-17.2.7-0.aarch64.rpm aarch64/python3-ceph-argparse-17.2.7-0.aarch64.rpm aarch64/python3-ceph-common-17.2.7-0.aarch64.rpm aarch64/python3-cephfs-17.2.7-0.aarch64.rpm aarch64/python3-rados-17.2.7-0.aarch64.rpm aarch64/python3-rbd-17.2.7-0.aarch64.rpm aarch64/python3-rgw-17.2.7-0.aarch64.rpm aarch64/rados-objclass-devel-17.2.7-0.aarch64.rpm aarch64/rbd-fuse-17.2.7-0.aarch64.rpm aarch64/rbd-mirror-17.2.7-0.aarch64.rpm aarch64/rbd-nbd-17.2.7-0.aarch64.rpm noarch/ceph-grafana-dashboards-17.2.7-0.noarch.rpm noarch/ceph-mgr-cephadm-17.2.7-0.noarch.rpm noarch/ceph-mgr-dashboard-17.2.7-0.noarch.rpm noarch/ceph-mgr-diskprediction-local-17.2.7-0.noarch.rpm noarch/ceph-mgr-k8sevents-17.2.7-0.noarch.rpm noarch/ceph-mgr-modules-core-17.2.7-0.noarch.rpm noarch/ceph-mgr-rook-17.2.7-0.noarch.rpm noarch/ceph-prometheus-alerts-17.2.7-0.noarch.rpm noarch/ceph-resource-agents-17.2.7-0.noarch.rpm noarch/ceph-volume-17.2.7-0.noarch.rpm noarch/cephadm-17.2.7-0.noarch.rpm noarch/cephfs-top-17.2.7-0.noarch.rpm -y yum install chrony haproxy keepalived -y

6. Install the closed source KSAL package.

   1 2 3

   cd /home unzip BoostKit-KSAL_1.10.0.zip rpm -ivh libksal-release-1.10.0.oe1.aarch64.rpm

   

   To obtain the KSAL software package, see Obtaining Software Packages. You are advised to upload the package to the /home directory.

7. SPDK requires user-mode huge pages. Therefore, privilege escalation is required for ceph-osd.

   1	setcap 'CAP_DAC_OVERRIDE+eip CAP_SYS_ADMIN+eip' /usr/bin/ceph-osd

8. Modify the Ceph user login configuration.
   1. Open the passwd file.

      1	vim /etc/passwd

   2. Press i to enter the insert mode and change the Ceph login shell to /bin/bash.

      

   3. Press Esc to exit the insert mode. Type :wq! and press Enter to save the file and exit.
9. Configure user resource limits (required for enabling UCX).
   1. Open the limits.conf file:

      1	vim /etc/security/limits.conf

   2. Press i to enter the insert mode and add the following content to the end of the file.

      1 2 3 4 5 6

      * soft nofile 1048576 * hard nofile 1048576 * soft nproc unlimited * hard nproc unlimited * soft memlock unlimited * hard memlock unlimited

   3. Press Esc to exit the insert mode. Type :wq! and press Enter to save the file and exit.
10. Commit the container as an image on the physical machine.

    1	podman commit 688247c8b260 [IP]:5000/ceph/ceph_release:v17.2.7

    

    • 688247c8b260 is the ID of the openeuler2203sp4_release container. You can run the podman ps command to query container IDs.
    • [IP] indicates the actual IP address of the local host.
    • Before committing the container, remove installed RPM packages from the /home directory to reduce the image size.
11. Export the deployment image.

    1	podman save -o ceph_release.tar a6e8aff2def8

    

    a6e8aff2def8 is the ID of the image committed in 10. You can run podman images to query image IDs.