在编译容器中编译Ceph

在配置好依赖的编译容器中编译Ceph,并把编译的RPM包拷贝到部署容器中用于镜像制作。

  1. 进入编译容器。

    1
    podman exec -it openeuler2203sp4_build /bin/bash
    

  2. “/home”目录下载并解压ceph-17.2.8源码。

    1
    2
    3
    cd /home
    wget https://download.ceph.com/tarballs/ceph-17.2.8.tar.gz --no-check-certificate
    tar zxvf ceph-17.2.8.tar.gz
    

    wget命令是从互联网下载文件,需要确保环境联网。

  3. 修改ceph.spec文件。

    1
    2
    3
    4
    5
    6
    7
    cd ceph-17.2.8
    sed -i 's/redhat-rpm-config/openEuler-rpm-config/g' ceph.spec
    sed -i 's#%if 0%{?fedora} || 0%{?rhel}#%if 0%{?fedora} || 0%{?rhel} || 0%{?openEuler}#' ceph.spec
    sed -i 's#%if 0%{?rhel} || 0%{?fedora}#%if 0%{?rhel} || 0%{?fedora} || 0%{?openEuler}#' ceph.spec
    sed -i 's#%if 0%{?fedora} || 0%{?suse_version} > 1500 || 0%{?rhel} == 9 || 0%{?openEuler}#%if 0%{?fedora} || 0%{?suse_version} > 1500 || 0%{?rhel} == 9#' ceph.spec
    sed -i '1a\%define _binaries_in_noarch_packages_terminate_build 0' ceph.spec
    sed -i '2a\%define _unpackaged_files_terminate_build 0' ceph.spec
    

  4. 编译Ceph。

    1
    2
    3
    4
    5
    cd ..
    tar cf ceph-17.2.8.tar.bz2 ceph-17.2.8/
    cp ceph-17.2.8/ceph.spec /root/rpmbuild/SPECS/
    cp ceph-17.2.8.tar.bz2 /root/rpmbuild/SOURCES/
    rpmbuild -bb /root/rpmbuild/SPECS/ceph.spec
    
    • 编译Ceph的过程中,至少需要100GB的磁盘空间存放临时文件,请注意为“/root/rpmbuild”目录预留充足空间。
    • 编译Ceph的过程中,需要确保环境联网。
    • Ceph 17.2.8版本会有OSD偶尔不稳定重启的异常,详见Ceph 17.2.8跑长稳时报错

  5. 编译selinux-policy依赖 (Ceph 17.2.8版本需要编译selinux-policy依赖,Ceph 17.2.7及以下版本可跳过此步骤)。

    1
    2
    3
    4
    5
    6
    7
    git clone https://gitee.com/src-openeuler/selinux-policy.git
    cd selinux-policy
    git checkout -b 2203sp3 origin/openEuler-22.03-LTS-SP3
    cd ..
    cp selinux-policy/* /root/rpmbuild/SOURCES/
    cp selinux-policy/selinux-policy.spec /root/rpmbuild/SPECS/
    rpmbuild -bb /root/rpmbuild/SPECS/selinux-policy.spec
    

  6. 将编译好的RPM包拷贝到“home”中。

    1
    2
    mv /root/rpmbuild/RPMS/aarch64 /home/local_rpm/
    mv /root/rpmbuild/RPMS/noarch /home/local_rpm/
    

  7. 在物理机上将编译好的RPM包导入到部署容器。

    1
    podman cp openeuler2203sp4_build:/home/local_rpm openeuler2203sp4_release:/home/
    

    若Ceph源码“src/osd/SnapMapper.cc”与fmt包的版本不匹配时,会出现编译报错,如图1 编译报错所示。报错时需要将“src/osd/SnapMapper.cc”文件中fmt::format相关代码行进行注释后重新编译。

    修改类似如下,共涉及4处修改。

    • 233行 - 234行
    • 272行 - 275行
    • 321行 - 324行
    • 327行 - 329行
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
        228 tl::expected<SnapMapper::object_snaps, Scrub::SnapMapReaderI::result_t>
        229 SnapMapper::get_snaps_common(const hobject_t &oid) const
        230 {
        231   ceph_assert(check(oid));
        232   set<string> keys{to_object_key(oid)};
        233 //  dout(20) << fmt::format("{}: key string: {} oid:{}", __func__, keys, oid)
        234 //         << dendl;
        235
        236   map<string, ceph::buffer::list> got;
    ...
        264 std::set<std::string> SnapMapper::to_raw_keys(
        265   const hobject_t &clone,
        266   const std::set<snapid_t> &snaps) const
        267 {
        268   std::set<std::string> keys;
        269   for (auto snap : snaps) {
        270     keys.insert(to_raw_key(snap, clone));
        271   }
        272 //  dout(20) << fmt::format(
        273 //              "{}: clone:{} snaps:{} -> keys: {}", __func__, clone, snaps,
        274 //              keys)
        275 //         << dendl;
        276   return keys;
        277 }
    ...
        305   std::set<snapid_t> snaps_from_mapping;
        306   for (auto &[k, v] : kvmap) {
        307     dout(20) << __func__ << " " << hoid << " " << k << dendl;
        308     // extract the object ID from the value fetched for an SNA mapping key
        309     auto [sn, obj] = SnapMapper::from_raw(v);
        310     if (obj != hoid) {
        311       dout(1) << fmt::format(
        312                    "{}: unexpected object ID {} for key{} (expected {})",
        313                    __func__, obj, k, hoid)
        314               << dendl;
        315       return tl::unexpected(result_t{code_t::inconsistent});
        316     }
        317     snaps_from_mapping.insert(sn);
        318   }
        319
        320   if (snaps_from_mapping != *obj_snaps) {
        321 //    dout(10) << fmt::format(
        322 //                "{}: hoid:{} -> mapper internal inconsistency ({} vs {})",
        323 //                __func__, hoid, *obj_snaps, snaps_from_mapping)
        324 //           << dendl;
        325     return tl::unexpected(result_t{code_t::inconsistent});
        326   }
        327  // dout(10) << fmt::format(
        328 //              "{}: snaps for {}: {}", __func__, hoid, snaps_from_mapping)
        329 //         << dendl;
        330   return obj_snaps;
        331 }
    
    图1 编译报错