Documentation
Rate This Document
Findability
Accuracy
Completeness
Readability
SubmitClose

Configuring bcache

Before enabling smart prefetch, stop the cluster service program.

As an example, the following steps describe how to configure a bcache drive for the SDB drive.

Creating a bcache Drive for the Data Drive

  1. Delete the drive data, create a bcache drive, and format the bcache drive to the ext4 file system format.
    wipefs -a /dev/sdb --force
make-bcache -B /dev/sdb --wipe-bcache
mkfs.ext4 /dev/[bcache]
  2. Modify the bcache drive parameters.
    1. Obtain the actual parameter of max_sectors_kb (maximum request size allowed by the device).
      cat /sys/block/sdb/queue/max_sectors_kb

      The following information is displayed:

      [Maximum number of sectors on a data drive] (for example, 1024)
    2. Obtain the actual parameter of read_ahead_kb (size of the read-ahead data).
      cat /sys/block/sdb/queue/read_ahead_kb
      The following information is displayed:
      [Number of data drives to be read ahead] (for example, 128)
    3. Set max_sectors_kb and read_ahead_kb based on the obtained parameters.
      echo Maximum number of sectors on a data drive > /sys/block/[bcache]/queue/max_sectors_kb
      echo Number of data drives to be read ahead > /sys/block/[bcache]/queue/read_ahead_kb

      For Maximum number of sectors on a data drive, Number of data drives to be read ahead, and [bcache], note the following:

      • Maximum number of sectors on a data drive and Number of data drives to be read ahead are subject to the actual values. Different data drives may have different parameters.
      • [bcache] is subject to the content of the /sys/block/ directory. All [bcache] devices in the directory must be processed in the same way.

Creating a bcache Drive for the RAM Drive

  1. Create a RAM drive.
    modprobe brd rd_nr=12 rd_size=$((1048576))

    In the command, 12 indicates the number of cache drives, and 1048576 indicates the cache drive size (1 GB), which is appropriate for general big data scenarios.

  2. Format the cache drive into a bcache drive.
    make-bcache --wipe-bcache -b 262144 -C /dev/ram0

    In the command, 262144 indicates that the block size of the cache drive is 256 KB.

  3. Mount the cache drive to the bcache drive.
    echo $uuid > /sys/block/[bcache]/bcache/attach
    In the command, the value of $uuid is the value of uuid corresponding to cset.uuid in the command output. Show the value of $uuid:
    bcache-super-show /dev/ram0
  4. Modify the bcache policy.
    echo writearound > /sys/block/[bcache]/bcache/cache_mode
    echo 1 > /sys/block/[bcache]/bcache/clear_stats
    echo 0 > /sys/block/[bcache]/bcache/readahead
    echo 10 > /sys/block/[bcache]/bcache/writeback_percent
    echo 0 > /sys/block/[bcache]/bcache/cache/congested_read_threshold_us
    echo 0 > /sys/block/[bcache]/bcache/cache/congested_write_threshold_us
    echo 1 > /sys/block/[bcache]/bcache/read_bypass
    echo lru > /sys/block/[bcache]/bcache/cache/cache0/cache_replacement_policy

    [bcache] is subject to the content of the /sys/block/ directory. All [bcache] devices in the directory must be processed in the same way.

After a cache drive is created for the RAM drive, the system allocates a memory space to the bcache. If services require a large memory space, increase the service memory space to prevent the services from failing due to insufficient memory. For example, Spark Kmeans consumes a large amount of memory. Adjust the memory resource configuration of Yarn to ensure that services will not fail due to insufficient memory.

Creating a bcache Drive for the Cache SSD

  1. Ensure that there is an idle cache SSD.

    For example:

  2. Format the cache drive into a bcache drive.
    make-bcache --wipe-bcache -b 262144 -C /dev/nvme0n1p1
    echo /dev/nvme0n1p1 > /sys/fs/bcache/register

    In the command, 262144 indicates that the block size of the cache drive is 256 KB.

  3. Mount the cache drive to the bcache drive.
    echo $uuid > /sys/block/[bcache]/bcache/attach
    In the command, the value of $uuid is the value of uuid corresponding to cset.uuid in the command output. Show the value of $uuid:
    bcache-super-show /dev/nvme0n1p1
  4. Modify the bcache policy.
    echo writeback > /sys/block/[bcache]/bcache/cache_mode
echo 1 > /sys/block/[bcache]/bcache/clear_stats
echo 0 > /sys/block/[bcache]/bcache/readahead
echo 10 > /sys/block/[bcache]/bcache/writeback_percent
echo 0 > /sys/block/[bcache]/bcache/cache/congested_read_threshold_us
echo 0 > /sys/block/[bcache]/bcache/cache/congested_write_threshold_us
echo 0 > /sys/block/[bcache]/bcache/cache/read_bypass
echo 0 > /sys/block/[bcache]/bcache/sequential_cutoff
echo lru > /sys/block/[bcache]/bcache/cache/cache0/cache_replacement_policy
  5. Modify the queue directory parameters of the bcache.
    In the following command, set the value of max_sectors_kb to a proper value based on the back-end drive configuration.
    echo 1024 > /sys/block/[bcache]/queue/max_sectors_kb

    [bcache] is subject to the content of the /sys/block/ directory. All [bcache] devices in the directory must be processed in the same way.

Mounting the bcache Drive of the Data Drive

  1. Manually mount the data drive of the bcache drive.
    mount /dev/[bcache]/ data/[data]

    Set [bcache] and [data] based on the actual data drive directory.

After creating the bcache drive, in /etc/fstab, delete the configuration of mounting the data drive upon system boot. Otherwise, the data drive is formatted into a non-bcache drive after the server is rebooted.