dm-cache实现之LVM

The cache logical volume type uses a small and fast LV to improve the performance of a large and slow LV. It does this by storing the frequently used blocks on the faster LV. LVM refers to the small fast LV as a cache pool LV. The large slow LV is called the origin LV.

Due to requirements from dm-cache (the kernel driver), LVM further splits the cache pool LV into two devices:

  1. the cache data LV
    • The cache data LV is where copies of data blocks are kept from the origin LV to increase speed.
  2. cache metadata LV
    • The cache metadata LV holds the accounting information that specifies where data blocks are stored 1). 2)

概览:

  • 利用底层dm-cache驱动,实现ssd做为后端hdd的缓存,极大提高lvm性能;
  • dm-cache的实现方式有2种:
    • lvm 实现,本文类型;
    • dmsetup 实现,过程较为复杂,需手动计算metadata的block数目,本文暂不涉及;
  • 所有涉及的LVs必须在同一VG;

  • OriginLV: large slow LV, lv_data;
  • CacheDataLV: small fast LV for cache pool data, lv_Cache;
  • CacheMetaLV: small fast LV for cache pool metadata, lv_CacheMeta;
  • CachePoolLV: CacheDataLV + CacheMetaLV, lv_Cache ,CachePool建立后会用‘CacheDataLV’的名字;
  • CacheLV: OriginLV + CachePoolLV, lv_data,原来的‘OriginLV’会变成‘lv_data_corig’,并且隐藏;

  • HDD: sdc,sdd
  • SSD: sdb

3.1 create OriginLV

创建后端由HDD组成的逻辑卷 lv_data.

# 创建pv
pvcreate /dev/sd{c..d} /dev/sdb
# 先只将HDD加入vg并分配全部空间
vgcreate vg_data /dev/sd{c..d}
vgs
lvcreate -n lv_data -l 100%FREE vg_data
lvs

3.2 extend vg

# 注意ssd的4K对齐
# 将ssd容量扩展到 vg_data
vgextend vg_data /dev/sdb

3.3 create CacheMetaLV

在ssd的容量内,创建用于记录元数据的 lv_CacheMeta .

# manpage说明 lv_CacheMeta 要比lv_Cache的1/1000还要小,且最小为8M
lvcreate -n lv_CacheMeta -L 1G vg_data /dev/sdb
lvs

3.4 create CacheDataLV

在ssd的容量内,创建用于Cache的 lv_Cache.

# 创建 lv_Cache,注意仅使用98%,不然lvconver会报空间不足.
# manpage说明 lv_CacheMeta 要比lv_Cache的1/1000还要小,但是我想的话MetaData多给点更好吧...
# lvm来做算简单的,dmsetup还要自己去算大小,比这个麻烦.
lvcreate -n lv_Cache -l +98%FREE vg_data /dev/sdb

3.5 create CachePoolLV

创建 CachePoolLV, 这一步比较难理解,这是dm-cache 底层驱动设计的,简单讲就是把前面两步在ssd上创建的lv做成一个 CachePool.

# 将全面创建的2个lv转换成CachePool,中间会设计到lv的名称变化,可以留意观察下.
lvconvert --type cache-pool --chunksize 256 --poolmetadata vg_data/lv_CacheMeta vg_data/lv_Cache
# 创建 cache 类型的lv,并指定缓存策略为 writeback,默认为writethrough
lvconvert --type cache --cachemode writeback --chunksize 256 --cachepool vg_data/lv_Cache vg_data/lv_data
# 修改缓存策略
lvchange --cachemode writeback /dev/vg_data/lv_data
# 列出lv的详细信息
lvs -o+cache_mode,cache_policy,chunk_size
lvs -o cache_read_hits,cache_read_misses,cache_write_hits,cache_write_misses,cache_used_blocks,cache_total_blocks

lvcreate 一步创建

lvcreate --type cache -L 100G -n CachePool vg_data/lv_data /dev/sdb

3.6 挂载lv

mkfs.xfs /dev/vg_data/lv_data
mount /dev/vg_data/lv_data /mnt/data
df -h
# 重启后会显示inactive,处理:
lvchange -ay /dev/vg_data/lv_data

# 分离Cache和Origin
lvconvert --splitcache vg_data/lv_Cache
# 分离Cache和Origin,并删除Cache,以下2种方式:
lvremove vg_data/lv_Cache
lvconvert --uncache vg_data/lv_Cache
 
# 移除Cache和Origin
lvremove vg_data/lv_Cache


1)
e.g. on the origin LV or on the cache data LV
2)
Users should be familiar with these LVs if they wish to create the best and most robust cached logical volumes.
  • storage/缓存技术/dm-cache/dm-cache实现之lvm.txt
  • 最后更改: 2019/04/16 18:31
  • (外部编辑)