一.检查本地磁盘
-
检查一下磁盘坏道
badblocks -b 4096 -sw /dev/sd*-w写入测试-s显示进度-o指定输出结果的文件
-
查看本地硬盘id.
不建议使用/dev/sd*,如果硬盘位置改变了,参数就对不上了
cat /dev/disk/by-id/
500g nvme
lrwxrwxrwx 1 root root 13 3月 25 12:02 nvme-Samsung_SSD_970_EVO_500GB_S5H7NS0NB28302N -> ../../nvme2n1
lrwxrwxrwx 1 root root 13 3月 25 12:02 nvme-Samsung_SSD_970_EVO_500GB_S5H7NS1NB24625E -> ../../nvme1n1
es.1T
lrwxrwxrwx 1 root root 9 3月 25 23:56 ata-ST1000DM003-1ER162_Z4Y1KJ8K -> ../../sda
lrwxrwxrwx 1 root root 9 3月 25 23:55 ata-ST1000DM003-1ER162_S4Y3QLMG -> ../../sdb
lrwxrwxrwx 1 root root 9 3月 25 23:55 ata-WDC_WD10EZEX-00BN5A0_WD-WCC3F0117592 -> ../../sdc
lrwxrwxrwx 1 root root 9 3月 25 23:56 ata-ST2000DM001-1ER164_W4Z02B74 -> ../../sdd
3t
lrwxrwxrwx 1 root root 9 3月 25 17:36 ata-ST3000DM001-1ER166_W500AP6C -> ../../sde
lrwxrwxrwx 1 root root 9 3月 25 23:57 ata-ST3000DM001-1ER166_W50087XE -> ../../sdf
1t
lrwxrwxrwx 1 root root 9 3月 25 23:57 ata-ST1000NM0033-9ZM173_Z1W29QQE -> ../../sdg
lrwxrwxrwx 1 root root 9 3月 25 17:37 ata-ST31000524NS_9WK3HXDS -> ../../sdh
使用别名重命名硬盘:
vim /etc/zfs/vdev_id.con
forexample:
alias o1 /dev/disk/ata-ST1000DM003-1ER162_S4Y3QLMG保存后
zpool create rename 01等效与zpool create rename ata-ST1000DM003-1ER162_S4Y3QLMG
二.建立zfs
1.建立mirror(和raid1差不多)
因为有两个盘是我老大给我的,信的过的东西,所有就用这两个盘来镜像另外两个盘,用作数据备份.
zpool create Joe ata-ST1000DM003-1ER162_Z4Y1KJ8K ata-ST1000DM003-1ER162_S4Y3QLMG mirror ata-ST1000NM0033-9ZM173_Z1W29QQE ata-ST31000524NS_9WK3HXDS -fcreate后面跟池的名字,例如Joemirror镜像
查看池状态:
➜ zpool status
pool: Joe
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
Joe ONLINE 0 0 0
ata-ST1000DM003-1ER162_Z4Y1KJ8K ONLINE 0 0 0
ata-ST1000DM003-1ER162_S4Y3QLMG ONLINE 0 0 0
mirror-2 ONLINE 0 0 0
ata-ST1000NM0033-9ZM173_Z1W29QQE ONLINE 0 0 0
ata-ST31000524NS_9WK3HXDS ONLINE 0 0 0
errors: No known data errors设置数据压缩类型:
➜ zpool list -v
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
Joe 2.72T 444K 2.72T - - 0% 0% 1.00x ONLINE -
ata-ST1000DM003-1ER162_Z4Y1KJ8K 928G 220K 928G - - 0% 0.00% - ONLINE
ata-ST1000DM003-1ER162_S4Y3QLMG 928G 172K 928G - - 0% 0.00% - ONLINE
mirror 928G 52K 928G - - 0% 0.00% - ONLINE
ata-ST1000NM0033-9ZM173_Z1W29QQE - - - - - - - - ONLINE
ata-ST31000524NS_9WK3HXDS - - - - - - - - ONLINE
➜ zfs set compression=lz4 Joe启用压缩参数:
➜ zfs get compressratio Joe
NAME PROPERTY VALUE SOURCE
Joe compressratio 1.00x -2.建立raidz
zpool create pt raidz ata-WDC_WD10EZEX-00BN5A0_WD-WCC3F0117592 ata-ST2000DM001-1ER164_W4Z02B74 ata-ST3000DM001-1ER166_W500AP6C ata-ST3000DM001-1ER166_W50087XE -f三.速度测试
写入2g数据测试
➜ dd if=/dev/zero of=/Joe/test1.img bs=2G count=1 oflag=dsync
0+1 records in
0+1 records out
2147479552 bytes (2.1 GB, 2.0 GiB) copied, 1.2736 s, 1.7 GB/s
➜ dd if=/Joe/test1.img of=/dev/zero bs=8k
262143+1 records in
262143+1 records out
2147479552 bytes (2.1 GB, 2.0 GiB) copied, 6.6709 s, 322 MB/s
➜ dd if=/dev/zero of=/pt/test1.img bs=50G count=1 oflag=dsync0+1 records in
0+1 records out
2147479552 bytes (2.1 GB, 2.0 GiB) copied, 6.834 s, 314 MB/s
➜ dd if=/pt/test1.img of=/dev/zero bs=8k262143+1 records in
262143+1 records out
2147479552 bytes (2.1 GB, 2.0 GiB) copied, 0.893735 s, 2.4 GB/s后来看到pt这个名字有点懵了. 那就是代表我把他只是存放一些经常读写但是不重要的文件.
so, 重建为raid0
➜ zpool destroy pt
➜ zpool create pt ata-WDC_WD10EZEX-00BN5A0_WD-WCC3F0117592 ata-ST2000DM001-1ER164_W4Z02B74 ata-ST3000DM001-1ER166_W500AP6C ata-ST3000DM001-1ER166_W50087XE -f
查看状态
➜ zpool list
NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
Joe 2.72T 590K 2.72T - - 0% 0% 1.00x ONLINE -
pt 8.16T 468K 8.16T - - 0% 0% 1.00x ONLINE -检查所有pool的情况
➜ zpool status -x
all pools are healthy四.添加高速缓存(L2ARC)
添加nvme作为高速缓存
➜ zpool add pt log nvme-Samsung_SSD_970_EVO_500GB_S5H7NS0NB28302N \\ 写入 写入是可以镜像的
➜ zpool add pt cache nvme-Samsung_SSD_970_EVO_500GB_S5H7NS1NB24625E \\读取 读取不可以镜像速度测试
➜ dd if=/dev/zero of=/pt/test1.img bs=10G count=2 oflag=dsync
dd: warning: partial read (2147479552 bytes); suggest iflag=fullblock
0+2 records in
0+2 records out
4294959104 bytes (4.3 GB, 4.0 GiB) copied, 2.37983 s, 1.8 GB/s
➜ dd if=/pt/test1.img of=/dev/zero bs=8k
524287+0 records in
524287+0 records out
4294959104 bytes (4.3 GB, 4.0 GiB) copied, 13.574 s, 316 MB/s感觉几个hdd组raid0后,有没有高速缓存都差不多,所以移除了缓存
➜ zpool remove pt cache nvme-Samsung_SSD_970_EVO_500GB_S5H7NS1NB24625E
➜ zpool add pt log nvme-Samsung_SSD_970_EVO_500GB_S5H7NS0NB28302N五.修改zfs缓存(ARC)
查看系统最大内存使用设置
cat /sys/module/zfs/parameters/zfs_arc_max我暂时分配了
20g内存作为缓存,注意这里使用的是字节
echo '20971520' > /sys/module/zfs/parameters/zfs_arc_max