■
HDD交換前{2TB,2.5TB,3TB}=>4TB@RAID5
HDD交換後{3TB,2.5TB,3TB}=>5TB@RAID5
アンマウント
umount /mnt/raid5/
オンライン拡張
mdadm --grow /dev/md126 -z max
時間がかかる
オンライン再構築後
# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] md126 : active raid5 sdc1[5] sdd1[3] sde1[4] 4883515392 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] bitmap: 0/10 pages [0KB], 131072KB chunk md127 : active raid1 sda1[3] sdb1[2] 2930134016 blocks super 1.2 [2/2] [UU] bitmap: 2/22 pages [8KB], 65536KB chunk unused devices:
ディスクチェック
# e2fsck -f /dev/md126
e2fsck 1.42.9 (28-Dec-2013) Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information /dev/md126: 6506/244178944 files (11.0% non-contiguous), 785230172/976691200 blocks
拡張
# resize2fs /dev/md126
resize2fs 1.42.9 (28-Dec-2013) Resizing the filesystem on /dev/md126 to 1220878848 (4k) blocks. The filesystem on /dev/md126 is now 1220878848 blocks long.
# mdadm --detail /dev/md126
/dev/md126: Version : 1.2 Creation Time : Sun Feb 5 00:22:35 2017 Raid Level : raid5 Array Size : 4883515392 (4657.28 GiB 5000.72 GB) Used Dev Size : 2441757696 (2328.64 GiB 2500.36 GB) Raid Devices : 3 Total Devices : 3 Persistence : Superblock is persistent Intent Bitmap : Internal Update Time : Fri Sep 6 09:54:36 2019 State : clean Active Devices : 3 Working Devices : 3 Failed Devices : 0 Spare Devices : 0 Layout : left-symmetric Chunk Size : 512K Consistency Policy : unknown Name : serv3:127 (local to host serv3) UUID : 59eb6cbf:57782701:f034a4f7:b9427151 Events : 98346 Number Major Minor RaidDevice State 3 8 49 0 active sync /dev/sdd1 5 8 33 1 active sync /dev/sdc1 4 8 65 2 active sync /dev/sde1
成功している。
マウント
# mount /dev/md126 /mnt/raid5/
RAID5を拡張。
昨日、RAID5(2TB,2.5TB,3TB)から不良セクタができていた
2TBを新品の3TBに変えた。交換前のRAID容量は最小の2TBに準じて
4TBだったが、今回最小容量が2.5TBになったのでRAIDとしては
5TBになるはず。
# parted /dev/md126 print
モデル: Linux Software RAID Array (md) ディスク /dev/md126: 4001GB セクタサイズ (論理/物理): 512B/4096B パーティションテーブル: loop ディスクフラグ: 番号 開始 終了 サイズ ファイルシステム フラグ 1 0.00B 4001GB 4001GB ext4
# umount /mnt/raid5/
# mdadm --grow /dev/md126 -z max
mdadm: component size of /dev/md126 has been set to 2441757696K
# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] md126 : active raid5 sdc1[5] sdd1[3] sde1[4] 4883515392 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] [================>....] resync = 82.6% (2017001304/2441757696) finish=110.7min speed=63937K/sec bitmap: 3/10 pages [12KB], 131072KB chunk md127 : active raid1 sda1[3] sdb1[2] 2930134016 blocks super 1.2 [2/2] [UU] bitmap: 2/22 pages [8KB], 65536KB chunk unused devices:
なかなか終わらない。
RAID5のストレージを交換。
# cat /var/log/messages | grep sdc
Sep 4 00:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 00:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 00:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 00:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 01:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 01:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 01:32:39 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 01:32:39 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 02:02:39 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 02:02:39 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 02:32:39 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 02:32:39 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 03:02:39 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 03:02:39 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 03:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 03:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 04:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 04:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 04:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 04:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 05:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 05:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 05:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 05:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 06:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 06:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 06:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 06:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 07:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 07:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 07:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 07:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 08:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 08:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 08:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 08:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 09:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 09:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 09:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 09:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 10:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 10:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 10:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 10:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 11:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 11:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 11:32:39 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 11:32:39 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 12:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 12:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 12:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 12:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 13:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 13:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 13:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 13:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 14:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 14:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 14:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 14:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 15:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 15:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 15:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 15:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 16:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 16:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 16:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 16:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 17:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 17:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 17:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 18:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 18:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 18:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 18:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 19:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 19:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 19:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 19:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 20:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 20:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 20:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 20:32:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors Sep 4 21:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 4 Currently unreadable (pending) sectors Sep 4 21:02:38 serv3 smartd[3868]: Device: /dev/sdc [SAT], 2 Offline uncorrectable sectors
# umount /mnt/raid5
# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] md126 : active raid5 sdd1[3] sde1[4] sdc1[1] 3906764800 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] bitmap: 0/15 pages [0KB], 65536KB chunk md127 : active raid1 sdb1[2] sda1[3] 2930134016 blocks super 1.2 [2/2] [UU] bitmap: 1/22 pages [4KB], 65536KB chunk unused devices:
# mdadm --manage /dev/md126 --fail /dev/sdc1
mdadm: set /dev/sdc1 faulty in /dev/md126
# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] md126 : active raid5 sdd1[3] sde1[4] sdc1[1](F) 3906764800 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [U_U] bitmap: 0/15 pages [0KB], 65536KB chunk md127 : active raid1 sdb1[2] sda1[3] 2930134016 blocks super 1.2 [2/2] [UU] bitmap: 2/22 pages [8KB], 65536KB chunk unused devices:
# mdadm --manage /dev/md126 --remove /dev/sdc1
mdadm: hot removed /dev/sdc1 from /dev/md126
# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] md126 : active raid5 sdd1[3] sde1[4] 3906764800 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [U_U] bitmap: 0/15 pages [0KB], 65536KB chunk md127 : active raid1 sdb1[2] sda1[3] 2930134016 blocks super 1.2 [2/2] [UU] bitmap: 2/22 pages [8KB], 65536KB chunk unused devices:
# parted /dev/sdc print
モデル: ATA WDC WD20EZRX-00D (scsi) ディスク /dev/sdc: 2000GB セクタサイズ (論理/物理): 512B/4096B パーティションテーブル: gpt ディスクフラグ: 番号 開始 終了 サイズ ファイルシステム 名前 フラグ 1 1049kB 2000GB 2000GB ext4 raid
# shutdown -h
Terminated
# parted /dev/sdc print
エラー: /dev/sdc: ディスクラベルが認識できません。 モデル: ATA WDC WD30EZRZ-00G (scsi) ディスク /dev/sdc: 3001GB セクタサイズ (論理/物理): 512B/4096B パーティションテーブル: unknown ディスクフラグ:
# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] md126 : active raid5 sdd1[3] sde1[4] 3906764800 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [U_U] bitmap: 0/15 pages [0KB], 65536KB chunk md127 : active raid1 sda1[3] sdb1[2] 2930134016 blocks super 1.2 [2/2] [UU] bitmap: 0/22 pages [0KB], 65536KB chunk unused devices:
# smartctl -a /dev/sdc
=== START OF INFORMATION SECTION === Model Family: Western Digital Blue Device Model: WDC WD30EZRZ-00GXCB0 Serial Number: WD-WCC7******** LU WWN Device Id: 5 0014ee 20f7ecfbc Firmware Version: 80.00A80 User Capacity: 3,000,592,982,016 bytes [3.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 5400 rpm Form Factor: 3.5 inches Device is: In smartctl database [for details use: -P show] ATA Version is: ACS-3 T13/2161-D revision 5 SATA Version is: SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s) Local Time is: Wed Sep 4 21:46:59 2019 JST SMART support is: Available - device has SMART capability. SMART support is: Enabled
# parted /dev/sdc
GNU Parted 3.1 /dev/sdc を使用 GNU Parted へようこそ! コマンド一覧を見るには 'help' と入力してください。 (parted) mklabel gpt (parted) mkpart primary ext4 0% 100% (parted) set 1 raid on (parted) print モデル: ATA WDC WD30EZRZ-00G (scsi) ディスク /dev/sdc: 3001GB セクタサイズ (論理/物理): 512B/4096B パーティションテーブル: gpt ディスクフラグ: 番号 開始 終了 サイズ ファイルシステム 名前 フラグ 1 1049kB 3001GB 3001GB primary raid (parted) quit 通知: 必要であれば /etc/fstab を更新するのを忘れないようにしてください。
# mdadm --add /dev/md126 /dev/sdc1
mdadm: added /dev/sdc1 # cat /proc/mdstat Personalities : [raid1] [raid6] [raid5] [raid4] md126 : active raid5 sdc1[5] sdd1[3] sde1[4] 3906764800 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [U_U] [>....................] recovery = 0.0% (1656532/1953382400) finish=294.5min speed=110435K/sec bitmap: 0/15 pages [0KB], 65536KB chunk md127 : active raid1 sda1[3] sdb1[2] 2930134016 blocks super 1.2 [2/2] [UU] bitmap: 0/22 pages [0KB], 65536KB chunk unused devices:
RAID1の片ドライブが外れていたので復帰させてみた=>次の日RAID解消された
ふと確認してみたらRAID1が片肺運転だった。
$ cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] md126 : active raid5 sdd1[3] sde1[4] sdc1[1] 3906764800 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] bitmap: 0/15 pages [0KB], 65536KB chunk md127 : active raid1 sda1[3] 2930134016 blocks super 1.2 [2/1] [U_] bitmap: 10/22 pages [40KB], 65536KB chunk unused devices:
sdbのSMARTを確認。
# smartctl -a /dev/sdb
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-3.10.0-514.6.1.el7.x86_64] (local build) Copyright (C) 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Device Model: WDC WD30EZRZ-00Z5HB0 Serial Number: WD-WCC4N7ES0157 LU WWN Device Id: 5 0014ee 2b8db729e Firmware Version: 80.00A80 User Capacity: 3,000,592,982,016 bytes [3.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 5400 rpm Device is: Not in smartctl database [for details use: -P showall] ATA Version is: ACS-2 (minor revision not indicated) SATA Version is: SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s) Local Time is: Wed Sep 12 17:54:24 2018 JST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED <<省略>> ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 124 3 Spin_Up_Time 0x0027 215 177 021 Pre-fail Always - 4241 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 14 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 4 7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 10 9 Power_On_Hours 0x0032 081 081 000 Old_age Always - 14250 10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 14 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 3 193 Load_Cycle_Count 0x0032 001 001 000 Old_age Always - 963269 194 Temperature_Celsius 0x0022 120 111 000 Old_age Always - 30 196 Reallocated_Event_Count 0x0032 196 196 000 Old_age Always - 4 197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 45 198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 43 199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 96 SMART Error Log Version: 1 No Errors Logged <<以下略>>
14250時間=約19.5か月。
Reallocated_Event_Count(セクタ代替処理発生回数)=4回。
Current_Pending_Sector(代替処理保留中セクタ数)=45か所。
Offline_Uncorrectable(代替不能セクタ数)=43か所。
# dmesg | grep md127
<<抜粋>> [ 8.837958] md/raid1:md127: active with 1 out of 2 mirrors
とあるので、前回停電から再投入したときから片肺だったと思われる。
ふと、RAID1に復帰させたらどうなるか試したくなった。
# mdadm --add /dev/md127 /dev/sdb1
# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] md126 : active raid5 sdd1[3] sde1[4] sdc1[1] 3906764800 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/3] [UUU] bitmap: 0/15 pages [0KB], 65536KB chunk md127 : active raid1 sdb1[2] sda1[3] 2930134016 blocks super 1.2 [2/2] [UU] bitmap: 2/22 pages [8KB], 65536KB chunk unused devices:
復帰した。
追記。
次の日の夜に解消されていたのを発見しました。
素直に電源を落として同じ容量のHDDと交換しました。
RAID5の壊れたストレージを交換
/dev/sde(2TB)がSMARTエラー。
3.0TB WD-Blueを調達。
電源を落として物理的に交換。
# parted /dev/sde
(parted) mklabel 新しいディスクラベル? gpt (parted) mkpart パーティションの名前? []? ファイルシステムの種類? [ext2]? 開始? 0% 終了? 100% (parted) set 1 raid on (parted) print モデル: ATA WDC WD30EZRZ-00Z (scsi) ディスク /dev/sde: 3001GB セクタサイズ (論理/物理): 512B/4096B パーティションテーブル: gpt ディスクフラグ: 番号 開始 終了 サイズ ファイルシステム 名前 フラグ 1 1049kB 3001GB 3001GB raid (parted) q 通知: 必要であれば /etc/fstab を更新するのを忘れないようにしてください。
# cat /proc/mdstat
Personalities : [raid1] md126 : active raid1 sda1[0] sdb1[2] 2930134016 blocks super 1.2 [2/2] [UU] bitmap: 0/22 pages [0KB], 65536KB chunk md127 : inactive sdd1[3](S) sdc1[1](S) 4395140096 blocks super 1.2 unused devices:
RAID5を一旦停止。
# mdadm --stop /dev/md127
mdadm: stopped /dev/md127
RAID5アレイを組み立て直す。
# mdadm --assemble /dev/md127 /dev/sdc1 /dev/sdd1 /dev/sde1
mdadm: /dev/sde1 misses out due to wrong homehost mdadm: /dev/md127 assembled from 2 drives - not enough to start the array while not clean - consider --force.
組み立てられないのでforceオプションを試す。
# mdadm --assemble /dev/md127 /dev/sdc1 /dev/sdd1 /dev/sde1 --force
mdadm: /dev/sde1 misses out due to wrong homehost mdadm: Marking array /dev/md126 as 'clean' mdadm: /dev/md127 has been started with 2 drives (out of 3).
# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] md126 : active raid1 sdb1[2] sda1[0] 2930134016 blocks super 1.2 [2/2] [UU] bitmap: 0/22 pages [0KB], 65536KB chunk md127 : active raid5 sdd1[3] sdc1[1] 3906764800 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [UU_] bitmap: 0/15 pages [0KB], 65536KB chunk unused devices:
やっぱり組立られないけど、1台欠番状態で起動はできた。
この状態に新規ドライブを追加する。
# mdadm --add /dev/md127 /dev/sde1
mdadm: added /dev/sde1
追加に成功した。
# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] md126 : active raid1 sdb1[2] sda1[0] 2930134016 blocks super 1.2 [2/2] [UU] bitmap: 0/22 pages [0KB], 65536KB chunk md127 : active raid5 sde1[4] sdd1[3] sdc1[1] 3906764800 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] [UU_] [>....................] recovery = 0.0% (478940/1953382400) finish=271.8min speed=119735K/sec bitmap: 0/15 pages [0KB], 65536KB chunk unused devices: