zvans18
Dabbler
- Joined
- Sep 6, 2016
- Messages
- 23
TL;DR for the purposes of remaking it with proper, larger ashift, is there any way to safely remove a special vdev mirror if it is ashift 9 and the rest of the pool mirrors are ashift 12? (either within TN or possibly utilizing a different boot disk with a more up to date ZFS version to make changes)
I added a special mirror via GUI about 5 months ago and only recently noticed the 250GB 970 EVO Plus reports itself as 512b and the special vdev was made with ashift 9 despite the ashift min (which to the best of my understanding has been implemented for some time). The 12 HDD mirrors established previously are all ashift 12 512e drives. Searching has only pulled up mixed results on if removing will hose the pool.
The drives aren't being completely hammered, but they have about 30TBW after 5 months or a little under 1DWPD (still double the warranty rate). What is the best path forward? Manually add a new special while forcing higher ashift, then remove the old to transfer metatdata? Or if this is currently unsafe, will this process be made safer in the future for GUI manipulation? Boot into a different OS, make changes, reimport pool? Just suck it up with larger disks?
I actually have 2 2TB 970 EVO Plus drives on the way, and even though the special was added later in the pool's life, peak utilization did go over 200GB. I figure worst case, if the special cannot be safely removed yet, I can combat write amplification with raw endurance (as well as be more appropriately sized for up to 150TB of data after recently increasing drive capacity)
Recreating the pool is not ideal simply due to the amount of data on it (~100TB). The smaller amount of irreplaceable data is already a backup of what is on my primary PC (which also has an offsite backup), but the majority of it is a media library. While all this data is replaceable, it will be very time and energy intensive to reacquire and sort. Technically I could break my mirrors for another pool and then recreate degraded mirrors and fill them back out once data is back, but this makes me nervous.
Finally, I still need to do a round of flashbench or something similar to try and determine the actual erase block size (4k? 8k?) as I have only seen speculation and an identical unanswered question in the Russian TN forum.
Full hardware details are in signature.
I added a special mirror via GUI about 5 months ago and only recently noticed the 250GB 970 EVO Plus reports itself as 512b and the special vdev was made with ashift 9 despite the ashift min (which to the best of my understanding has been implemented for some time). The 12 HDD mirrors established previously are all ashift 12 512e drives. Searching has only pulled up mixed results on if removing will hose the pool.
The drives aren't being completely hammered, but they have about 30TBW after 5 months or a little under 1DWPD (still double the warranty rate). What is the best path forward? Manually add a new special while forcing higher ashift, then remove the old to transfer metatdata? Or if this is currently unsafe, will this process be made safer in the future for GUI manipulation? Boot into a different OS, make changes, reimport pool? Just suck it up with larger disks?
I actually have 2 2TB 970 EVO Plus drives on the way, and even though the special was added later in the pool's life, peak utilization did go over 200GB. I figure worst case, if the special cannot be safely removed yet, I can combat write amplification with raw endurance (as well as be more appropriately sized for up to 150TB of data after recently increasing drive capacity)
Recreating the pool is not ideal simply due to the amount of data on it (~100TB). The smaller amount of irreplaceable data is already a backup of what is on my primary PC (which also has an offsite backup), but the majority of it is a media library. While all this data is replaceable, it will be very time and energy intensive to reacquire and sort. Technically I could break my mirrors for another pool and then recreate degraded mirrors and fill them back out once data is back, but this makes me nervous.
Finally, I still need to do a round of flashbench or something similar to try and determine the actual erase block size (4k? 8k?) as I have only seen speculation and an identical unanswered question in the Russian TN forum.
Full hardware details are in signature.
