Recently we've noticed that during the monthly scrub, for the first hour or so it slams the disks to 100% increasing latency and taking the site down. Specs:
Motherboard: Supermicro X10DRH
CPU: Xeon E5-2620 v3 @ 2.40GHz (12 cores)
Ram: 64GB ECC
OS disk: 2 x Kingston DataTraveler 3.0 64GB mirrored
Data disk: 12 x HGST Ultrastar 7K6000 4 TB SAS 12Gb/s (6 x 2 disk mirrored vdevs)
FreeNAS 11.2-U5
This was with
Pool is at 76% utilization and was 33% fragmented last I checked.
Scrubs and SMART long tests seem to be taking longer each month to where I can only do one of each otherwise they'll overlap. I wonder if the utilization + fragmentation is contributing to this?
I just ordered a new server with 12 x 14TB drives and SSDs for OS. This will put us at 44% utilization.
Until we get this server installed, I'm wondering if there is any way to throttle down whatever the scrub is doing in that first hour. It doesn't seem to respect scrub_delay at all. Any help here would be much appreciated.
Motherboard: Supermicro X10DRH
CPU: Xeon E5-2620 v3 @ 2.40GHz (12 cores)
Ram: 64GB ECC
OS disk: 2 x Kingston DataTraveler 3.0 64GB mirrored
Data disk: 12 x HGST Ultrastar 7K6000 4 TB SAS 12Gb/s (6 x 2 disk mirrored vdevs)
FreeNAS 11.2-U5
This was with
vfs.zfs.scrub_delay: 40
as well. I've since set it back to the default of 4 because I noticed that after the initial spike, the additional load is manageable (and scrubs take over a week otherwise).Pool is at 76% utilization and was 33% fragmented last I checked.
Scrubs and SMART long tests seem to be taking longer each month to where I can only do one of each otherwise they'll overlap. I wonder if the utilization + fragmentation is contributing to this?
I just ordered a new server with 12 x 14TB drives and SSDs for OS. This will put us at 44% utilization.
Until we get this server installed, I'm wondering if there is any way to throttle down whatever the scrub is doing in that first hour. It doesn't seem to respect scrub_delay at all. Any help here would be much appreciated.