toyebox
Explorer
- Joined
- Aug 20, 2016
- Messages
- 87
Good day all,
The machine i'm working with is as follows:
FreeNAS 11.3-U1
Xeon e3-1275v6
32G ECC memory
LSI 9260-8I (IT mode)
sas expander (RES2SV240)
Pool1 - 10x 4TB Seagate NAS drives (RaidZ-2)
Pool2 - 4x 10TB WD100EMAZ drives (RaidZ)
Long story short, i obtained the 4x 10TB drives temporarily to rebuild pool1. Pool2 has 20TB of large file data on it that i am currently replicating to Pool1. Pool1 has compression off, sync off, and record size set to 1M.
The issue i'm seeing when watching "zpool iostat 1" is that the reads(Pool2) and writes(Pool1) are sporadic and irregular. They will peak around 300MB/s (both read and write) then drop to 200, 100, and then about 3 seconds of 0MB/s reads/writes. I'm currently at 9.5TB's copied, using zfs send/recv (exact command: zfs send -vL pool2/videos@sending | zfs recv -s pool1/videos)
I've tested both pools with a test dataset with compression off, using dd writing and reading 50G of data with the results as follows:
pool1 - write: 650MB/s read: 910MB/s
pool2 - write 420MB/s read: 550MB/s
This just doesn't seem normal to me. Drive and CPU temps are normal, CPU load is around 5%. I have tried this replication on FreeNAS 11.2-U7 as well as remaking pool1 on 11.2-U7 (for consistency of pool features).
Currently Pool1 was created on 11.3-U1 and Pool2 was created on 11.2-U7 (not upgraded). I doubt this would make a difference though.
Any help is appreciated. I just don't see this as normal behavior, but maybe it is. I have been using FreeNAS for 9 years with various pool sizes (4 drives - 96 drives) and I just don't ever remember this being an issue. Thank you again in advance!
EDIT: I just looked at the latency for all the drives, and two of my drives have an average of 40ish ms for latency, where the rest are around 2ms. The drives are ST4000LM024 drives which i cannot find if they are SMR or not. they are 2.5" drives unfortunately. I have a feeling this is where my problem lies, specifically with writing if they are SMR drives :/. Can anyone concur?
The machine i'm working with is as follows:
FreeNAS 11.3-U1
Xeon e3-1275v6
32G ECC memory
LSI 9260-8I (IT mode)
sas expander (RES2SV240)
Pool1 - 10x 4TB Seagate NAS drives (RaidZ-2)
Pool2 - 4x 10TB WD100EMAZ drives (RaidZ)
Long story short, i obtained the 4x 10TB drives temporarily to rebuild pool1. Pool2 has 20TB of large file data on it that i am currently replicating to Pool1. Pool1 has compression off, sync off, and record size set to 1M.
The issue i'm seeing when watching "zpool iostat 1" is that the reads(Pool2) and writes(Pool1) are sporadic and irregular. They will peak around 300MB/s (both read and write) then drop to 200, 100, and then about 3 seconds of 0MB/s reads/writes. I'm currently at 9.5TB's copied, using zfs send/recv (exact command: zfs send -vL pool2/videos@sending | zfs recv -s pool1/videos)
I've tested both pools with a test dataset with compression off, using dd writing and reading 50G of data with the results as follows:
pool1 - write: 650MB/s read: 910MB/s
pool2 - write 420MB/s read: 550MB/s
This just doesn't seem normal to me. Drive and CPU temps are normal, CPU load is around 5%. I have tried this replication on FreeNAS 11.2-U7 as well as remaking pool1 on 11.2-U7 (for consistency of pool features).
Currently Pool1 was created on 11.3-U1 and Pool2 was created on 11.2-U7 (not upgraded). I doubt this would make a difference though.
Any help is appreciated. I just don't see this as normal behavior, but maybe it is. I have been using FreeNAS for 9 years with various pool sizes (4 drives - 96 drives) and I just don't ever remember this being an issue. Thank you again in advance!
EDIT: I just looked at the latency for all the drives, and two of my drives have an average of 40ish ms for latency, where the rest are around 2ms. The drives are ST4000LM024 drives which i cannot find if they are SMR or not. they are 2.5" drives unfortunately. I have a feeling this is where my problem lies, specifically with writing if they are SMR drives :/. Can anyone concur?
Last edited:
