Storage Upgrade Advice

[cwxc818]

Hi Everyone,

I've been running the following setup for ~1.5yrs, and while it's been bulletproof (*knock on wood*), I'm running out of storage (~85% full).

MB: X10SL7-F
CPU: E3-1241 v3
HDD: 6x4TB HGST Deskstar NAS 7200RPM (RAID-Z2)
RAM: 32GB Crucial ECC
Boot: 2x16GB SanDisk Cruzer Fit CZ33
Case: NZXT Source 210 (https://www.newegg.com/Product/Product.aspx?Item=N82E16811146075) [I'd like to reuse this if possible]
(note that this has 8x3.5" bays and 3x5.25" bays so I purchased a 2x5.25" to 3x3.5" and 1x5.25" to 1x3.5" + 2x2.5" adapter so I have a total of 12x3.5" bays and 2x2.5" bays)

This is mostly for Plex, home picture/video backup, PC backups. Usually 1 user but sometimes up to 2. I've recently acquired 8x8TB WD Red/White 5400RPM NAS HDD's via WD Easystore's and I'm looking for input on what you would do to upgrade storage? I was thinking of these options (leaning towards #1 right now):

1.) Replace existing 4TB HDD's one by one with 8TB HDD's (increases storage from 16TB to 32TB, leaves room for a future _TBx6 RAID-Z2, can re-purpose existing 4TB HDD's for something else, leaves 2 spare 8TB HDD's, similar power consumption, similar performance, least overall storage initially, more flexibility later)
2.) Add another 6x8TB RAID-Z2 (increases storage from 16TB to 48TB, maxes out available 3.5" bays, leaves 2 spare 8TB HDD's, more power consumption, similar or better performance?, mid overall storage initially, mid flexibility later)
3.) Start over with 11(?)x8TB RAID-Z3 (increases storage from 16TB to 64TB, maxes out available 3.5" bays, need to buy more 8TB HDD's, more power consumption, worse performance?, most overall storage initially, least flexibility later)

I have a few additional questions/concerns:
- X10SL7-F is limited to 32GB RAM. Should I be concerned about this as I increase to 32TB+ of storage (1GB RAM per 1TB HDD rule)?
- X10SL7-F has 8xSAS2 ports (via LSI), 4xSATA3.0, 2xSATA6.0. Is it ok to mix and match HDD's on these ports?
- Some transcoding and it's usually 1, sometimes 2 users using Plex at a time (although I could see 2-3 users being more common in the future). Also planning on adding more 4K/HEVC content. Is E3-1241 v3 sufficient or should I be looking at upgrading the CPU?
- Mixing and matching existing 4TB 64MB Cache 7200RPM HDD's with 8TB 256MB Cache 5400RPM HDD's (separate VDevs) is ok right?

Sorry for the long post, but hopefully this paints a clear picture of what I have going on. Everyone's expertise/opinion is more than welcome!

 

[Chris Moore]

note that this has 8x3.5" bays and 3x5.25" bays so I purchased a 2x5.25" to 3x3.5" and 1x5.25" to 1x3.5" + 2x2.5" adapter so I have a total of 12x3.5" bays and 2x2.5" bays

No. Send those back...

 

[cwxc818]

No. Send those back...

If I may ask, what's wrong with them? Any other suggestions on how to keep the same case and expand from 8x3.5 bays so I can fit more HDD's if I go with my #2 or #3 expansion option?

 

[Chris Moore]

I don't know why you have the two 2.5 bays, maybe there is a good reason for that. but I would suggest this would be a better fit for those three 5.25 bays:
https://www.newegg.com/Product/Product.aspx?Item=N82E16816215081
Then, because the data won't move itself around, you should (one by one) replace the 4TB drives with the 8TB drives, to put the existing data on the newer drives, then create a second VDev using the 4TB drives. This will have the effect of spreading any new data across all 12 drives, where the old data will only live on the 8TB drives that are faster because of the newer hardware in the drives. Later, you can replace the 4TB drives again and have all 8TB drives.

I have a few additional questions/concerns:
- X10SL7-F is limited to 32GB RAM. Should I be concerned about this as I increase to 32TB+ of storage (1GB RAM per 1TB HDD rule)?

No, for your use case, this is not a concern.

- X10SL7-F has 8xSAS2 ports (via LSI), 4xSATA3.0, 2xSATA6.0. Is it ok to mix and match HDD's on these ports?

I would get a SAS expander to connect to the 8 SAS ports that would increase your SAS port count to 16 and put all the storage drives on the SAS connectors. Or, you could just add another SAS controller in one of your card slots, giving you another 8 SAS ports. You can use the SATA ports, and I did that with a previous build, but I find the SAS ports give more consistently fast transfers from the drives where the speed on SATA ports appears to fluctuate and be slower in general even though the rated speed might be the same.

- Some transcoding and it's usually 1, sometimes 2 users using Plex at a time (although I could see 2-3 users being more common in the future). Also planning on adding more 4K/HEVC content. Is E3-1241 v3 sufficient or should I be looking at upgrading the CPU?

I have no problem with my v1 CPU transcoding and it is only a 3.2GHz, so I don't think that should be a concern.

- Mixing and matching existing 4TB 64MB Cache 7200RPM HDD's with 8TB 256MB Cache 5400RPM HDD's (separate VDevs) is ok right?

Sure, no problem at all. I have a VDev of 2TB drives and a VDev of 4TB drives in my system right now. The drives internal cache doesn't change anything functionally even if you had different drives in the same VDev.

 

[Chris Moore]

This one would be even better:
https://www.newegg.com/Product/Product.aspx?Item=N82E16816133030

 

[tvsjr]

I would suggest option #2. IMO, if you're moving files around with any regularity, the two vdevs will "level" over time. Replacing each drive in-place, with a huge resilver each time, then adding those same drives back as a second vdev doesn't seem worthy of the effort. If you need MOAR BITS down the road, at that point, replace your 4TB drives with 8TBs/10TBs/whatever.

The 1GB/TB rule has been suspended to some extent. Making that jump in storage won't cause the system to suddenly halt and catch fire. So, upgrade your drives and see what happens. If you start seeing issues and your ARC is getting thrashed, maybe consider a MB/CPU/memory upgrade down the line. Ditto with Plex... worst case is things start buffering/jumping/etc. and you say "well, crap, time for MOAR MEGAHERTZ." :)

One thing you didn't list is your power supply. Do you have a large enough PSU to handle the additional drives?

 

[Evertb1]

Is E3-1241 v3 sufficient or should I be looking at upgrading the CPU?

Yes! Be done whit that CPU and send it to me for a nice price. Just kidding. Your CPU should be able to handle that with a margin.