Disk / RAM Configuration Questions

[kjparenteau]

Greetings! I recently acquired an R720xd and want to step up my game with FreeNAS, but I have a question about how I should setup the drives. Here's the new hardware...

R720xd, 12x 3.5" bays in the front with 2x 2.5" flex bays in the rear
2x CPU E5-2630 v2 @ 2.60GHz
384GB RAM
PERC H710 Mini
12x 6TB SATA
2x 256GB Samsung 860 EVO SSD's
4x 2TB Samsung 850 Pro SSD's

Note: I am aware of the H710 being unable to JBOD the disks, so I am working on a solution for a different HBA like the LSI 9211-8i in a different post HERE.

Here are the configurations I am considering... I will be running a mix of media server/Plex natively with FreeNAS, and then also creating some zvol's for VMware using iSCSI. When I attempted some testing between my current Frankenstein FreeNAS build similar to config 2 below, I didn't really see much of a difference between the standard disks and the SSD's. Please see the performance examples below to see what I mean, but maybe I am also not doing a good test either. :) Any assistance or recommendations with the hardware I have, please let me know. Thanks!

CONFIG 1
2x 256GB in RAID1 (FreeNAS OS)
12x 6TB in RAIDZ2 (60TB Usable)

CONFIG 2
2x 256GB in RAID1 (FreeNAS OS)
8x 6TB in RAIDZ2 (36TB Usable)
4x 2TB in RAIDZ1 (6TB Usable)

PERFORMANCE TESTS
HDD Pool - (6x 3TB 7200RPM SATA in RAIDZ1)

SSD Pool - (4x 2TB Samsung 850 Pro SSD's in RAIDZ1)

 

[kjparenteau]

Nobody has any suggestions on this...? Is it beneficial to use the SSD's in FreeNAS considering the numbers are so similar?

 

[John Doe]

sorry, what is actually the question?

see 2 different configurations and a benchmark ssd vs hdd

 

[kjparenteau]

sorry, what is actually the question?

see 2 different configurations and a benchmark ssd vs hdd

My apologies the question was not more clearly outlined... This is part question, and part guidance... Is it actually worth sacrificing a ton of raw storage to have the SSD's when the numbers from the test are so similar? I know very well the difference between a 7200 RPM disk and SSD when it comes to a PC, however with FreeNAS and a massive amount of RAM cache, etc., I am not as familiar with... Having more disks in the pool will help generate more IOPS, but from the 2 configuration options outlined below, I can add 4x 2TB SSD's, but I sacrifice a ton of RAW storage from the large disks.

CONFIG 1
12x 6TB in RAIDZ2 (60TB Usable)

CONFIG 2
8x 6TB in RAIDZ2 (36TB Usable)
4x 2TB in RAIDZ1 (6TB Usable)

So the question is... With these two options, would the SSD's (all drives being SATA) show worthwhile benefit over the larger storage space? Or is the massive amount of RAM leveraging the ZFS cache, etc. big enough to where I would see almost no benefit from the SSD's and should use the larger disks for more storage?

I hope that helps to clarify the question better. Thank you in advance!

 

[Patrick M. Hausen]

Putting the VMware storage on SSDs is definitely worth it in my experience. I am actively moving everything away from VMware to FreeNAS as storage and hypervisor, but I am quite confident my experience can be transferred to VMware and iSCSI.

There's a great guide by @jgreco on that topic:

The path to success for block storage

It seems like I haven't written a sticky for awhile, but just in the last week I've had to cover this topic several times. ZFS does two different things very well. One is storage of large sequentially-written files, such as archives, logs, or data files, where the file does not have the middle...

www.ixsystems.com

 

[kjparenteau]

Putting the VMware storage on SSDs is definitely worth it in my experience. I am actively moving everything away from VMware to FreeNAS as storage and hypervisor, but I am quite confident my experience can be transferred to VMware and iSCSI.

There's a great guide by @jgreco on that topic:

The path to success for block storage

It seems like I haven't written a sticky for awhile, but just in the last week I've had to cover this topic several times. ZFS does two different things very well. One is storage of large sequentially-written files, such as archives, logs, or data files, where the file does not have the middle...

www.ixsystems.com

I typically agree with SSD over spindle drives, however as I mentioned in the original post with the benchmark testing, it didn't seem to make as much of an impact as I was expecting. So the question still remains... Is it worth losing over 20TB of storage in trade of 6TB of SSD that so far doesn't seem beneficial?

 

[Patrick M. Hausen]

Ah ... you absolutely must configure your pool as mirror pairs for block storage. Then redo your benchmarks. I'm curious ;)

 

[kjparenteau]

Ah ... you absolutely must configure your pool as mirror pairs for block storage. Then redo your benchmarks. I'm curious ;)

So I have read several times recently about different configuration options such as this... Wouldn't I lose a lot of space by running mirrored pairs? The performance gain vs space lost seems like a catch-22, vs running something more simple like RAIDZ-1 no?

 

[Patrick M. Hausen]

If you have an IOPS heavy application you want to deliver IOPS in most cases. Hardware is - generally - cheap. So, yes, for my VMs @work as well as @home I "waste" half of my disks.