Freenas for 300 users vdi instant clone

[Chris Moore]

HArdisk 12 Intel ssd 480GB with raidZ2 (how about it?)

What model SSD are you looking to use?

 

[mhejek]

What model SSD are you looking to use?

SSD DC S4600

 

[mhejek]

What model SSD are you looking to use?

if i use JBOD expander does it will decrease performance or increase?

 

[Chris Moore]

i understand how it works.
Check your calculations

for example with 12 pcs of Intel DC3710 (read 85000IOPS, write 35000 IOPS) - not new, not very expensive, not very fast SSD, you will get in RAID6 (RAIDZ2) approx 106935 IOPS for writing and 1020000 IOPS for reading. Calculations from http://wintelguy.com/raidperf.pl. i know about synchronous write, but "very fast proper SLOG SSD" is very expensive. And if you have a lot of IOPS, i think you don't need it. But you are absolutelly right for HDD - a lot of mirror vdevs and SLOG

I actually didn't re-run the numbers using SSD specs and I have not created an all SSD pool myself. The numbers are interesting but I remember another forum user posting results for their all SSD pool and that they were not getting the performance they expected. If I recall, they were using consumer SSDs, and that may have made a big difference.

 

[Chris Moore]

if i use JBOD expander does it will decrease performance or increase?

I am not sure what hardware you are implying or how you intend to connect it. It would be better if you could refer to specific model and part number and give a better description of how you mean to connect it to the system. Using SSDs, it is possible to connect more drives than a single SAS controller has the bandwidth to fully utilize.

 

[jgreco]

I am not sure what hardware you are implying or how you intend to connect it. It would be better if you could refer to specific model and part number and give a better description of how you mean to connect it to the system. Using SSDs, it is possible to connect more drives than a single SAS controller has the bandwidth to fully utilize.

An SAS expander in a JBOD will tend to negatively impact performance as it is just another point of contention. Even an HBA has a bit of a performance tax over a direct SATA controller on the system PCH.

 

[Chris Moore]

An SAS expander in a JBOD will tend to negatively impact performance as it is just another point of contention. Even an HBA has a bit of a performance tax over a direct SATA controller on the system PCH.

Just my experience, in my system, When I went from two SAS controllers to a single controller with a SAS expander, the amount of time required to scrub my pool went down by TWO hours with no other changes. Results may vary.

 

[Chris Moore]

if i use JBOD expander does it will decrease performance or increase?

You can only connect a certain number of SSDs to a SAS controller before you have more IO capacity in the SSDs than the SAS controller can accommodate.

 

[jgreco]

Just my experience, in my system, When I went from two SAS controllers to a single controller with a SAS expander, the amount of time required to scrub my pool went down by TWO hours with no other changes. Results may vary.

That's probably not terribly useful without some significant analysis, and not particularly relevant to the discussion here anyways:

Two HBA's on full PCIe slots, as opposed to "PCIe-lane-challenged" (i.e. x4-in-x8 slots), should be able to run substantially faster than a single HBA. It might not be relevant depending on the disk configuration. There's a little extra overhead for managing two HBA's but this should be offset by the transfer speed increase, as long as there is a sufficient number of disks. If one of the slots ended up as an x4, though, I could easily see the time increasing by two hours for a large pool.

Unfortunately, the opposite is true for SSD's, where modern SSD's are pretty much able to max out a 6Gbps SAS lane, so your HBA with eight lanes is effectively maxxed out when running eight SSD's, and beyond that point you can easily become oversubscribed when doing large accesses or scrubs. You will also be running at around the capacity of the PCIe bus, which for the typical SAS2008 controller is 500MB/s per lane, so the calculus for SSD pools is a little complex. You could potentially run an x8 wideport to a 12 bay JBOD and not have much of an impact, but beyond that it gets a little challenging and it needs a more formal architecting.