Sun Sun J4400 Storage Array 20 SAS 10K 4TB Drives

[Nick Lutz]

I've been using FreeNAS for the past year, and have run into an interesting application for FreeNAS. I'm looking to create a "mega store" for near-line backup. I have a Sun J4400 24 bay storage array with 21x 10K 4T drives, 3x 18GB, SSD's, and a Sun Fire X4140 Dual AMD Opteron 2384 Quad cores 2 x 2.75GHz with 64GB RAM. In that unit I have a single 128GB SSD and 5x 300GB SAS 15K drives.

I've configured the unit as follows:

FreeNAS 9.2.1.6 on 8GB pen drive

VOL01
20x 4TB SAS 10K disks RAIDZ - actually 2x10 4TB RAID groups as one volume
2x 18GB SSD LOG disks
1x 128GB SSD Cache disk

VOL02
5x 300GB SAS 15K disks RAIDZ
1x 18GB SSD LOG

1 spare 4TB SAS drive (Hot spare not possible with FreeNAS)

Any ideas on improvement? I only have about 3TB data on it now, and it seems to perform quite well, easily saturating the 4 onboard 1Gbit NICs. I do have a 4 port 10Gbit copper card, but no drivers for FreeNAS/FreeBSD exist for that NIC. I have ordered some Intel 10Gbit cards and will implement them when they arrive.

 

[Nick Lutz]

FYI: I tried to get FreeNAS to configure 3x 7disk RAID Z groups, but it refused to cooperate.

 

[HoneyBadger]

I'm a little concerned about the use of RAIDZ1 with the bigger disks, it's opening you up to failure risk during a resilver esp. with vdevs that wide.

Personally I'd configure VOL01 as two 10-drive RAIDZ2 vdevs (which also gives you 8 data disks for that nicely divisible data striping) and then add the mirrored SLOG/L2ARC SSDs.

VOL02 I'm OK with being RAIDZ1 as it's using smaller 15K disks that probably have a lower URE, and RAIDZ1 gives you a 4+1 setup for better striping.

Re: the NIC, 10Gbps support under FreeBSD/FreeNAS right now is a bit spotty, the only ones that reliably work are the Chelsio series I believe.

And if those are the 18GB SSDs I think they are (ZeusIOPS) consider me officially jealous. ;)

Hell, consider me jealous anyways, that system looks awesome.

 

[Nick Lutz]

They are ZeusIOPS... 1 of the 4 failed though... was a sad day! This was hardware that was sitting on the loading dock ready for surplus! I saw some potential and we now are using it daily.

 

[HoneyBadger]

Sounds like an excuse to replace that "unreliable flash memory" with ZeusRAMs instead. ;)