My FreeNAS setup

[chr1573r]

Edit: I built a second system, so I have expanded my original post!


System #1: Britannia - Jan 2013

I just wanted to share my setup, in case someone find it interesting or useful :)

It is mostly based on parts from an old home server a friend of mine were getting rid of,
so both the motherboard and the cpu are from 2008-2009 I believe.

After trying out different configurations, I'm pretty pleased with the end result!
7.1 TiB available storage and on average 85MB/s CIFS file transfers

Hardware:

Motherboard: Asus P5Q Deluxe
CPU: Intel Core2 Quad CPU Q6600 @ 2.40GHz
CPU Cooler: Arctic Freezer 13 PRO
RAM: Corsair DDR2, 8GB (4x 2GB)
Storage:

• 4x WD Green 2 TB (wd20ears)
• 2x Seagate Barracuda 2 TB (ST2000DM001)
• 1x OCZ Vertex PLUS R2 2.5" 60GB SSD SATA/300 MLC

Case: Fractal Design Core 3000
Other: Unbranded 4GB USB Drive (FreeNAS installation)


Configuration:

FreeNAS-8.3.0-RELEASE-p1-x64 (r12825)

The 6 HDDs are hooked up to the on-board 6 x SATA 3Gb/s ports (Intel controller, the SSD is hooked up to one of the "Asus Drive Xpert" ports (Silicon Image Sil5723 controller)
So no seperate controller/raid card were necessary.

All HDDs are used in a single RAIDZ2 ZFS Volume and the SSD set as cache, giving 7.1 TiB of usable space.

I'm using the one of the built-in gigabit NIC's as well.

Otherwise, the FreeNAS configuration is mostly default settings, with the exeption of the CIFS config:

(Bonus points for those who get the name reference!)


Performance:

So far I have only tested CIFS, and I'm getting on average between 80-90MB/s on both reads and writes

The first time I was playing around with this system I got far worse results (Max 35 MB/s writes),
but it was probably caused by an inferior gigabit switch or something on the network layer.

Here are two charts showing load and interface traffic while transfering large files to a CIFS share:

System #2: Area11 - July 2013

So I decided to build a similar system since I got my hands on some additional hardware.
This system will in time be used as a replica of my first system.

The main difference from my first build is the weaker processor, no ssd, faster hdds.

Hardware:

Motherboard: Asus P5Q Deluxe
CPU: Intel Core2 Duo CPU E6550 @ 2.33GHz
CPU Cooler: Arctic Freezer 13 PRO
RAM: Corsair and Kingston 1066MHz DDR2, 8GB (4x 2GB)
Storage:

• 6x WD Black 2 TB (WD2002FAEX)

Case: Fractal Design Core 3000(Usb 3.0 edt)
Other: Kingston 8GB USB Drive (FreeNAS installation)


Configuration:

FreeNAS-8.3.0-RELEASE-p1-x64 (r12825)

The 6 HDDs are hooked up to the on-board 6 x SATA 3Gb/s ports (Intel controller)
No seperate controller/raid card were necessary.

All HDDs are used in a single RAIDZ2 ZFS Volume, giving 7.1 TiB of usable space.

I'm using the one of the built-in gigabit NIC's as well.

The FreeNAS configuration is mostly default settings, also in CIFS.


Performance:

I have only done some very basic testing, and it seems that performance varies quite a bit on both systems. I'm not entirely sure why, but then again my testing is very limited.

The next 4 screenshots shows performance during 4 transfers; two uploads and downloads of two different 19gb files.

I do recall getting slightly better performance when I tried a transfer earlier this week, not sure why. On a daily basis, this system seems to be on par with the first one, despite weaker specs in some areas.

When I transfered the last file, I noticed a strange pattern in the network tab in the Win7 taskmanager:

It seems like the speed was much more stable at the end of the transfer.. Strange.. Maybe some tweaks could give a more consistent speed.

Bonus! Pictures of Area11


That's about it! :)

-Christer

 

[sampei]

LOL this is funny except for the RAM (I have 16GB) this might have been my machine...

 

[chr1573r]

LOL this is funny except for the RAM (I have 16GB) this might have been my machine

Funny that! :D

I'm happy to report that this setup is still working very well!
It seems like it even performs slightly better without the additional aux parameters and also without AIO.
Peaking at about 800mbit on large file transfers, which is impressive considering the parts used in the build, but also due to the fact that the traffic between my workstation and freenas passes through a linux firewall based on Pentium 4! (HP dc7100 sff)

FreeNAS is great!

 

[Daisuke]

I get writes from NAS to local disk at 90MB/sec. Writes from local disk to NAS are 50MB/sec.
I tried your CIFS settings and did not do anything for the writes to MAS, same speed at 50MB/sec.
No bottleneck on network or NAS disks.

 

[cyberjock]

I could be wrong but I believe all of those settings are the defaults for Samba now.. LOL. In fact, the buffers might be smaller than the default. Haha.

 

[Nindustries]

Any chance you could post a little benchmark of you with and without the SSD? Thanks for sharing your build!

 

[chr1573r]

Any chance you could post a little benchmark of you with and without the SSD? Thanks for sharing your build!

Sure, I'm putting together a second FreeNAS box with similar specs soon and this one will be without an SSD cache.
The new system will act as a replica.

I've checked the L2ARC cache stats every now and then on my current FreeNAS box, and it seems that the SSD is rarely used for caching.
It appears that the RAM cache handles everything more or less.

I'll test the performance on the new system when it is ready and make a new build post ;)

 

[Nindustries]

've checked the L2ARC cache stats every now and then on my current FreeNAS box, and it seems that the SSD is rarely used for caching.
I'll test the performance on the new system when it is ready and make a new build post ;)

That's what I was afraid of :D Can't wait for the post tough

 

[chr1573r]

That's what I was afraid of :D Can't wait for the post tough

I have updated my original post ;)