ian351c
Patron
- Joined
- Oct 20, 2011
- Messages
- 219
For anyone who wants to follow along, I know Socket 2011v3 is considered trailblazing here in the forums...;)
The purpose of this build is to replace my existing FreeNAS server with something that reflects the lessons learned with that build and will allow me to accomplish the following goals:
Current Environment:
New Environment (v0.2):
I Bought a Corsair H60 Liquid Cooler for the Xeon. I've been liking these things since I bought one about 4 years ago. They're quiet, self contained, and not attractive to dust bunnies. All my research said that it should work, and even the box said it was compatible with Socket 2011. It turns out that Socket 2011 can be different from Socket 2011... As the nice support person from Corsair said: "Our Hydro coolers are compatible with the LGA 2011-3 (square mounting style). Most server-type boards use a rectangular mounting style which we unfortunately do not support." :( I mangled the mounting bracket to make it fit (mostly) and am getting good temps, but I may end up getting a dust bunny magnet after all.
Server class motherboards are so worth it! :D IPMI is the best thing since sliced bread (and much, much cheaper than a KVM over IP box that wouldn't work when the system was off anyways).
Since I have the opportunity, I'm playing around with some benchmarks. I'll post more about those in a follow up.
Relevant Threads (though I'm sure there are others I should mention but have forgotten):
Cyberjock's Hardware Recommendations
Jgreco's SuperMicro X10 FAQ
Jgreco's Burn In Thread (Though I'm not as paranoid, er I mean thorough as he is...;))
https://forums.freenas.org/index.php?threads/benchmarking-zfs.7928/
https://forums.freenas.org/index.php?threads/checking-new-hdds-in-raid.12082/#post-85381
The purpose of this build is to replace my existing FreeNAS server with something that reflects the lessons learned with that build and will allow me to accomplish the following goals:
- Home File Server
- Storage of Family Pics and Home Movies (with backup to Crashplan)
- Storage of the Family Media Library (a few TB of music, movies and TV shows)
- Home Media Server - Plex
- Personal Cloud Server - Owncloud
- Log Analysis - Splunk
- Backup - Second FreeNAS box that is semi-portable
- Time Machine
- Future Projects
- Home Automation
- Home Security
- 10GbE when it gets a bit cheaper
- Central Authentication (Samba as Domain Controller)
- etc.
Current Environment:
- FreeNAS: Core i3-2100T, 16GB RAM (non-ECC), ASRock H67M-ITX, 3Ware 9650-SE, 6x 3TB RAIDZ2
- Note that the RAM is non-ECC (lesson learned)
- The 3Ware RAID controller is running each disk in Single Mode, which gets me write caching with no actual RAID functionality, but since the controller is ~6 years old at this point and only has 256MB RAM, it's not buying me much in the way of performance and I still have to deal with the fact that if the controller dies it will take all the disks with it (i.e. they can't just be plugged into motherboard SATA ports and be recognized by FreeNAS as the same disks)
- ESXi: Core i5-2400S, 32GB RAM, Local SSD for VMs
- Linux VM running Plex, Owncloud, Splunk, MythTV, cacti, Pocketmine, Crashplan...
- Another (isolated) Linux VM for DMZ type stuff
- Windows VM for Windows Stuff
- pfSense VM for playing with IPv6 (my firewall doesn't support GIF tunnels)
- Infoblox DDI VM
- Juniper SSL VPN VM
- Quantum vmPro for VM Backup to NAS
New Environment (v0.2):
- FreeNAS: Xeon E5-1620v3, 32GB ECC DDR4, SuperMicro X10SRL-F, IBM M1015, 6x 4TB RAIDZ2
- Note: At first I was just going to use the built in SATA Ports, but the nerd in me really wanted the M1015 and for ~$100 I couldn't say no.
- The idea here is to run "production" services, especially those that would benefit from direct access to the FreeNAS storage in a jail and leave the ESXi server for ESXi specific and/or lab stuff.
- Also, Plex is getting to the point where it needs more horsepower than I have in the ESXi box, but running the NAS box without ECC RAM is a Bad Idea(TM), and I don't have the budget to redo both boxes at once.
- ESXi: Same as Current, but with some of the services migrated to a jail on FreeNAS
- Backup: The old FreeNAS guts, transplanted into a BitFenix Prodigy case so it's semi-portable
I Bought a Corsair H60 Liquid Cooler for the Xeon. I've been liking these things since I bought one about 4 years ago. They're quiet, self contained, and not attractive to dust bunnies. All my research said that it should work, and even the box said it was compatible with Socket 2011. It turns out that Socket 2011 can be different from Socket 2011... As the nice support person from Corsair said: "Our Hydro coolers are compatible with the LGA 2011-3 (square mounting style). Most server-type boards use a rectangular mounting style which we unfortunately do not support." :( I mangled the mounting bracket to make it fit (mostly) and am getting good temps, but I may end up getting a dust bunny magnet after all.
Server class motherboards are so worth it! :D IPMI is the best thing since sliced bread (and much, much cheaper than a KVM over IP box that wouldn't work when the system was off anyways).
Since I have the opportunity, I'm playing around with some benchmarks. I'll post more about those in a follow up.
Relevant Threads (though I'm sure there are others I should mention but have forgotten):
Cyberjock's Hardware Recommendations
Jgreco's SuperMicro X10 FAQ
Jgreco's Burn In Thread (Though I'm not as paranoid, er I mean thorough as he is...;))
https://forums.freenas.org/index.php?threads/benchmarking-zfs.7928/
https://forums.freenas.org/index.php?threads/checking-new-hdds-in-raid.12082/#post-85381
Last edited: