Looking for you oppinion

[zorg]

Hello

I am building a new storage server for a small business and would love to hear your opinion about it before I buy the hardware.
The main use for this server is to backup about 30 workstations with lots of similar data, so I was hoping to make use of deduplication is this scenario. Thought on this is most welcome.

The hardware I have found sofar and are ready to buy are this:
The start point is 4x 3TB SATA drives and 2x 60GB SSD for cache.

The Enclosure is a Refine R3.
file server v5..

Intel 330 Series 60GB SSD 2
Corsair CMPSU-750HXEU 1
ASUS P8B-C/SAS/4L 1
Intel Xeon E3-1230V2 - Box 1
Kingston 16GB 1333MHz DDR3 ECCReg (2x8) 1
Western Digital WD Red - 3TB 4 (so i can expand later)
Kingston DataTraveler Ultimate 3.0 16GB 1 (to install freenas to)

 

[cyberjock]

There is no deduplication for FreeNAS. It won't be here until zfs is upgraded to v28. We are currently on v15. The next major update(8.3.0) should include deduplication(v28) if I remember correctly. I believe that deduplication is kept on a per-file basis. This means your images will not be deduplicated because it will see the image as one file. Obviously each workstation will have a different image making deduplication worthless for your intended application. I don't have exact numbers, but I seem to remember that deduplication needs a boatload more RAM. I'm doubting 16GB would cut it.

Overall, you should reconsider your plan because it has multiple holes in it. Of course, I could be mistaken. I haven't spent much time reading about dedup because it's not offered yet in the production versions of FreeNAS.

Also, you shouldn't be setting up a L2ARC unless you know you need one(aka databases are used heavily).

You should probably read the presentation in my sig...you don't seem to have a firm grasp of what you are doing.

Edit: I just read in another forum 2GB of RAM per TB is what is advertised, but some places claim up to 5GB per TB of data!

 

[zorg]

Hey again

I see what you mean, and have read your guide! I will have to wait with production deployment untill deduplication is ready, since I am NOT going to buy 30TB of storage for those pc's ;)
About the type of files, I will have to make a filebased backup instead of image, I am aware of that.
This is for a Internet gaming cafe, and most of the data is Games, those are big files that will be almost identical on each pc. I have removed the 2 ssd's and added a 32GB Ram kit instead, that might be better.
Although I am most likely also using this partly within my vmware system, but I think speed is probably not a big issue.

I have had a storage server on the drawing board a long time, and have been using different setups over the 15 years I have had my cafe.. but this is the first time I am trying to make a backup system for all my pc's there..
Although you are most likely an expert in freenas, I do have experience with different other file appliances.

Thanks for taking the time!
jc

 

[cyberjock]

You COULD go with PXE booting if you want to be a pimp daddy. I saw a setup that used PXE and had 1 image for all 30 workstations and they were slow to bootup in the morning if they were all booted simultaneously, but they ran great all day long with gamers playing on them. You'll have to change your network configuration to support more bandwidth. Do note that I'm not sure if FreeNAS will work as a PXE server.

Adding updates and whatnot was easy. Update the 1 image file and automagically all your systems were updated!

...Just an idea. :P

 

[zorg]

There is several other ways to accomplish what I am trying to do, but I do not think PXE boot is one of them :|
When a costumer arrives, we turn on the pc, not before, so startup time is a big issue. And there are to my settings that are different for each pc, so it wont be easy to use anything like 1 image.
But, there is some software we are looking at, wich may let us be able to host some of our games on a NAS, that might be interesting aswell. Or mount a games drive with iSCSI maybe.. We are running 1Gbits network already.

Let us say for now, I just want to have the hardware confirmed for a solid setup.. the motherboard and the added LSI controller, and I was going to use software raid within freenas, that is good i hope.
There should be space enough to be able to expand to 10 disks, when the need arises.

Oh well ;)

-jc

 

[cyberjock]

If you're booting only 2-3 computers at any one time PXE can definitely handle that. Gb network speeds are the limitation typically.

If you are looking to expand keep in mind you must make a new vdev so expanding will cost you more drives to redundancy. If you know you'll want the space soon you're better off building it right the first time. The biggest drawback to ZFS is you can't expand the zpool like you could a hardware RAID with OCE.

 

[zorg]

Revised setup:

PSU: Corsair CMPSU-750HXEU
Mobo: ASUS P8B-C/SAS/4L
CPU: Intel Xeon E3-1230V2 - Box
OS Disk: Kingston DataTraveler Ultimate 3.0 16GB
Ram: Kingston 32GB DDR3 1600MHz (4x8) Value
Disks: 5x Western Digital WD RE4 RAID - 2TB
Case: Fractal Design Define R3 Black

Better disks, with lower capacity though, but I then have for 1 VDev with fail over 1 disk. And will be able to add 5 more disks (to a new VDev :)) without changing hardware except for adding drives.

 

[survive]

Hi zorg,

Before you go to much further with your dedupe plan I encourage you to read this (the FreeNAS 8.3 release notes):

http://freenas.svn.sourceforge.net/viewvc/freenas/trunk/ReleaseNotes?revision=12036&view=markup

starting at line #18.

is there any reason you can't run some of this stuff off the network? If there are 30 machines all with roughly the same content that doesn't change much that's all reads, all the time I would imagine an L2ARC would work wonders for you.

-Will

 

[cyberjock]

ROFLMFAO! From the link:

Furthermore, importing an unclean pool can require between 3-5GB of RAM per TB of deduped data, and if the system doesn't have the needed RAM it will panic, with the only solution being adding more RAM or recreating the pool. Think carefully before enabling dedup! Then after thinking about it use compression instead.

Emphasis mine. That is HILARIOUS! I got a good belly laugh at that one.

I find it entertaining that everywhere I've read about dedup people always comment with something to the effect of "people try to save money with less hard drives, but ultimately dump much more money into CPU and RAM to get dedup to work". It's kind of like "why add the feature then?" I guess someday CPU power and RAM might be cheap enough to make it worthwhile.

 

[zorg]

He he, fair enough.... ! How fairly does the included compression work?

Survive: Yes, I am investigating this too, to be able to run several of our games off of the network. We are looking into this software http://www.gizmopowered.net/ that will enable (and integrate) this with our existing solution.

I am considering hard to not use deduplication at all, but maybe I can work with backups through software, and make gamefiles reside only once on the server... it might be possible.

Should I get a single 60GB SSD just as read cache (L2Arc)? I mean if I do have clients load games directly from the NAS, it might help, or does 32GB Ram cover most of that?

 

[Stephens]

zorg, your setup is looking pretty good now, but I have to ask... are you still planning to use dedupe?

 

[zorg]

After I read the notes from freebsd 8.3 I am a little scared of it to be honest... and it complicates the administration etc. a bit to much maybe.
This would really be better if I can just work together a solid solution (normally I require about 4-5 years of usage when I build servers), and then work with how I do the actual backup.
Right now I am looking at Deltacopy http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp , which is a wrapper for rsync.. I am a heavy linux user, and run most my servers with Ubuntu server.. but am also invested alot in windows.

Would be nice to be able to use something like rsync with windows, then maybe I can make it use same backup set and only store differences from a basis backup from the first pc. Am I being unclear now? =) Or does my idea sift through my gibberish hehe.

Oh and does anyone have any thoughts on compression on freenas/zfs?

Thanks for your help.

 

[paleoN]

PSU: Corsair CMPSU-750HXEU

The PSU is overkill.

Better disks, with lower capacity though, but I then have for 1 VDev with fail over 1 disk. And will be able to add 5 more disks (to a new VDev :)) without changing hardware except for adding drives.

I'd recommend raidz2 with that many 2TB disks. I would also consider that 6 disks are an optimal raidz2 config.

Should I get a single 60GB SSD just as read cache (L2Arc)? I mean if I do have clients load games directly from the NAS, it might help, or does 32GB Ram cover most of that?

You could always wait and see how much you need one. Then you would have a better idea of what size SSD to get as well.

Oh and does anyone have any thoughts on compression on freenas/zfs?

It works quite well by all accounts. Just don't go crazy and set it to gzip 9. Use lzjb or with a reasonable gzip value.

 

[zorg]

The PSU is overkill.

I'd recommend raidz2 with that many 2TB disks. I would also consider that 6 disks are an optimal raidz2 config.


You could always wait and see how much you need one. Then you would have a better idea of what size SSD to get as well.

It works quite well by all accounts. Just don't go crazy and set it to gzip 9. Use lzjb or with a reasonable gzip value.

Regarding PSU, well, I did not find any other with enough sata connectors, feel free to suggest otherwise.

With 6 disks in raidz2, is 1 or 2 disks for parity, I was looking around and am still a little unsure what configurations it supports. I have an additional NAS (Netgear NX system) for the critical data, so 1 parity disk might be just fine.

I will wait with adding SSD, untill I get a little more familiar with freenas.

Will test compression within a vm, to see what effect I can gain from it.

Thanks

 

[hentedisken]

ASUS P8B-C/SAS/4L 1
Kingston 16GB 1333MHz DDR3 ECCReg (2x8) 1

Hey Zorg!

Have you got it up and running?

I am doing the P8B-E/4L with the 16 GB kit from Kingston, but can't get it to boot. Only get four beeps and a blank monitor. I think it's the RAM.

Just wondering if you had success or not :)

Thanks

 

[zorg]

Heysan

I have, but I ended up running server 2012.. although that has nothing to do with the issue you have.
Actually I also had problems getting started with my server, but in the end, it was be course the CPU was a version where the motherboard needed a BIOS update before it would work.
So i borrowed a i3 CPU and got it updated, then the XEON worked.
I had some RAM that did not work though, but unfortunately, I forgot which.. I ended up with "only" 16GB available ram.

I also slapped in a 128GB Samsung 830 SSD as OS disk, and 5 WD RE 2TB disks.. It performs grand.

One last thing.. I had the Controller from IBM with an LSI Chip, that needed new firmware.. it was a tad tricky to do, since the bios was EFI, but after all it is running smooth and have for about 2 months.

Good luck with yours m8.

Regards
z