DIY rack mounted NAS/SAN

[minea cristian]

Hello all.
We are a small graphic/video editing company and, as the title states, I'm in the process of building a NAS/SAN server.We are editing HD video files directly from the server so we already have a 8Gb fiber channel "LAN".

My intention is to build a 20+ Tb RAID5 storage which can be accessed simultaneously by 8 video editing workstations thru the fiber channel link for online video editing.
Now, I was thinking of 8/12 3,5" hot swappable bays rack mounted case with 3TB SEAGATE BARRACUDA ST3000DM001 hard drives; 16 Gb RAM; 2 Intel Xeon CPUs.
Power consumption is not a problem.
As for the budget, I'm trying to stay as low as possible, maintaining high the performances.

As this is my first time trying a build like this, I have some questions:
1) What Mother Board should I use? I'm not interested in mini-ITX as the form factor is not a problem and I need 2 Ethernet slots and at least 3 PCIe expansion slots for installing the fiber channel network adapter, the SATA expansion card and the 2-Port Ethernet Card(although I didn't fully understand why I need 4 cat-5 cables connected to the LAN network....someone can explain better, please?)

2) Because the NAS would be accessed by multiple users at the same time, I'm going dual CPU's.What should i look for?

3) Do you have any case recommendation?

4) in terms of PSU should i go dual PSU or maintain 1 big (1000W+) unit? I already have the entire rack on UPS so power blackout should not be a problem.

5) does the FreeNAS software by itself do it, or i need to install Linux?

If you have any questions, I'm here to answer them.
If there is anybody with fiber channel NAS experience, I would gladly want to hear his opinion/suggestion.

cheers,
Cristian.

P.s. As English is not my native language, I would like to excuse in advance for the spelling errors.

 

[cyberjock]

Well, Fiber Channel isn't supported on FreeNAS... so that might be a show stopper right off the bat.

 

[Dennis.kulmosen]

What you are asking for do not come with FreeNAS, maybe you should look at the Facilis TerraBlock or DDP. Yes its pricey but it comes with integrated metadata controller and clients for fibrechannel support and access to the same content as the NAS part. But again you are looking at about 17.000 dollars for around 12TB to start with.
If you can instead use 10GBe ethernet then freenas is the way to do it.

 

[minea cristian]

does 10 GB ethernet allow me to edit HD files, at the same time, from 3-5 different pc's?

C.

 

[cyberjock]

10Mb lets you edit HD files at the same time. It might be really damn slow though.

Honestly though, the question you are asking is not something that's easy to answer. It depends on factors only you will know such as the bitrate of the files, the size of the files, the speed of the PCs. *If* your bottleneck is your LAN and you are upgrading to 10Gb it will help. But if your LAN isn't your bottleneck then upgrading your LAN is pointless.

 

[Dennis.kulmosen]

Yes it does. You can edit about 5 streams of 4K prores HQ on a single 10GBe. But as Cyberjock says it depends on your actual setup. Shared storage for video editing is not for the faint of heart, the demands is very high. And trust me if its done wrong the editors will lynch you in the nearest tree. ;-)

 

[diehard]

With a 20TB "raid5" and 16GB of RAM.. 10GB Ethernet will not be the bottleneck at all.

 

[anodos]

Honestly, in your situation I would consider giving iXsystems a call and seeing what they recommend. Since it is a business use case and you don't have a lot of experience with FreeNAS, you should considering getting things with proper hardware warranties and support contracts. I've never heard of anyone losing a job for purchasing a support contract, but the other way around... well... that happens. It costs more to do things the right way, but it's money well spent.

 

[minea cristian]

With a 20TB "raid5" and 16GB of RAM.. 10GB Ethernet will not be the bottleneck at all.

...that would be?

... maybe you should look at the Facilis TerraBlock or DDP.

I already have a Facilis Terrablock 24D with 13, 2Tb HDD, fiber channel connected(that's the reason I was looking for a fiber channel solution, because I already have the fiber channel "LAN"), but it's almost full, and I'm not eager to spend 20k Euros to buy another one.So I was looking for something more affordable.

.. But as Cyberjock says it depends on your actual setup.

I'm using Hp Z400 workstations for the video editing....with Intel Xeon W3565, 16 Gb RAM and nVidia GeForce GTX 560. I'm already editing HD files from the Terrablock without any problem. So the PC's can't be the bottleneck either.

with the 10Gb ethernet...means that i would have to run another set of wires with another switch and PCi express card's, right? This will cost me another...?

 

[Dennis.kulmosen]

What format and codec are you editing? How many streams at once do you edit? How many clients?
If you are editing normal 1080 250bps files or smaller you should be fine with normal 1GBe ethernet for the clients but consider to put them in a switch that are connected to the FreeNAS with a 10GBe uplink or put many quad 1GBe ethernet cards in the FreeNAS box and direct connect the clients.

 

[anodos]

...that would be?

Your mention of RAID5 and 16GB RAM raises a few problems
1) you shouldn't use hardware RAID.
2) you shouldn't use RAID5 / RAIDZ1. See cyberjock's link regarding them.
3) A RAIDZ vdev has the IOPS of a single disk.
4) 16GB RAM is probably inadequate for your needs. See http://doc.freenas.org/index.php/Hardware_Recommendations#RAM

(3) and (4) mean that your zpool will bottleneck before your network.

 

[Dennis.kulmosen]

Your mention of RAID5 and 16GB RAM raises a few problems
1) you shouldn't use hardware RAID.
2) you shouldn't use RAID5 / RAIDZ1. See cyberjock's link regarding them.
3) A RAIDZ vdev has the IOPS of a single disk.
4) 16GB RAM is probably inadequate for your needs. See http://doc.freenas.org/index.php/Hardware_Recommendations#RAM

(3) and (4) mean that your zpool will bottleneck before your network.

Good catch. Did not see those details.

For 5 clients video editing i would not go below 20 drives with matching RAM put in 4xRAIDZ if its only editing. If its going to hold critical data i would go with 24 drives in 4xRAIDZ2. If you plan on using 4TB drives your RAM demands would be 24x4=96GB dont use zil or L2ARC those will only slow you down in video workflows, add more RAM if its not fast enough.

 

[cyberjock]

I'm gonna bow out of the conversation. This is just too much confusion for me.

Good luck to all.

 

[Dennis.kulmosen]

Your mention of RAID5 and 16GB RAM raises a few problems
1) you shouldn't use hardware RAID.
2) you shouldn't use RAID5 / RAIDZ1. See cyberjock's link regarding them.
3) A RAIDZ vdev has the IOPS of a single disk.
4) 16GB RAM is probably inadequate for your needs. See http://doc.freenas.org/index.php/Hardware_Recommendations#RAM

(3) and (4) mean that your zpool will bottleneck before your network.

Good catch. Did not see those details.

For 5 clients video editing i would not go below 20 drives with matching RAM put in 4xRAIDZ if its only editing. If its going to hold critical data i would go with 24 drives in 4xRAIDZ2. If you plan on using 4TB drives your RAM demands would be 24x4=96GB dont use zil or L2ARC those will only slow you down in video workflows, add more RAM if its not fast enough.

Edit: Sorry for the double post. My Tapatalk did some funny things. :-S Admin you can delete this.