Looking into FreeNAS for photo archive

[AllanB]

I'm a computer geek/photographer. I shoot RAW and these add up fast; also I do some video. I'm looking to make a FreeNAS as an archive for photos and video. Editing will be performed on a different computer.

My previous archive build was an Ubuntu system. It ran 12 x 2 TB WD green drives, software RAID 6 using mdadm, with the XFS file system running on that. Networking via Samba. Not a particularly powerful system, an i3 with 4 GB RAM. Still it worked quite well for years. It's currently powered down with a degraded RAID, one drive dead; it's full anyway and I need something larger. Right I'm working with the archive divided over several local drives on my editing computer.

FreeNAS and ZFS have some nice advantages over Linux/mdadm/XFS, mainly concerning data integrity. I'm looking to replace probably the entire build except the case, a Fractal Design Define XL Black ATX Full Tower that can fit 14 drives.

My current area of research is mainly RAM requirements. For the new archive I'm thinking 8 x 8TB WD red drives running RAID-Z3, so 40 TB capacity, perhaps expanding that eventually to 12 drives for 72 TB capacity. Following the rule-of-thumb "1 GB per 1 TB space", that would need 72 GB. This constrains the CPU socket choices somewhat. LGA 1150 can only go up to 32 GB, and LGA 1151 can go to 64 GB. Going higher requires LGA 2011. So how hard-and-fast is the rule; can this hypothetical system get by with 64 GB or does it really need more?

Also, hello everyone.

 

[BigDave]

If I'm understanding your needs correctly.

Editing will be performed on a different computer.

I would build with 1151 with 64GB if you only have say three users.
The more users the more RAM you need. Don't forget about ZIL and SLOG drive useage, that may also be something
to consider.

 

[AllanB]

Thanks. It will usually be one user, occasionally two. I was going to put the FreeNAS software on an SSD, and SLOG can go on that as well. I'm not sure what size is needed but SSD prices have gotten very reasonable (they're dropping much faster than hard drive prices.)

 

[BigDave]

I was going to put the FreeNAS software on an SSD, and SLOG can go on that as well.

This is a more advanced discussion on zil, slog use. You might find it useful.

edited for spelling

 

[Robert Trevellyan]

FreeNAS as an archive

I don't know why you would need an SLOG for this.

how hard-and-fast is the rule

It's very soft and squishy.

I agree with @BigDave that 64GB of RAM will probably be plenty. In fact, I suspect you'd be just fine with 32GB with only two users and an archive workload. Of course, your workload may evolve, so having the potential to add RAM would be nice.

 

[James Doyle]

Same use case. Shooting a D700 and a Olympus OM-D EM1. I prefer to dump my entire memory card to SAN storage, for later review and processing - sometimes months later, then erase the card, and put it back in the camera bag. I have a dedicated ethernet just for the SAN and it makes a huge difference. The Macbook and the Windows desktop thus use WiFi for internet sharing, but also have gig ethernet to the FN box. There are folks who do video editing that are doing 10 GbE to the desktop, but it's a very expensive step up for that. For now, I'm using the Intel i350-F2 server board for dual 1GbE - with CPU offload for TCP being part of what that board does. The Intel Ethernet board accepts SFP modules - so you can mix and match Copper and Fiber as needed. You will find runs of LC fiber are very cheap and very easy to run from say, basement to second floor.

Your goal should be performance and fault tolerance. I use two cheap 16GB SATA DOMs for the FreeNAS boot in mirrored mode. Inexpensive reliability for the boot device. Next, 32 GB ECC RAM. For SLOG devices, I have a pair of Micron mSATA SSD's mounted together on SATA-III utility mounting board. So, mirrored SLOG. Currently not running an L2ARC device. Controller is an LSI 9207-4i4e. The total cost of this is about $1200 but I shopped around on Ebay for controller pulls and used an outdated motherboard (Intel BTS1200, entry level Xeon E3, Cheap DDR3 ECC that was onsale). If you chase more cores, or newer boards, or more RAM than needed the total project cost can rise immensely.