FreeNAS for video production?

[TDI]

Hi, I'm setting up a home server that will store a large amount of data and used in video production for a home business (1 user, but accessed from multiple machines at different times). I am currently using an external 6TB G drive that is being backed up on to another computer on the network... when I remember. This is clearly not ideal, and I would like something with more redundancy (I will still be doing regular backups).

I am currently playing around running FreeNAS on an old amd system that only has one drive in it, but after a lot of research I have ordered new parts to put together a new system.

The specs are:
Supermicro Ultra ATX DDR3 1600 LGA 1150 Motherboard X10SLM-F-O (closest match I could find on the supermicro site was X10SLM+-F)
Fractal Design Node 804 Case
Intel Pentium G3240 Haswell Dual-Core 3.1 GHz LGA CPU
CORSAIR CX series CX430 430W power supply
Kingston 4GB 240-Pin DDR3 SDRAM ECC Unbuffered DDR3 1333 Server Memory (this will be upgraded to 2x8GB and 2x4GB before any real use.. I am just waiting on a check to clear)
No hard drives yet, but I am planning to use 3x4TB drives to start and expand down the line as money permits. (I understand that I may have to rebuild to get everything to work optimally). I will be using some old drives to play around with configurations and test different setups for now, but I will be upgrading to NAS drives before production use.

This is just the build I am starting with and I plan on doing a lot of testing on this before I actually put it into production. I got the idea to use a NAS from a company I worked at where their IT guy used a NAS to allow the small group of editors to edit HD video and it worked fairly well. It would have worked much better if he had used a file system that allowed OSX to search the drive... (I already tested this with FreeNAS using AFP and it works great).

Anyway, I have a few questions that I couldn't really find any answers on.
1) What would be the optimal configuration if I'm going to primarily be using this for video editing?Is ZFS the best option?
2) Can I set up a trash system for AFP like I can for CIF? I can't find anything on this and I would really like to have some kind of protection against accidentally deleting files.
3) I'm pretty sure I will need to upgrade my storage at some point in the future. Would it be better to just back everything up (to at least two different locations) and rebuild the Zpool/VDev with all the drives, or just add another Vdev to the Zpool? Since I have only been running one drive, I have not had a chance to mess around with this yet.
4) I read somewhere that it is now best to boot from a small hard drive or SSD instead of a USB drive, but I can't find it again. Is this true?

I'm sure I will have more questions in the future. If I have asked anything thaty is covered elsewhere, please point me in the right direction.

If anyone has used FreeNAS for a similar job, I would love to hear any advice you might have.

Thanks!

 

[nojohnny101]

1) FreeNAS only can do ZFS, so if you do not want to use ZFS, then you will have to use something other than FreeNAS.

2) there is no trash system that I am aware of for AFP. The only built in solution I can think of in FreeNAS would be snapshots. Would taking a snapshot at the end of each day be good enough? It would impractical to take snapshots every 5 or 10 minutes because that would quickly get out of control. Read more about snapshots here to understand how they work.

3) I recently upgraded from 4 x 2tb RAIDz2 to a 6 x 3tb RAIDz2. Through research, it seems 6 drive RAIDz2 is the sweet spot with a good raw storage:usable storage ratio. If you can only start with 4 though then yes, you would just have to dump everything in your FreeNAS to either external drives or if your motherboard has extra sata ports then you could keep everything internal and transfer that way. but you are correct, once you create a vdev (pool) you can't expand it later. You could just add another vdev but then your protection is only as good as the lowest RAIDz type.

4) I boot from a flash drive. Some of the veterans on here advocate for the SSD route but I saw just buy 2 x 16GB brand name flash drives and then keep one as a mirror. restoring the OS and config files is so trivial in FreeNAS that I don't see it worth purchasing one. If you have one laying around, then go for it, it is technically better.

 

[Robert Trevellyan]

2) Frequent snapshots can be set to expire quickly too, to keep the number of snapshots under control. For example, hourly snapshots could be set to expire after 24 hours.
3) It's better to plan your storage from the start, so you know how many drives you need to start with, and how many you'll need x years out. Replicating, trashing and rebuilding is OK with moderate-sized pools, but becomes impractical with large pools. Video production is the type of application I would expect to consume a lot of storage.
4) The most important thing is always to have an up-to-date backup of your configuration. That way you can quickly and easily recover from a failed boot device.

 

[David Dyer-Bennet]

We've got two FreeNAS boxes, storing files mostly for video projects. We like it; in particular it gives us most enterprise-grade features for a lot less money, and much easier to manage. With sensible use of the "scrub" feature it's much much safer than most alternatives for long-term archiving, also (the ZFS block checksums will detect small errors far earlier than depending on the hardware; in fact in the early ZFS days, in its original Solaris home, ZFS had a rep for finding hardware with "high error rates" much more solidly than other software).

That said, working over a network is slower than working on local files in general (certainly at the gigabit ethernet level). For video editing it isn't a problem IMHO (having done hundreds of hours of it) -- the rate at which you need to read files is limited to the play speed, which is quite slow in data terms. And the safety of having your project files, in particular (they being what changes frequently), on ZFS is huge.

For my initial small server (mostly still photos, much smaller than video), I used multiple mirror vdevs in the zpool. It was very nice to be able to upgrade by adding a vdev; it's also nice to be able to upgrade the disks in the vdev without a second server (attach a *third* disk to the mirror, a bigger disk, let it resilver; detach one of the original disks; Attach a *new* third disk to the mirror, let it resilver; detach the remaining original disk; and you have just increased the size of a vdev without ever reducing the redundancy below the original size). However, for bigger arrays, RAIDZ is of course a lot more space-efficient, and adding vdevs is problematic when you need so many controller ports (and just physical drive slots) to do it, so our video servers are maxed out in their original deployment, and we kick things over to another server when it's time to grow.

I haven't found memory use to be particularly heavy (unless you're using deduplication; don't, unless you have special reasons). 8GB is the recommended minimum, and I've got two boxes with 16GB running -- and I've never seen them actually use as much as 8GB.

 

[TDI]

Thanks for the replies! I was under the impression that you had more options than ZFS due to some posts or videos where people didn't recommend using ZFS. I guess they just didn't like FreeNAS. I also thought I saw a picture that showed an option other than ZFS, but I may have been mistaken.

I'll have to look in to snapshots. I'm not familiar with them. It sounds like they may take up a lot of disk space. Is it easy to get back data if I were to accidentally delete something or is it a pretty involved process? I'm planing on doing daily backups to another computer, so as long as the files are from a previous day, it might be easier to just copy them back from the backup.

Speaking of backups, is there a good backup/sync solution that I can run from the FreeNAS server that can sync the server drives to another computer? I use Chronosync To backup the external drive I am using right now and it's great, but I would like something that I can use without having to leave the Mac running in addition to the server and the backup computer. Most backup programs I've looked at tend to want to use a compressed format, but having to restore 6 TB of data in case of an emergency would be an overnight job (at least) and I would much rather only have to copy the needed data out of the backup if I am working on something that needs to be done ASAP.

I decided to go with a Flash drive since Supermicro was nice enough to put a USB3 port right on the motherboard. I didn't like the idea of either having one plugged into the back of the computer (where it could get broken and possibly damage the motherboard when moving the computer) or wiring some kind of internal USB adapter. I'm using a tiny SanDisk Ultra Fit which is hardly bigger than the USB port, so it not in the way when I need to get into the computer for anything.

I added a 4 TB drive to my two 3TB drives and tried to do a RAIDz across them, but it doesn't seem possible. I was under the impression that you could add multiple sized drives and it would only use the size of the smallest drive (so it would be like having 3 3TB drives), but this doesn't seem possible. I'm pretty sure I've done this using hardware raids before on other computers. Is this just a limitation of FreeNAS? This isn't a huge deal because I'm still just testing and plan on adding more drives before I start using this for work. Right now just I've been copying files overnight to the drives just to put some stress on the server and drives and see if anything breaks.

As far as memory, I just upped it to 16GB and it seems fine. Honestly didn't seem to be having any problems at 4GB, so i'm going to stay with 16 for now. I guess if you are running things like PLEX from the drive this may be more of an issue, but I'm not planning on having anything other than a backup program running off of the server.

 

[David Dyer-Bennet]

Snapshots take no space when created. Over time, depending on what happens *elsewhere*, they may take space (if the *current* version of a file has changed, and the snapshot needs to then keep the old version around). Keeping hourly snapshots around forever does eventually consume huge amounts of space . They're very easy to access -- if you set the right property, a ".zfs" directory appears at the top of each filesystem, and you can navigate into that directory and find all the snapshots directly visible there, shared out via SMB or NFS or whatever.

You can do RAIDZ or mirrors across drives of different sizes, getting the smallest of them as the size usable on each, as you expected. You can't *add* a drive to a RAIDZ usefully, but I think you understand that; creating a new RAIDZ with drives of different sizes has worked fine for me. So I don't know what's going wrong for you there.

The absolute best backup is another FreeNAS box; you then use ZFS send / ZFS receive to transactionally replicate content (including snapshots) from one to the other, across whatever network pipe you want. I've been using that since 2007 or so (slightly different, going to a USB-connected external drive) and it's wonderfully better than any other backup scheme I've ever seen. The backup is then a fully usable ZFS filesystem.

I ran my Solaris ZFS server on 4GB and it was plenty. People here say the 8GB requirement is important, and I haven't experimented with violating it, and it's certainly possible that a different OS would have diffferent memory requirements . But my experience with 16GB is that it's a great plenty for FreeNAS.

 

[TDI]

Thanks for the detailed help David. I may be doing something wrong with raid setup across different sized drives, so I'll do some more research on that. Backing up to another FreeNAS is a good idea, but probably not something I'll be able to afford for a while. I'm sure I'll probably have more questions in the future, but for now I think I have a lot more reading to do.

Thanks again!

 

[TDI]

Ok, I ran into my first issue with FreeNAS today. I accidentally deleted a file I shouldn't, so I cloned a snapshot as is says to in the Docs, but I can't find the file. I tried creating a filter in the web interface, but it does not find the name or any part of the name of the file. I tried loading several of the snapshots that I had including the newest and oldest and the file is not in any of them. I assumed that if I just opened the last snapshot that the file would be there. Do I have this all wrong? I was able to recover the file from a backup, but I'm trying to figure this out in case I accidentally delete something that hasn't been backed up yet, or ther eis an issue with my backups. Also, what is the best way to get rid of the files from the cloned snapshot? Should I just delete them from the file browser, or is there a way to do this though the web GUI?

 

[Robert Trevellyan]

How are your snapshots set up? Some people only set a top level snapshot and forget to either make it recursive or add snapshots for other datasets, which leaves them with no snapshots of the child datasets.

How did you access the contents of the cloned snapshot? Did you make a new share linked to it, or use the CLI to lists the contents, or something else?

I tried creating a filter in the web interface

Did you try to use the snapshot list filter to find a file? The snapshot filter filters snapshots, not their contents.

I assumed that if I just opened the last snapshot that the file would be there.

What do you mean by "opened the last snapshot"?

what is the best way to get rid of the files from the cloned snapshot?

When you clone a snapshot, you end up with a standalone dataset named for the snapshot you cloned. When you're done with it, delete that dataset.

 

[TDI]

How are your snapshots set up? Some people only set a top level snapshot and forget to either make it recursive or add snapshots for other datasets, which leaves them with no snapshots of the child datasets.

I had not set it as recursive, which is probably the cause of whole issue.

How did you access the contents of the cloned snapshot? Did you make a new share linked to it, or use the CLI to lists the contents, or something else?

I cloned the Snapshot which loaded it into the Root directory and browsed through it like I would any other files.

What do you mean by "opened the last snapshot"?

I cloned the snapshot and opened the snapshot in the file browser.

When you clone a snapshot, you end up with a standalone dataset named for the snapshot you cloned. When you're done with it, delete that dataset.

I just found it under Volumes.

 

[Robert Trevellyan]

OK, sounds like you know what to do.

 

[TDI]

I'm definitely going to do a little more testing on this and making sure I understand how it works before relying on it for anything. Thanks for the help.