Need help on how to best to setup drives

[Trioxide]

I am new to FreeNAS and am looking to setup a system for my home. Looking to use FreeNAS-8.0.4-RELEASE_MULTIMEDIA-p1-x64, as I would like to do utorrent, backup and perhaps iTunes. Right now for computers I have a personal computer and laptop I use. Then I have a server which has my movies on it and another computer that runs my cameras. So I figured instead of having data spread all over the place put it in one location, which makes backups easier also. Right now I am using 1.5TBs for my movies and music, which get read from often. For data I have about 50 Gb and for the cameras I need at least 250 Gb, which is written very often.

My question is on how to setup the harddrives for FreeNAS? Of course I am looking at RaidZ so I have redundency for my data, which will be backed up regularly also. I am concerned about access and speed. Should I have 3 separate drive systems for my files: 1 for data like pictures, documents, etc, 1 for movies and 1 for the cameras or can I put all on 1 and the access be fine?

Like I said I am new to this and would appreciate any suggestions. I have worked with Windows but not this system so it is all new to me. I have a Intel Core 2 4300 1.8 GHz processor and have 4 Gb Ram. I have 3-1.5TB drives in a Raid 5 for the movies and music right now. Data is on the personal computers. Thinking to purchase some 2TB drives for RaidZ1 config but do I need that for my movies or should I just go with a mirror?

Thanks

 

[Joshua Parker Ruehlig]

I'll first just tell you what I have learned that I wish I knew when starting with ZFS, then see if I can give you some suggestions.

___ATTAINED KNOWLEGDE___
*ZFS is less and more flexible then any other raid system, a zpool is a collection of vdevs (disks or mirrors or raidz), it can also have cache/log drives (usually SSDs), and spare drives (to take over when another disk fails)
-vdevs disk amount CANNOT change one created, vdevs CAN be added to a zpool and will increase the size of the pool, vdevs CANNOT be removed once added to a zpool, disks within a vdev CAN be replaced with larger disks
-the size of the vdev is = (smallest disk) * (number disks - redundant disks)
-zfs datasets live on top of your zpool and are like partitions, zpools can be snapshoted/deleted - take advantage of these, they are awesome!
-zfs volumes are like disks ontop of your zpool (used for iscsi)
-check if your disks are 4K (they likely are if they are newer) create vdevs with the force 4KB option, this CAN'T be changed later
-There is possibly a performance increase by using 2^n disks for data is a raidz1/2. This means 3/5/9 disks in raidz or 4/6/8/10 drives in a raidz2
-zfs checksums zpools/datasets/files. a scrub checks stuff against the checksums and fixes errors by grabbing a non corrupt version (from redundant copies created from raid/mirroring) - I RECOMMEND YOU SCHEDULE THIS ONCE A WEEK FOR CONSUMER LEVEL DRIVES, I found unrecoverable errors after not doing this for a few months. If you scrub often enough you will catch all errors. Whats interesting is on any other filesystem (not sure about BTRFS) errors created in corrupted files are completely undetected. you could have slightly modified files and never know it!

___PERSONAL RECOMMENDATIONS___
*8.0.4 multimedia is fine but I personally would wait for 8.2, it is 'almost' there and the plugin system is cleaner running in a freebsd jail then on the actual freenas system.
*What would be ideal for you is to have a zpool with several datasets on top of it. You can have different network shares, compression level, and permissions for each share.
*you could even create uses to have owners of shares, then setup rsync to automatically sync changed data from your client computer (not sure how this works with windows...)
*The problem I do see with your setup is having 3x1.5TB drives but wanting more 2TB drives, they create a vdev with unusable space. You could create a zpool with a vdev of 3x1.5TB in raidz, then add a vdev of 3x2TB into the same zpool. The zpool would then have 7TB usable space, each vdev could sustain a 1 disk loss.
*using the configuration above has some disadvantages... losing 2 drives in the same vdev causes a complete loss of the pool (not the case if you ran a raidz2 with 6 drives, though you'd lose space remember)
*using the configuration above has some advantages... By striping vdevs you would have increased IOPS performance. By creating seperate vdevs you could migrate your current data to your zpool.

PROPOSED STEPS (Not perfect, read protosds post below)
1) buy 3x2TB drives
2) create zpool with vdev of 3x2TB, move files over network onto your zpool(possibly into datasets).
3) connect 3x1.5TB disk to freenas machine
4) add 3x1.5TB disk as a raidz vdev to your zpool. zpool will re-stripe itself

PERFORMANCE
-File transfer performance will be likely limited by your network, when transferring large amounts of data use a Gigabit connection between the 2 machines. Wireless i almost always a limiting factor.
-Currently I have 5x2TB in a RAIDZ, local benchmarks show a read/write of 300MBps / 280MBps
-My Network Layout: Desktop <=> Unmanaged Gigabit Switch <=> FreeNAS (over NFS, for Windows use CIFS)
-I get around 60MBps=480Mbps read/write over the network from the desktop. This can vary from 40-75MBps=320-600Mbps depending on file sizes.
-I believe I am CPU bound on my FreeNAS side, your CPU is 45% faster then mine (e350 vs Core2 E4300) and may perform better.

 

[ProtoSD]

Great post Joshua, a great reference for noobs wanting to understand ZFS. There are a few typos I fixed ;)

. You could create a zpool with a vdev of 3x1.5TB in raidz, then add a vdev of 3x2TB into the same zpool. The zpool would then have 7TB usable space, each vdev could sustain a 1 disk loss..

It's a bad idea to have vdevs of unequal size in a pool, while it's possible and I had the same idea when I started, it's like having 1 wrong sized wheel on your car. I'd have to dig, but there's another thread that talks about why, that cleared it up for me.

4) add 3x1.5TB disk as a raidz vdev to your zpool. zpool will re-stripe itself

If you start out with 2 vdevs your data will be striped, but if you add vdevs later that's not always the case, it doesn't automatically restripe your data when you add a vdev, it does it as you add data.

One other thing, if you lose any vdev in a pool, you lose your entire pool.

 

[Joshua Parker Ruehlig]

thanks for the clarification Protosd.
I guess then Trioxide should stick with 1.5TB then? O maybe buy 2TB and sell his 1.5TBs after, that just seems like a hastle...

 

[Trioxide]

Joshua, thanks for sharing that. Not sure it answered what I was asking though. Do I need 3 different zpools, with raidz1 so needing at least 3 disks per zpool, for the different systems: data, movies and cameras? I ask that as they do different things and I don't want performance to be affected when movies are being played and the camera server is trying to write data, which it does very frequently. So if I understand that right I would have say 1 zpool with 3 250GB HDs for the data, another zpool with 3 2TB HDs for the Movies and a final zpool with 3 250GB HDs for the camera data?

Or are you saying I just need 1 zpool with say 3 3TB HDs and performance will be fine?

 

[ProtoSD]

Trioxide,

I think you'll get better performance with a single pool containing 3 vdevs. Even if you had 3 separate pools, they'll most likely share the same controller anyway. You're reads and writes from all your different systems will still be focused through your controller no matter how you slice up your disks, but looking back at your hardware I'd say your biggest weakness is a toss up between your processor and your RAM.

I think mirrors *might* give you a slight boost in performance, and you can also have pools of mirrors. I think you might need to do some experimenting with your hardware to see what's possible, it is a little on the low-end.

 

[Joshua Parker Ruehlig]

Joshua, thanks for sharing that. Not sure it answered what I was asking though. Do I need 3 different zpools, with raidz1 so needing at least 3 disks per zpool, for the different systems: data, movies and cameras? I ask that as they do different things and I don't want performance to be affected when movies are being played and the camera server is trying to write data, which it does very frequently. So if I understand that right I would have say 1 zpool with 3 250GB HDs for the data, another zpool with 3 2TB HDs for the Movies and a final zpool with 3 250GB HDs for the camera data?

Or are you saying I just need 1 zpool with say 3 3TB HDs and performance will be fine?

No, I don't recommend having 3 different zpools. I recommend one zpool consisting of 1 or more vdev. on top of the zpool you can create your datasets for data, movie, camera. sorry if i didn't make it clear, lol.

With 3 disk in a zpool performance should still be fine streaming movies and uploading pictures. Streaming a 1080p movie really only takes 10-20mbps. on a gigabit network your only taking 1-2% of your theoretical max from streaming a 1080p film. At my house I often have 2 1080ps streaming while uploading encoded movies (at 60-70MBps) and downloading movies from the internet (5MBps). I have never had any stuttering caused by disk / network lag when playing a movie. As long as were talking about a wired Gigabit network.

If I were you I'd just make a 3x2TB zpool. and a 6x250GB zpool (assuming those are the disk you have/will have).

 

[cyberjock]

Here's my configuration:

-8x1.5TB drives on a RAIDZ2(vdev1)
-8x1.0TB drives on a RAIDZ2(vdev2)

All drives share the same zpool and I can have 4 drives(2 in each vdev) fail without a loss of data. Benchmarks on the machine with a dual core i3 processor and 16GB of RAM were 400MB/sec+ doing DD test with 100GB test read and write. I can get 90+% utilization on both Intel NICs simultaneously. This would be equivalent to streaming something like 100+ 1080p videos simultaneously.

The CPU should be fine for what you intend to use it for. However, your RAM is a little low. If you can go with more than 4GB I would.

 

[Trioxide]

Thank you everyone for your help. So it looks like I can keep my 3-1.5TB drives for 1 vdev and buy some more drives for another vdev to increase the size of the pool, as long as they are all the same size in the vdev, to make 1 zpool? And it is suggested I upgrade my RAM to say 6GB?

 

[cyberjock]

Thank you everyone for your help. So it looks like I can keep my 3-1.5TB drives for 1 vdev and buy some more drives for another vdev to increase the size of the pool, as long as they are all the same size in the vdev, to make 1 zpool? And it is suggested I upgrade my RAM to say 6GB?

Exactly! The more RAM the better. There is no such thing as too much with regards to RAM.