Help with space calculation for ZFS RAIDZ2.

[TasMot]

Hi,
This is a new to ZFS question. I had a freenas server that was cobled together from old parts. When I realized that using Carbonite to make back-ups of my systems was cost prohibitive (18 virtual machines plus 2 physical machines at $59 per was going to be $1,180/year for backups), I decided to go with a FreeNAS system to hold my backups instead. So I bought some new guts, motherboard (GIGABYTE GA-970A-UD3 AM3+ AMD 970 SATA 6Gb/s USB 3.0 ATX AMD Motherboard), CPU (AMD FX-4100 Zambezi 3.6GHz (3.8GHz Turbo) Socket AM3+ 95W Quad-Core Desktop Processor FD4100WMGUSBX), RAM 8 Gig of RAM (Team Xtreem Dark Series 8GB (2 x 4GB) 240-Pin DDR3 SDRAM DDR3 1600 (PC3 12800) Desktop Memory Model TXD38192M1600HC9DC-D ) and disks (Western Digital Caviar Green WD20EARX 2TB 64MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive -Bare Drive - OEM ). Now I knew I would lose some space for going with RAIDZ2. However; what I am showing is that after getting everything set up, I created the RAIDZ2 array and ended up with 3.5 TB of space. Is this correct? I started with 8TB and ended up with less than half. That doesn't seem quite right. If I were to quickly get another 2 TB drive and add it, would that bump up my space to 5.5 TB?

Thanks for your help,

Tom

 

[ben]

Same as other people on the forums who asked this: 1) You lose two disks to parity 2) you lose another small amount of each disk to filesystem overhead and scratch space reservations 3) the drive manufacturer uses base-10 terabytes and the OS uses base-2 ones, which should more properly be called "Tibibytes". This is the primary cause of the discrepency between 4TB and 3.5TB.

Addendum: you can't expand existing RAID-Z* arrays, and a 5-disk RAID-Z2 is suboptimal anyway. If you did add that disk, it would increase your available space to about 5.25TB.

 

[TasMot]

OK, if 5 is suboptimal, would 6 be better (might have to hide it from the wife, but I have room for the drives in the same machine) and how much space would that give me?

 

[StephenFry]

OK, if 5 is suboptimal, would 6 be better (might have to hide it from the wife, but I have room for the drives in the same machine) and how much space would that give me?

Yes, 6 is an optimal number. The next optimal number is 10. I think a LOT of people on this forum use a 6-drive RAID-Z2. It's a proven configuration. You'd still lose 2 drives, so 4 are left: 4x2=8TB. Like ben saysm, because of the way FreeNAS reports space, it will probably say something like 7.2TiB.

For example, my NAS currently reports in Windows as having 4,947,573,069,824 bytes free. FreeNAS tells me I barely have 4.5TiB left.

In short: forget about all this. With RAID-Z2 you always lose two drives, so the rest will -for 99%- be usable for your data. As I said, 6 or 10 are considered optimal for reasons you can read in the 'ZFS best practice guide'. No one will hold it against you if you choose a different number though... I'd recommend against going above 10 (8, if I'm honest). That's RAID-Z3 territory right there.

 

[ben]

Optimal drive numbers are 2^x + parity level (so 3, 5, 9 for single parity, 4, 6, 10 for double parity). In all cases you get drive size in base-2 bytes minus parity drives minus some misc. overhead. So you'll end up with more like 7TB in that case (RAID-Z2, 4 disks worth of data, 2 lost to parity).

 

[StephenFry]

Now that is an exact answer. Don't forget that 7TB (as reported by FreeNAS) will be well over 7,800,000,000,000 bytes. At least, that's what I ended up with.

 

[TasMot]

Thank you to everyone who helped, I really appreciate it. I really appreciate the facts and not an instant redesign. I wonder how the info on how to figure out the amount of space could be set up as an obvious "sticky" posting at the top since this seems like a frequent question (based on the answers here). As for my part, now I've got to convince my wife this was a good idea after all, and spend the $250 on 2 more drives.

Thanks again, I appreciate the very quick help.

Tom

 

[ben]

Sounds like a good addition to the FAQ, I'll look into it (there have certainly been enough threads on the subject.)

 

[Patrick van Eck]

Hi All,

The thread is a good read thanks
I am a newbie with FreeNAS so please excuse my ignorance however wanted to ask I have a configuration of the following

HP N54L Microserver
Dual Core AMD Turion 2,2Ghz CPU
5x4TB Western Digital RED Drives
16GB DDR3 ECC RAM

What would be the best configuration available in FreeNAS using RAIDZ1, RAIDZ2 with capacity = performance and stability in mind
The primary use of the server is to store multimedia

Pat

 

[AiRLAC]

RAIDZ1 is not safe anymore for any drive over 1TB.
Go with RAIDZ2, in your home-user case you can ignore the 6-disk-for-RAIDZ2 rule of thumb :)

 

[TasMot]

One more thing I have found out after lots of pain is that if you are going to be doing a lot of large writes (as you indicated multimedia), if you are going to be doing a lot of writing to the disks you need a ZIL. If you instead mean a write once stream many, then you can ignore this. I encountered a huge problem with using my FreeNas array as a backup device. I created the 6 disk array mentioned above but kept running into lots of problems with writing backups to the array due to the way VmWare writes large "chunks" each time in a synchronous way. I ended up needing to spend another $100 on a new controller and SSD drives to put in as a ZIL device to alleviate HUGE write delays. I will be implementing this over the next week.