billhickok
Dabbler
- Joined
- Oct 8, 2014
- Messages
- 36
The best use case for ZFS with 4TB+ HDD's is archival large file storage, where you are mostly laying down large items and letting them sit there for years. Small file storage is more difficult for spinning media because of the seek overhead. At a point, you will lose the ability to store and retrieve the files before the drive's expected service life is reached... if you do the math, if you are storing 4KB files on an 8TB HDD, assuming you can write 100 of them per second (controlled by seek speed), doing nothing else, it'd take 250 days to fill the disk. I don't care to discuss the fact that it is a contrived example and not realistic from some points of view - it demonstrates the problem spinny rust faces when storing small files in the long term. Also, the ZFS scrub mechanism is a metadata traversal so it tends to suffer on pools storing lots of smaller files.
I just ran into this myself. I had two servers running an E3-1230v2 with 32gb ecc ram, 12x3tb hdd, and performance tanked. I did some testing and determined I needed more ram but was capped by the motherboard. So I had to upgrade; went with an E5-1620v2 with 64gb of ram and things are faster than they have ever been. The board will support 512gb of ram but I doubt I'll put more then 128gb in it. Cost me quite a bit but it was still cheaper then buying something off the shelf.
Sorry, I should point out that these servers aren't my home use they are deployed in a business. My home server is setup with the 1gb ram per 1tb of storage and has operated without issue for years now.
This makes me a bit reluctant about my plan to do 10 x 4TB in RAID Z2. I do have enough data to eventually fill up most of that space, and the majority of my data are small files (anywhere from 5-30mb files.) Though I likely won't be constantly utilizing (streaming or playing) most of the small files, they'll mostly just sit as an archive. I already have another NAS (Synology) which houses most of my big files.
I do plan on doing weekly transfers of around a thousand small files on the ZFS setup. Again, most of the time i'll probably be the only user utilizing this server as it's in a home environment. I really don't want to risk one day running into a brick wall and being forced to upgrade my hardware however. I'm planning on picking up all the hardware in the next few weeks and for me, it's gonna be a pretty significant investment in terms of money. What do you guys think...is there a decent chance i'll run into issues with 32gb of RAM for 10 x 4TB and 1 vdev?