More RAM is more better than L2ARC. Even when you put the L2ARC on fast SSDs. Unless you have specific performance requirements and KNOW SSD L2ARC or SLOG will improve your performance, leave them out of the configuration. They add complexity and cost and often don't do what you think they will do. Plus, with a backup server, you're not reading all that often, right? The chances of something you need being in L2ARC is low anyway - you've got 100T of data and a large SLOG would be 500G or 1T? That's needle in a haystack. Also, if you've put 500G or 1T of RAM into the server, it would be faster coming out of ARC on RAM than L2ARC on SSD.
SLOG is good for bursty writes that need immediate confirmation. The characteristics of a backup repository don't match the use case of SLOG as the writes are large and continuous. SLOG is going to fill and you're still going to be waiting on the conventional disks to write. I fear you'll burn through SSDs and not see any performance benefit.
A single ZFS pool is fine. A single VDEV of 22 drives is bad.
You've got 45 bays to play with. I'd do 9 drives per RAIDZ2 VDEV. Start out with two VDEVs in your pool. This balances performance with space efficiency and 45 is evenly divisible by nine which is pleasing to my OCD.
- 9 drives * 8TB - two parity drives = 56T = VDEV
- 9 drives * 8TB - two parity drives = 56T = VDEV
- 56T VDEV + 56T VDEV = Storage Pool = 112T
Two RAIDZ2 VDEVS will give you roughly double the pool performance (IOPS and throughput) than one RAIDZ2 VDEV of 22 drives (give or take).
When you need additional space, toss another nine drives into the server, create a RAIDZ2 VDEV and add that VDEVto the pool. With each VDEV you add to the pool, performance will increase. At nine drives per pool, with 8TB drives, you've got enough bays for 280T, give or take.
It is easy to add capacity to FreeNAS when you have plenty of bays. If you expect your first year's storage needs to be under 56T, start with one nine-drive VDEV of 8T drives. Next year, you'll pay the same price for 10T drives and you can add them. Need more space a year later? Add nine 12T drives to the pool.
I have no idea which HGST capacity is the most reliable. Their reputation is fairly solid so my only hesitation would be in buying the largest, latest drive - let someone else pay a premium to test the new drive size. Buy on price per byte in any of those capacities and you'll likely be fine.
Cheers,
Matt
