Backup write performance.

[Rodrigo]

Hi,

I am building a new server to receive my backup databases using NFS.
I will be have 20 remote server writing on freenas NFS share at same time.
The volume of data per incremental backup (monday-saturday) is 200G per day and full backup is 2.5T (Sunday).

My config:

Freenas Verison: FreeNAS-11.2-U5 (c129415c52)
Mother Board Supermicro X8DT3
x2 E5620 2.40GHz
32GB Memory RAM ECC
1x 10GB DUAL Lan interface Broadcom Limited NetXtreme II BCM57810, Jumbo Frame enable. (Using pcie v2 16x)
1x Lsi 9211-8i 6gbps in IT MODE/JBOD. ( Using pcie v2 8x )
2X SSD 120GB for boot, plugged in SATA onboard ports.
4x SATA 6TB ARCHIVE 5900RPM in RAID-Z plugged in LSI Controller, I will expand one more vdev with 4x disks in RAID-Z

I think needs the Zil cache for write performance, and I will looking for Intel Optane 900P 280GB.

Can I put the 900P in a pcie v2 8x ? Or I have another arrange in this hardware for better performance ?
Maybe another type device for Zil cache ?

Best Regards,


Rodrigo

 

[lightwave]

Hi Rodrigo,

Based on a quick back-of-the-envelop calculation, I'm not sure the ZIL will make a whole lot of difference for your use case. Adding a second vdev would probably be a better way to increase performance, especially for the Sunday full backups.

@Everybody: I'm relatively new to FreeNAS, so feel free to correct me I'm mistaking

Also, I would consider going for RAIDZ2 in your case to be able to handle a double disk failure. While unlikely, it is not unheard of for a second disk to fail during the strain of re-silvering from the first disk failure.

My calculations:

The WB RED disks should be able to write around 150 MB/s (based on some benchmarks I found) which would give you approximately the same write performance for the RAIDZ. The ZIL should be able to write at around 2000 MB/s. Without ZIL, the RAIDZ becomes the bottle neck. With the ZIL, the network will be the bottle neck at around 800 MB/s.

I will assume that the clients can deliver data quickly enough to saturate your FreeNAS setup throughout the entire backup session. This is highly unlikely in a real world scenario as the different backups are bound to finish at different times. This means that the ZIL will provide a smaller improvement than calculated below unless the clients deliver multiple heavy bursts of data that could have been held by the ZIL but are too big to cache in memory (i would consider this unlikely).

Incremental backups: The ZIL will be able to hold the entire backup. Hence, the network will be the limiting factor and the backup will take about 4 minutes in theory (200 GB at 800 MB/s). Without the ZIL, we will be down at 150 MB/s resulting in a backup time of 22 minutes. In reality, the clients are unlikely to be able to feed at 800 MB/s so the improvement provided by the ZIL will likely be smaller.

Full backups: Here the backups will fill the ZIL after around 10% of the backup has been taken. After this point, the RAIDZ will become the bottle neck. Regardless of whether you use the ZIL or not, the backups will take at least 4.5 hours to complete (2.5 TB at 150 MB/s) in a perfect world, and most likely significantly longer in reality.

Adding a second RAIDZ vdev should, in theory, increase the write performance to 2 x 150 MB/s as the writes can be interleaved between the vdevs.

 

[Rodrigo]

Hello @All

Thank you for answer @lightwave

For the raiz-z2 I understand but a have a problem with space, my server chassis supports only 8 disks bays. and I need the space for the backup policy retention.

I forgot to say this system will be replicate to another freenas in another site for the backup redundancy, So I think raid-z1 is enough for this case.

I will considered and learning with your calculation, thanks for this. But I still with the pci-e question:

Can I put the 900P in a pcie v2 8x ? I will be have problem with this ?

Thanks advanced.

 

[Arwen]

...
4x SATA 6TB ARCHIVE 5900RPM in RAID-Z plugged in LSI Controller, I will expand one more vdev with 4x disks in RAID-Z
...

Are these Seagate Archive SMR disks?

If so, they perform POORLY for continuous write mostly applications, like backups. I have an 8TB model for my backups, and it's writing is slow. Except that in my case, I am am not concerned about backup windows. It simply takes as long as it needs to take.

I don't have any information on the 900P in a PCIe v2 slot with 8 lanes, other than it would likely only use 4 lanes.

 

[jenksdrummer]

PCIe devices that use x-lanes will continue using x-lanes regardless of whatever max generation spec they are offered; IE; a spec device @ PCIe Gen3-x4 in a Gen2 slot will still be x4. Or a device that's Gen2-x8 in a Gen3 slot will still consume x8, but run at Gen2 speed. Least spec/lanes wins...