Hello everyone,
I am new to TrueNAS and ZFS but I have read a lot about them because I have been trying to build a NAS. Since the main purpose of this built was for storage and less for fast read/writes or VMs I decided to go for HDDs for my main vdev. However, since I managed to find some SATA SSDs at good prices I am looking at the opportunity to go more towards a mixed build since it would be nice to have some better virtualization performance and faster metadata caching.
My motherboard is a Fujitsu D3417-B21 with 64GB ECC UDIMM RAM which has 6 native SATA ports. I also managed to find a Broadcom 9207-8i SAS2308 which plugged in a PCIE 3 slot will extend my SATA ports by 8. That leaves me with 14 SATA ports in total. My other PCIE slot I will use to upgrade networking to a 10Gbit NIC.
I have 6x WD Red Plus WD120EFBX 12 TB 7200 rpm 256 MB 3,5 (+1 spare) and 6x Samsung 960GB SSD SM863 MLC (+1 spare)
The CPU will be Intel Xeon E3-1275v6 if its interesting for some reason.
This brings me to the first question. I have seen a lot of people report improved performance with metadata special vdevs. Essentially the metadata can be saved in this special vdev and thus the read and write of files is much faster than simply doing so only with the HDD vdev. The only catch is that it adds an extra failure point to the system as if this vdev dies you lose the whole pool. This is not a big deal for me since I would anyway deploy a 3-way mirror to add some redundancy and make sure that I dont lose my pool. However, in all of the examples of this working I saw people were always doing it with nvme SSDs (which makes sense because nvme is around 5 times faster than a SATA SSD). However, my motherboard only has one m2 slot so thats not an option for me.
My question is: Can I even build the 3-way mirror metadata using SATA SSDs instead of nvme? Does it help me read and write from my HDD vdev faster?
Assuming that the answer to that question is yes I was thinking going with the following configuration:
A)
250gb nvme slot for booting (backed up constantly)
Pool1
- 6 wide raidz2 HDD vdev (backups and media)
- 3-way metadata special vdev (using the SATA SSDs)
Pool2
- 3 wide raidz1 SSD vdev (this for VMs and raidz1 because I dont care about redundancy as I will be keeping constant backups)
This configuration leaves me with 1 SSD and 1 HDD as backups and 2 free SATA slots in mobo.
If the answer to the above question is no then:
B)
2TB nvme slot for L2ARC ( im not expecting to use the full 64GB of ram so this might be counterproductive?)
Pool1
- 6 wide raidz2 HDD vdev (backups and media)
Pool2
- 6 wide raidz2 SSD (VMs and video editting)
Pool3
- 1 SSD for booting (no redundancy because of backups)
This configuration leaves me with 1 HDD and no SSDs as backup although I would have to buy one just to be sure. This would also leave only 1 SATA slot free on the mobo.
I hope you guys are able to give me a bit of direction because at this stage I am very torn between these two options. I think it really depends on if the SATA SSDs are suitable for the metadata special vdev.
If you have any other ideas please let me know.
All tips appreciated
I am new to TrueNAS and ZFS but I have read a lot about them because I have been trying to build a NAS. Since the main purpose of this built was for storage and less for fast read/writes or VMs I decided to go for HDDs for my main vdev. However, since I managed to find some SATA SSDs at good prices I am looking at the opportunity to go more towards a mixed build since it would be nice to have some better virtualization performance and faster metadata caching.
My motherboard is a Fujitsu D3417-B21 with 64GB ECC UDIMM RAM which has 6 native SATA ports. I also managed to find a Broadcom 9207-8i SAS2308 which plugged in a PCIE 3 slot will extend my SATA ports by 8. That leaves me with 14 SATA ports in total. My other PCIE slot I will use to upgrade networking to a 10Gbit NIC.
I have 6x WD Red Plus WD120EFBX 12 TB 7200 rpm 256 MB 3,5 (+1 spare) and 6x Samsung 960GB SSD SM863 MLC (+1 spare)
The CPU will be Intel Xeon E3-1275v6 if its interesting for some reason.
This brings me to the first question. I have seen a lot of people report improved performance with metadata special vdevs. Essentially the metadata can be saved in this special vdev and thus the read and write of files is much faster than simply doing so only with the HDD vdev. The only catch is that it adds an extra failure point to the system as if this vdev dies you lose the whole pool. This is not a big deal for me since I would anyway deploy a 3-way mirror to add some redundancy and make sure that I dont lose my pool. However, in all of the examples of this working I saw people were always doing it with nvme SSDs (which makes sense because nvme is around 5 times faster than a SATA SSD). However, my motherboard only has one m2 slot so thats not an option for me.
My question is: Can I even build the 3-way mirror metadata using SATA SSDs instead of nvme? Does it help me read and write from my HDD vdev faster?
Assuming that the answer to that question is yes I was thinking going with the following configuration:
A)
250gb nvme slot for booting (backed up constantly)
Pool1
- 6 wide raidz2 HDD vdev (backups and media)
- 3-way metadata special vdev (using the SATA SSDs)
Pool2
- 3 wide raidz1 SSD vdev (this for VMs and raidz1 because I dont care about redundancy as I will be keeping constant backups)
This configuration leaves me with 1 SSD and 1 HDD as backups and 2 free SATA slots in mobo.
If the answer to the above question is no then:
B)
2TB nvme slot for L2ARC ( im not expecting to use the full 64GB of ram so this might be counterproductive?)
Pool1
- 6 wide raidz2 HDD vdev (backups and media)
Pool2
- 6 wide raidz2 SSD (VMs and video editting)
Pool3
- 1 SSD for booting (no redundancy because of backups)
This configuration leaves me with 1 HDD and no SSDs as backup although I would have to buy one just to be sure. This would also leave only 1 SATA slot free on the mobo.
I hope you guys are able to give me a bit of direction because at this stage I am very torn between these two options. I think it really depends on if the SATA SSDs are suitable for the metadata special vdev.
If you have any other ideas please let me know.
All tips appreciated