Zepher8062
Cadet
- Joined
- Aug 12, 2021
- Messages
- 1
Howdy everyone! I've been casually getting back into the zone with server systems as of late with my free time (thanks covid!) but I am running across some head-scratchers that I cant just seem to ... well stop scratching my head about. I don't consider myself a rookie, but I definitely get lost in some of the terminology, so maybe I can get some pointers in the right direction. I'm not good at writing, so I apologize if this turns lengthy.
I have a VM server that I would like to eventually have its VMs hosted separately from the main box on a storage unit. Later on I will add SSD/NVMS cards and set a pool up for that, but for now I just want to get a mechanical array up for general site storage. Here come my TrueNAS box. I already have a Supermicro X10DRI as my mobo for the VMS, and I like it very much. After doing more research for the NAS box, I ended up deciding that the same board would be sufficient, and it wasn't too expensive either. Apart from only having a dual 1Gb NIC, it was a good choice, but I can grab a used dual-port 10Gb NIC for about $50. However, now the market prices are all wonky because of the pandemic and my options have changed. I like to buy new equipment (warranty purposes) but am not opposed to used/refurbed. Looking around eBay I found another option but I am worried about the architecture. I have never built a full-blown NAS before (just hobby stuff), and although I don't plan to push it enterprise-hard, I don't want to have any problems.
Here are three boards I am contemplating.
The X10DRI and X10DRI-T are basically the same board except the '-T' has a dual 10Gb NIC with the x540 chip instead of i350. I can't find a DRI-T for a respectable price though, so have to fall back on the base DRI here. The DRI-T4+, however, looks similar but is pretty different under the covers. It stouts a quad-10Gb NIC I/O and 8 additional slots for memory. It looks like a beast for the price, but the data architecture is all different and I'm worried I'll have bottlenecking issues in the future when I try to set up my m.2 cards for the SSD/NVME VM pool.
If you take notice, the LN4+ has all but one of its PCI slots routed through Processor 2 and leaves only 1 3.0x16 on Proc 1. The Base board has a fairly balanced setup. I assume this change is because the LN4+ needed more bandwidth for the Quad 10Gb ports but it's a bit annoying it kills the functionality of almost all the PCI cards. Right now I only have one processor available so I would be limited to just the one x16 slot, which I would need to use for my SAS HBA. (Speaking about the LN4+) With this general setup, the data would I/O from the NIC to Proc 1 to the RAM, to the HBA. I know there's more to it than that, but just being crudely general. The plan was later to add a second proc and install a dual or quad port m.2 card with NVME sticks, install a 10gb NIC on a proc 2x8 lane, and keep the majority of data traffic on the proc 2 side of the machine, hoping to minimize data through the QPI. I am trying to avoid all the data being unnecessarily funneled through a single processor or NIC chip, and from what I understand trueNAS has very good DP/MP support to prevent this.
Am I completely off here or is this a legitimate concern? I figured if I had a set of HBA/Storage and NIC on a single processor side it would tend to keep the data on that side for load balancing. e.g. a SSD pool on Proc 2 and HHD pool on Proc 1. Is the LN4+ a good board for the price? Maybe this would make a better VM board and I should just swap it with the one I already have. Then install a sperate NIC on each proc side and leave the on-board 1Gb's for kernel/web access.
Here is my proposed setup for reference (but mainly asking for advise on mobos at this time)
I have a VM server that I would like to eventually have its VMs hosted separately from the main box on a storage unit. Later on I will add SSD/NVMS cards and set a pool up for that, but for now I just want to get a mechanical array up for general site storage. Here come my TrueNAS box. I already have a Supermicro X10DRI as my mobo for the VMS, and I like it very much. After doing more research for the NAS box, I ended up deciding that the same board would be sufficient, and it wasn't too expensive either. Apart from only having a dual 1Gb NIC, it was a good choice, but I can grab a used dual-port 10Gb NIC for about $50. However, now the market prices are all wonky because of the pandemic and my options have changed. I like to buy new equipment (warranty purposes) but am not opposed to used/refurbed. Looking around eBay I found another option but I am worried about the architecture. I have never built a full-blown NAS before (just hobby stuff), and although I don't plan to push it enterprise-hard, I don't want to have any problems.
Here are three boards I am contemplating.
The X10DRI and X10DRI-T are basically the same board except the '-T' has a dual 10Gb NIC with the x540 chip instead of i350. I can't find a DRI-T for a respectable price though, so have to fall back on the base DRI here. The DRI-T4+, however, looks similar but is pretty different under the covers. It stouts a quad-10Gb NIC I/O and 8 additional slots for memory. It looks like a beast for the price, but the data architecture is all different and I'm worried I'll have bottlenecking issues in the future when I try to set up my m.2 cards for the SSD/NVME VM pool.
If you take notice, the LN4+ has all but one of its PCI slots routed through Processor 2 and leaves only 1 3.0x16 on Proc 1. The Base board has a fairly balanced setup. I assume this change is because the LN4+ needed more bandwidth for the Quad 10Gb ports but it's a bit annoying it kills the functionality of almost all the PCI cards. Right now I only have one processor available so I would be limited to just the one x16 slot, which I would need to use for my SAS HBA. (Speaking about the LN4+) With this general setup, the data would I/O from the NIC to Proc 1 to the RAM, to the HBA. I know there's more to it than that, but just being crudely general. The plan was later to add a second proc and install a dual or quad port m.2 card with NVME sticks, install a 10gb NIC on a proc 2x8 lane, and keep the majority of data traffic on the proc 2 side of the machine, hoping to minimize data through the QPI. I am trying to avoid all the data being unnecessarily funneled through a single processor or NIC chip, and from what I understand trueNAS has very good DP/MP support to prevent this.
Am I completely off here or is this a legitimate concern? I figured if I had a set of HBA/Storage and NIC on a single processor side it would tend to keep the data on that side for load balancing. e.g. a SSD pool on Proc 2 and HHD pool on Proc 1. Is the LN4+ a good board for the price? Maybe this would make a better VM board and I should just swap it with the one I already have. Then install a sperate NIC on each proc side and leave the on-board 1Gb's for kernel/web access.
Here is my proposed setup for reference (but mainly asking for advise on mobos at this time)
CPU: Intel® Xeon® Processor E5-2670 v3 <- These are really cheap right now $100-$150 New
HBA: SM AOC-S3008L-L8e
NIC-2: SM AOC-STGN-I2S
HDD: SAS3 12gb/s Not Decided (Recommendations appreciated)
Chassis: SM SuperChassis 826BE1C-R920LPB
Thanks for any help! Maybe someone can make sense of my unorganized madness of thought.
