Need help to build a new server

[Proxipupuce]

I haven't used the jails since version 9. Isn't the iocage folder actually a dataset? If it is, you could just snapshot and zfs send/recv.

I confess I don't really know. When I did my research, I found that different people were using the two methods I suggested. However, on the "zfs send/recv" method some people say they've had problems with the "zfs destroy" command to delete the old iocage folder.

The "iocage export/import" method:
https://www.truenas.com/community/threads/moving-iocage-jails-to-new-pool.92111/

The "zfs send/recv" method:
https://www.truenas.com/community/threads/11-2-migrating-the-iocage-dataset-to-an-other-pool.70168/

In addition, these two topics refer to "fstab" for jails. My understanding was that this referred to mount points, and that I could just modify them from the GUI. Am I right?

 

[Etorix]

However, one thing I don't understand is how to get the login information, like IP address, login and password?

Old Supermicro boards go by ADMIN/ADMIN by default (it's in the manual); newer boards come with a default unique password, which can be found on labels on the board itself and its box.
IP address defaults to DHCP, so you query your DHCP server (provider box/modem) for that. It's also displayed on the BIOS "welcome" page if you plug in a monitor in despair.

 

[chuck32]

Okay, how about fractal nodes?

Back when my server was miniITX I owned that case (still do but not the a server anymore).

The mechanism to lift off the top could have been a little bit better, but it's not the same as lifting just a side panel so...

Other than that I have been really content with the case and I like the design to insert the HDDs.

I haven't followed the whole thread, just make sure you pay attention to CPU cooler height the max specified height is not the same when you put 6 HDDs in it. But that's all written and shown in the manual.

I haven't had problems with thermals or anything. I would recommend it for the form factor. And fractal has always had a nice build quality in my opinion.

 

[Proxipupuce]

Old Supermicro boards go by ADMIN/ADMIN by default (it's in the manual); newer boards come with a default unique password, which can be found on labels on the board itself and its box.
IP address defaults to DHCP, so you query your DHCP server (provider box/modem) for that. It's also displayed on the BIOS "welcome" page if you plug in a monitor in despair.

Okay, thanks!

 

[Proxipupuce]

Back when my server was miniITX I owned that case (still do but not the a server anymore).

Thanks for your feedback! From what I can see, yes, the Fractal Nodes boxes are really very good.

 

[Proxipupuce]

Sorry to ask all these questions, but I still have one.
Let's take the Supermicro X11SSH-F motherboard as an example. It has eight SATA ports and one M.2 NVME port.

Currently, I have :
3x 3 TB western digital red (RAIDZ1)
2x 120 GB SSD SATA
1x 240 GB SSD M.2 NVME

In the next few years, I'd like to upgrade to :
6x 6 TB HDD RAIDZ2
2x 120 GB SSD SATA (Mirror)
2x 240 GB SSD M.2 NVME (Mirror)

Here, all eight SATA ports and the M.2 port would be filled.

1) So I'd like to mirror a second M.2 NVME SSD with my current M.2 NVME SSD, as no M.2 slots will be available, is it a good idea to buy an M.2 to PCIE adapter?

2) My goal would then be to have :
- the mirrored boot pool
- the plug-in/jail pool also mirrored
- the storage pool in RAIDZ2
Is this a good idea?

3) Finally, where is the best place to install truenas? On the two mirrored M.2 NVME SSDs, or on the two mirrored SATA SSDs? Whether it's one or the other, the remaining choice would be for plugins/jails.

 

[Etorix]

1) yes (or get a motherboard with more M.2…)
2) reasonable ideas (especially the raidz2 storage; mind that you'll temporarily need all SATA ports to do the move from 3w Z1 to 6w Z2—with one degraded pool)
As an alternative to mirorring, the app pool could be backed up to the HDD pool.
A redundant boot is not really useful in a home NAS; just keep a copy of the configuration file outside of the NAS, and reinstall in case of boot drive failure.
3) Mostly irrelevant. (My own choice would be: Both on M.2, and keep all SATA ports for HDDs.)

 

[Proxipupuce]

A redundant boot is not really useful in a home NAS; just keep a copy of the configuration file outside of the NAS, and reinstall in case of boot drive failure.

Yes, that's what I'm doing right now! If it's not useful, that's fine, it's less money. ^^

3) Mostly irrelevant. (My own choice would be: Both on M.2, and keep all SATA ports for HDDs.)

I agree with you. In fact, I was saying this for several reasons:

- I don't intend to use eight disks, even in several years' time, six maximum.
- A case with eight bays will have little chance of fitting where I want to put it (I've looked, a node 804 won't fit). So, I compromise by taking bigger disks, but fewer in number.
- At the moment, I have 6 TB usable and I fill them to 48%, with 6×6 TB in RAIDZ2, I'd have 22-23 TB usable... I'm not even sure I'll be able to fill them before the end of their lifetimes.
- I already have the two 120 GB SSDs at my disposal, and I think it's silly to let them gather dust.

On the other hand, if mirroring the prisons isn't useful, maybe use the other SSD for caching?

 

[Etorix]

L2ARC is only to be considered with 64 GB RAM or more, AND if arc_summary indicates that it would still be useful. If it is, you'd want it on NVMe.

 

[Proxipupuce]

Okay, it's clearer to me now.

Thanks to all of you for helping me, I now have a clearer idea of what I want in terms of equipment.

I've already found a case: Nanoxia Deep Silence 4 (It's perfectly the right size, with the right number of bays and good soundproofing). What's more, it has good reviews on the test sites.
And I've also chosen my power supply: Corsair RM650 2023

1) Any opinion on these two components? For the rest, I'll keep looking and thinking, then I'll come back here ^^.

2) A question about IPMI, if ever, for X or Y reason, I can't connect, can I plug in a monitor to access the BIOS, even without a graphics card and IGPU? In fact, my question is: does IPMI (the Aspeed AST2400 BMC chip) replace a graphics card/IGPU for connecting a monitor?

3) Last question, this time about jails: I've read many threads referring to "fstab" for jails. My understanding was that this referred to mount points and that I could simply modify them from the GUI. Am I right?

 

[chuck32]

1) Any opinion on these two components? For the rest, I'll keep looking and thinking, then I'll come back here ^^.

I lost track of your selected hardware. Here is a PSU sizing guide.

In the next few years, I'd like to upgrade to :
6x 6 TB HDD RAIDZ2
2x 120 GB SSD SATA (Mirror)
2x 240 GB SSD M.2 NVME (Mirror)

Possible that you may want to go like 750W instead of 650W. I personally have no experience with Corsair PSUs. Seasonic PSUs are often recommended here and I use them exclusively. Plus they come with 10 - 12 year warranty. Maybe see if a Seasonic Focus PX-750 / Seasonic PRIME-PX-750 matches your budget. That's just my 0.02, The RM650 may be fine.

can I plug in a monitor to access the BIOS, even without a graphics card and IGPU?

Yes.

3) Last question, this time about prisons: I've read many threads referring to "fstab" for prisons. My understanding was that this referred to mount points and that I could simply modify them from the GUI. Am I right?

You mean jails? I have no experience in that, usually you should be able to manage everything in the GUI instead of digging around the system files.

 

[Proxipupuce]

Possible that you may want to go like 750W instead of 650W. I personally have no experience with Corsair PSUs. Seasonic PSUs are often recommended here and I use them exclusively. Plus they come with 10 - 12 year warranty. Maybe see if a Seasonic Focus PX-750 / Seasonic PRIME-PX-750 matches your budget. That's just my 0.02, The RM650 may be fine.

Thanks for your feedback! I'll take a look at these power supplies.

You mean jails? I have no experience in that, usually you should be able to manage everything in the GUI instead of digging around the system files.

I'm having trouble with jails myself, but I can't see myself switching to TrueNAS Scale to be able to replace them... I don't think I know enough about it. But you're right, if "fstab" does refer to mount points, then yes, I can manage them from the GUI.

Yes.

So I guess this allows the motherboard to boot without a graphics card or IGPU? I absolutely want to avoid a problem I once had where my motherboard couldn't boot due to a lack of graphics support.

 

[chuck32]

But you're right, if "fstab" does refer to mount points, then yes, I can manage them from the GUI

Fstab is the configuration file on Unix systems where partitions and mounts are handled.
I do not know if you need to modify it in order to use jails. I can't imagine it's a hard requirement but even if you would need to work on a fstab file, it's not that hard ;)

o I guess this allows the motherboard to boot without a graphics card or IGPU? I absolutely want to avoid a problem I once had where my motherboard couldn't boot due to a lack of graphics support.

Yes, it's a server board, not every server needs a dedicated GPU unlike most consumer builds.

 

[Proxipupuce]

Fstab is the configuration file on Unix systems where partitions and mounts are handled.
I do not know if you need to modify it in order to use jails. I can't imagine it's a hard requirement but even if you would need to work on a fstab file, it's not that hard ;)

All right, I'll give it a try then!

Yes, it's a server board, not every server needs a dedicated GPU unlike most consumer builds.

yes, that's what I thought!

 

[Proxipupuce]

So, now that I've chosen my power supply and case, I set about looking for the other components.
Unfortunately, I'm having a lot of trouble finding motherboards at a reasonable price.

However, a few days ago, I mentioned this €350 offer:

Motherboard: Supermicro X11SSH-F
CPU: Intel Xeon E3-1240v6
RAM: Samsung 4 x 16 GB ECC DDR4-2666

Today, I can finally get it for less than 300€.
I know that the next generation would have been a better choice, as Etorix had said, but all the motherboards of this generation that I've seen exceed 250€, without RAM or processor.

So I'm wondering if, at 300€, it's worth my while to buy the components written above.

That would give me the following configuration:

- Motherboard: Supermicro X11SSH-F
- Processor: Intel Xeon E3-1240v6
- RAM: Samsung 4 x 16 Go ECC DDR4-2666
- System installed on a PNY 240 GO NVME SSD connected on the motherboard (PCIE x2, no x4)
- Storage: 3x 3 TB western digital red RAIDZ1 connected in SATA 6 Gb/s (later 6x 6 TB RAIDZ2)
- Iocage on 120 Go SSD connected in SATA 6 Gb/s
- PSU : Corsair RM650 2023 / Seasonic Focus PX-750
- Case: Nanoxia Deep Silence 4

What do you think of this configuration? Do you have any suggestions?

 

[Etorix]

Fair configuration for a fair price in the current market.
A x2 slot is more than enough for boot. For more NVMe drives there are three PCIe slots, so up to three drives on plain and cheap adpaters before you have to resort to PLX switches; that makes for many possibilities.

 

[Proxipupuce]

Hello, I hope you're well!

I have several questions regarding the new configuration I want to have and I need some answers!

1) I'm currently using an Intel I210 T1 GBE (1Gb) PCIE network card in my server. Since the network chips on my new motherboard will also be Intel chips, is there any reason why I'd rather use my PCIE network card than the motherboard's built-in network card?


2) I've read that, when setting up a server, it's necessary to test the memory for at least 24 hours with MemTest. However, since ECC RAM corrects errors, is it necessary to carry out this test with ECC memory? Won't ECC RAM distort the test by correcting errors?

3) I'm currently using these SATA cables: https://amzn.eu/d/fTuwanq
I was wondering, does the choice of SATA cables matter? Are there better cables out there, or do cables like mine do the trick?

4) From what I know, it's generally not advisable to update the BIOS if everything's working fine. However, is this also true for servers, where you should always update the BIOS?

5) Reading the guide to power supply sizing, I read that a drive could consume up to 3 A (for 12 V, 35 Watts) at start-up. So I was wondering, is it risky to use the four connectors of a cable to plug in 4 hard disks? Will it heat up the cable or is there no problem?

Thank you very much for your help!

 

[Ericloewe]

1) I'm currently using an Intel I210 T1 GBE (1Gb) PCIE network card in my server. Since the network chips on my new motherboard will also be Intel chips, is there any reason why I'd rather use my PCIE network card than the motherboard's built-in network card?

Not really, no.

) I've read that, when setting up a server, it's necessary to test the memory for at least 24 hours with MemTest. However, since ECC RAM corrects errors, is it necessary to carry out this test with ECC memory? Won't ECC RAM distort the test by correcting errors?

Yes and no. Memtest86 can read the memory controller data to determine when errors were corrected, and thus alert that an error was found, even though it was corrected. Memtest86+ (not the plus) also seems to be slowly acquiring this capability. Specifics will vary.
Theoretically, it should even be possible to inject memory errors, but that functionality is apparently commonly disabled in system firmware.

3) I'm currently using these SATA cables: https://amzn.eu/d/fTuwanq
I was wondering, does the choice of SATA cables matter? Are there better cables out there, or do cables like mine do the trick?

Short is better, beyond that they're all the same crap, for better and for worse.

4) From what I know, it's generally not advisable to update the BIOS if everything's working fine. However, is this also true for servers, where you should always update the BIOS?

I view that as crap advice perpetuated by people who don't want to be bothered dealing with the occasional failed update, leaving users hanging without various fixes and improvements. Update away and update often, but don't do stupid things like removing power during the process.

5) Reading the guide to power supply sizing, I read that a drive could consume up to 3 A (for 12 V, 35 Watts) at start-up. So I was wondering, is it risky to use the four connectors of a cable to plug in 4 hard disks? Will it heat up the cable or is there no problem?

Typically not a problem with modular PSU cables, but do be careful with Y cables and that sort of thing.

 

[Etorix]

2) I've read that, when setting up a server, it's necessary to test the memory for at least 24 hours with MemTest. However, since ECC RAM corrects errors, is it necessary to carry out this test with ECC memory? Won't ECC RAM distort the test by correcting errors?

Memtest tests that memory is operating to specifications. There are bad memory modules in the wild, including ECC modules. Not many hopefully, but it's best to check.

 

[Proxipupuce]

Yes and no. Memtest86 can read the memory controller data to determine when errors were corrected, and thus alert that an error was found, even though it was corrected. Memtest86+ (not the plus) also seems to be slowly acquiring this capability. Specifics will vary.
Theoretically, it should even be possible to inject memory errors, but that functionality is apparently commonly disabled in system firmware.

Okay, thanks!

I view that as crap advice perpetuated by people who don't want to be bothered dealing with the occasional failed update, leaving users hanging without various fixes and improvements. Update away and update often, but don't do stupid things like removing power during the process.

On this point, I totally agree with you. I've always updated the bios of my computers as soon as they were released, for example for AGESA or security patches. I was just wondering if this was also true for servers. There will be no problem with power disconnection during the update, I use UPS on all my devices.