Xeon Gold 6144. My first NAS

[Ericloewe]

Not that it will matter for SATA hard disks, but the server specs SAS3 backplanes.

It does have the practical difference of requiring SFF-8643 cables at the backplane end, instead of SFF-8087.

 

[TFAiSO]

I have started the build. Motherboard is in place. I’m already stuck. :/ I found out that there is no mentioning in the MB-manual or chassis manual which heat sink should be placed on cpu1 and cpu2. The heat sinks are different. When I hold them over a cpu-slot, they seem to both fit. Why would then SM recommend two different coolers?! The models are:
SNK-P0068PSC and the SNK-P0068PS.

Next thing: the MB-manual states I should connect all power slots with psu wires. 24pin OK. But, there are three 8pin connectors and only 2 8pin cables. :/

I have asked SM about the above but no answer yet (weekend).

 

[Ericloewe]

Next thing: the MB-manual states I should connect all power slots with psu wires. 24pin OK. But, there are three 8pin connectors and only 2 8pin cables. :/

It's best if you take some pictures, otherwise it's hard to say much that is useful.

I have started the build. Motherboard is in place. I’m already stuck. :/ I found out that there is no mentioning in the MB-manual or chassis manual which heat sink should be placed on cpu1 and cpu2. The heat sinks are different. When I hold them over a cpu-slot, they seem to both fit. Why would then SM recommend two different coolers?! The models are:
SNK-P0068PSC and the SNK-P0068PS.

You have two different CPU heatsinks? Is one active and the other passive? If so, the active one is probably meant for whatever CPU has fewer fans blowing across it.

 

[TFAiSO]

It's best if you take some pictures, otherwise it's hard to say much that is useful.


You have two different CPU heatsinks? Is one active and the other passive? If so, the active one is probably meant for whatever CPU has fewer fans blowing across it.

Both heat sinks are passive. I will show pics in my next post.

 

[TFAiSO]

Pictures of MB, heat sinks and chassis:
http://s350.photobucket.com/user/TFAiSO1/library/

 

[Evertb1]

Pictures of MB, heat sinks and chassis:
http://s350.photobucket.com/user/TFAiSO1/library/

I think it is allowed to attach pictures to your post. I just hate clicking away those stupid ads.

 

[TFAiSO]

I think it is allowed to attach pictures to your post. I just hate clicking away those stupid ads.

It’s free image hosting. Anyway, the photos are there and I can’t upload them on this forum as they are too big. (Normal size from my mobile camera)

 

[Ericloewe]

Then please compress them a bit and upload them.

 

[danb35]

In any event, if one heatsink is larger than the other, the larger one probably goes toward the back of the case.

 

[Stux]

Because the CPU at the back has to make do with already heated air. Also I think you’ll find the smaller one has a section of unused space to provide some cooler air to the back cpu.

 

[TFAiSO]

Wow. Thank you so much for your insight. I’ll put the larger one at the back tomorrow morning. Smallest in the front.
Can I use a single molex cable to slap an eight pin adapter on? SM said this chassi is 100% compatible with the MB but I don’t think so.
I’ll compress and save the photos. That post will be edited. No more ads from them!

 

[Martin Maisey]

B2 would be too dear. I’ll go for the personal at 5$ a month using a vm. I hope a windows in vm can view the files and back it up.

I suspect you may struggle if the files are stored natively in ZFS (as opposed to NTFS over a zvol virtual disk) - it doesn't back up network drives by design:

https://help.backblaze.com/hc/en-us...y-don-t-you-backup-network-drives-NAS-drives-

I imagine there might be ways to hack it, but it would pretty clearly be against Ts&Cs and I guess might get you banned.

B2 is what they want you to use for this purpose. You can understand why they don't want you backing up a NAS where you've spent (presumably) about $4k on the disks alone for $5 a month.

 

[TFAiSO]

I suspect you may struggle if the files are stored natively in ZFS (as opposed to NTFS over a zvol virtual disk) - it doesn't back up network drives by design:

https://help.backblaze.com/hc/en-us...y-don-t-you-backup-network-drives-NAS-drives-

I imagine there might be ways to hack it, but it would pretty clearly be against Ts&Cs and I guess might get you banned.

B2 is what they want you to use for this purpose. You can understand why they don't want you backing up a NAS where you've spent (presumably) about $4k on the disks alone for $5 a month.

In this case, I’ll do LTO. B2 would carry an annoying cost. Too bad all “freebies” are gone.

 

[Martin Maisey]

In this case, I’ll do LTO. B2 would carry an annoying cost. Too bad all “freebies” are gone.

Truly unlimited plans were completely uneconomic for those offering them. Which is probably the reason CrashPlan Home died. Personally I wish they'd kept it running and capped or capacity priced. It was a good service hobbled by a silly business model, and now I'm having to invest time in setting up something less convenient, rolling my own scripts and monitoring, and re-uploading everything to B2.

At least I'm only backing up ~400GB of truly irreplaceable data of my 5TB usable, which makes it cheap. Everything else is lossless BD, DVD, music rips etc., and it's all replicated to a second server in the attic. I figure if my house burns down I've got bigger problems than missing music and films.

Sorry for delivering more bad news, but you might want to read threads like this before rushing out and buying an LTO drive:

https://forums.freenas.org/index.php?threads/lto-and-freenas.48736/



Sent from my iPad using Tapatalk

 

[TFAiSO]

I'm back to the build. I have got the memory, cpu's and power cables finally attached. Air shroud slightly modded and should be fine.

I am not 100% of the vdev set-up.
Option1:
I now got 13 x 10TB drives. I intend to keep 2 in spare and do raidz2.
One idea is to do what is suggested. 2 vdevs: 6HDD's per vdev in raidz2. VdevA and VdevB will then be striped together to be part of the zpool.
Option2:
On the other hand, I could do a larger vdev like: 6HDD's useful + 2 parity drives = 8 raidz drives in vdevA. Then keep the other drives laying around until I get 3-4 more to repeat it and add another 8-drive raidz2 vdev.

I guess the speed will be faster with option1, but how slow can option2 be? The drives in themselves should be quite fast to handle multiple streams over 10GBe.

Secondly, there are two backplanes both handling sas3. I see in the manual for these backplanes, I can do cascading, multiple channels etc etc. No, I won't put an HBA in an external chassis. I will keep everything in this single chassis, but I am confused which set-up I should do. I guess I should simply connect 2 cables from the HBA to each of the backplanes, and not touch any jumper settings at all... Will that cut it? (using SFF-8643 to SFF-8643 wires)
Rear 12 port BPN-SAS3-826EL1-N4 backplane
Front 24 port BPN-SAS3-846EL1 backplane

To answer my own question in half, the M1215 has:

• Two internal mini-SAS HD connectors (SFF-8643)

As EricLoewe pointed out, the backplanes takes SFF-8643 as well.

 

[TFAiSO]

I have read all manuals and there is a ton of configuration options.
To simply connect the SAS-card to each backplane -thereby being able to utilize all 36 drive bays, does anyone know which cable/cables I need? I guess I don't need to do cascading. I also wonder if I have to change any jumpers... I think I need to call the support to straighten out this matter.

 

[danb35]

All you should need are two, SFF-8643-to-SFF-8643 cables. I don't believe (though I'm not certain of this) you'll need to adjust any jumpers.

 

[TFAiSO]

THANKS! :) I'll check I have two of those and then it's time to fire it up!

 

[TFAiSO]

THANKS! :) I'll check I have two of those and then it's time to fire it up!

Incredible. I thought I would have to spend more time shopping, waiting days to configure.
I had two Supermicro CBL-SAST-0531 laying around that are of just that type: SFF-8643 to SFF-8643.
As well, I had one Supermicro CBL-SAST-0593-01 that is a bit shorter. Now I can run the shorter to the closest backplane and the longer to the other.. yay!

 

[Chris Moore]

I guess the speed will be faster with option1, but how slow can option2 be? The drives in themselves should be quite fast to handle multiple streams over 10GBe

Each additional vdev should multiply your speed. I didn't take the time to look the specs up for your drive, but to put some numbers on it, if you have a 6 drive vdev at RAIDz2, and each drive is able to run at about 170 MB/s, the vdev should give you around ~290 MB/s of IO performance. If you have the same configuration for the vdev, but you have two of them, the pool IO performance should increase to ~580 MB/s. These are only approximations.
If you have a single vdev of 8 drives, it should give around 388 MB/s, and if you add a second vdev to the pool (with the same configuration) it should increase the IO performance to around 777 MB/s.
To get the full performance of the 10GB network link, you will need about 24 drives or you drives would need to be faster than the 170 MB/s that I estimated. These numbers are all estimates and approximated. I have two vdevs of 6 drives in RAIDz2 in my server and I see performance about on par with these estimates.