Out of space, planning the next build

[xvzacrw]

I've been a long time lurker, but this is my first post... Even though I've done a lot of reading and learning, I still consider myself a newb, so please help!

I'm running FreeNAS 11 and I just hit 80% capacity so I need/want to build a new NAS. I currently have 5 x 3TB HDDs for a usable total of ~10TB. Including a small SSD for booting, my tiny case (LIAN LI PC-Q25B) is packed full. I've got 16GB RAM and an AMD-A4-6300 processor. Nothing fancy, but it has worked like a champ for the last two years.

The new NAS, like the current, will just be for storage. It might have a SMB share, but I have separate servers for other things. There will be at most 10 users at a time.

I have 8 x 8TB WD Reds (technically 1 is a white label) that I picked up during the last sale. My plan is to use those in a 24 or 36 bay chassis and in the future when I need to expand, I can add 8 more drives at a time as a new vdev. (Still learning, is this correct?)

I've been trying to find a good price for a Supermicro chassis, but they seem to be rare these days. Assuming I do get my hands on one, I would still need other components.

This is just one example, but I see a posting for a Supermicro 24 Bay for $350 that comes with
SUPERMICRO MBD-X8SIE-O LGA 1156 Intel 3420
Xenon 2.8ghz Processor (No more info given)
12GB DDR3 ECC RAM (Would need more)
Adaptec 5808 raid w/ 512mb cache
12 x 300GB 15k SAS Drives

With this I would need a new backplane? And probably Motherboard and therefore processor and RAM, since this one only supports up to 32GB RAM? So I'm really only getting the chassis...



Any recommendations?

Much appreciated!

 

[nightshade00013]

The Supermicro you are looking at is a turd in every sense of the word. The only thing in it that will be worthwhile is the chassis as you already stated.

And that raid card should be thrown into a shredder if you are thinking about FreeNAS with it.

If you are going to buy a new board for the capabilities needed you may as well go for a full chassis that is already good to go.

Something in the E3 v2 or E5 v2 range or newer is what I would look at https://www.ebay.com/itm/Supermicro...2-6ghz-8-Core-128gb-24-Bay-JBOD-/232604715524

The backplane is SAS2, so no need to replace. It's powerful enough to do anything you are currently doing and then some. Probably even get rid of some of your other servers and consolidate saving you more money. Only thing I would do is add a pair of battery backups along with the drives I plan to use.

 

[danb35]

in the future when I need to expand, I can add 8 more drives at a time as a new vdev. (Still learning, is this correct?)

Yes, this is correct. Actually, you can add any number of drives as a new vdev, but a best practice is for them to be matched.

The system @nightshade00013 linked to should be ready to go, with a possible firmware update for the HBA. I'd prefer the RAM be configured with fewer, and higher-capacity, DIMMs to allow for easier expansion, but what's in there should be fine.

 

[Chris Moore]

I've been a long time lurker, but this is my first post... Even though I've done a lot of reading and learning, I still consider myself a newb, so please help!

I'm running FreeNAS 11 and I just hit 80% capacity so I need/want to build a new NAS. I currently have 5 x 3TB HDDs for a usable total of ~10TB. Including a small SSD for booting, my tiny case (LIAN LI PC-Q25B) is packed full. I've got 16GB RAM and an AMD-A4-6300 processor. Nothing fancy, but it has worked like a champ for the last two years.

The new NAS, like the current, will just be for storage. It might have a SMB share, but I have separate servers for other things. There will be at most 10 users at a time.

I have 8 x 8TB WD Reds (technically 1 is a white label) that I picked up during the last sale. My plan is to use those in a 24 or 36 bay chassis and in the future when I need to expand, I can add 8 more drives at a time as a new vdev. (Still learning, is this correct?)

I've been trying to find a good price for a Supermicro chassis, but they seem to be rare these days. Assuming I do get my hands on one, I would still need other components.

This is just one example, but I see a posting for a Supermicro 24 Bay for $350 that comes with
SUPERMICRO MBD-X8SIE-O LGA 1156 Intel 3420
Xenon 2.8ghz Processor (No more info given)
12GB DDR3 ECC RAM (Would need more)
Adaptec 5808 raid w/ 512mb cache
12 x 300GB 15k SAS Drives

With this I would need a new backplane? And probably Motherboard and therefore processor and RAM, since this one only supports up to 32GB RAM? So I'm really only getting the chassis...



Any recommendations?

Much appreciated!

I guess you're looking at eBay to get a bargain, but that system is a generation older than anything I would consider.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[xvzacrw]

Thanks everyone for the advice! That looks like a great find.

It's powerful enough to do anything you are currently doing and then some. Probably even get rid of some of your other servers and consolidate saving you more money.

I'm intrigued, but if I go that route, am I running FreeNAS in a VM or am I using jails/vms within FreeNAS? I know I can do either, but I have read that 1) keep the NAS hardware separate from other things if possible and 2) don't trust your data to FreeNAS in a VM. ..Comments?

Let's say I consolidate, migrating plex, sonarr, etc.. to FreeNAS. Would you still recommend the same server or would that change your recommendation?

Thanks!

Thoughts on this one?

 

[nightshade00013]

You could do any number of things. The key for keeping the data safe is FreeNAS has to have the whole drive so you have to be able to pass them through. A lot of people here do just that. But personally I just use Jails for most everything I need to have done. It's quick, simple and easily snapshotted so you can restore if you mess something up. There are a few things that will not run in a Jail but I have found that is a very small percentage of what I want to do and really not that important overall.

Really it's personal preference on all that though.

As far as Plex and such I run it on a dual E5640 system with 48GB of RAM. The only thing you would have to be careful with is if you plan to transcode 4K a lot and honestly I would just keep a 4K and 1080P copy of stuff since while there is some 4K stuff out there unless EVERY device you have is 4K it's not worth it to have to constantly drop the quality. I would rather burn a few GB of space than transcode all the time wasting power.


As far as the two systems the CPU's are nearly identical passmark wise with a small edge to the 2650. The 2650 is a 95w vs a 130w for the 2680 so the passmark/watt is better on the 2650. You get more ram with the 2680 but it's slower speed and you will have to come up with your own SAS HBA. I would stick with the one I linked and then change the RAM later. The prices on the server RAM will only drop and for now I doubt you will need more and you will do it on slightly less power.

https://www.cpubenchmark.net/cpu.php?cpu=Intel+Xeon+E5-2650+v2+@+2.60GHz&id=2042
https://www.cpubenchmark.net/cpu.php?cpu=Intel+Xeon+E5-2680+@+2.70GHz&id=1221

 

[Chris Moore]

Thanks everyone for the advice! That looks like a great find.



I'm intrigued, but if I go that route, am I running FreeNAS in a VM or am I using jails/vms within FreeNAS? I know I can do either, but I have read that 1) keep the NAS hardware separate from other things if possible and 2) don't trust your data to FreeNAS in a VM. ..Comments?

Let's say I consolidate, migrating plex, sonarr, etc.. to FreeNAS. Would you still recommend the same server or would that change your recommendation?

Thanks!

Thoughts on this one?

This one is a bit more money, but it gives you more RAM, so you have more room to run VMs. Just keep in mind that it does not include the SAS controller, not a big thing since you can buy one for around $50. I would go for it if I were looking to buy a server, but I have two extra SAS controllers sittin on my desk...

 

[Chris Moore]

As far as Plex and such I run it on a dual E5640 system with 48GB of RAM. The only thing you would have to be careful with is if you plan to transcode 4K a lot and honestly I would just keep a 4K and 1080P copy of stuff since while there is some 4K stuff out there unless EVERY device you have is 4K it's not worth it to have to constantly drop the quality. I would rather burn a few GB of space than transcode all the time wasting power.

I agree, I have 720p and 1080p for most of my devices and only one 4k TV, so I just encode for 1080p and it looks good enough, for me. Even the things I encoded at 720p back when that was my target still look great on the 4k TV. I don't want to burn the storage to save everything at 4k and then have to transcode all the time to bring it down to 1080 or 720.

 

[Chris Moore]

My plan is to use those in a 24 or 36 bay chassis and in the future when I need to expand

I would stay away from the 36 bay chassis. It is more space efficient but it is harder to cool because they have drives in the front and back of the chassis so you need more pressure from the fans to cram enough air through there to cool the drives. That makes for louder fans or hotter drives and neither of those are great options.
If you find (later) that you need more drive bays, a second 24 bay expansion chassis is not hard to connect externally by means of a second SAS controller or even some cabling adapters and connect it to the existing SAS controller. Once you buy a server like this, you can expand the capacity a massive amount. I have an entire 42U rack at work that is effectively all one server because all the expansion chassis in the rack are all cabled back to one server that runs 128 hard drives.

 

[xvzacrw]

I would stay away from the 36 bay chassis. It is more space efficient but it is harder to cool because they have drives in the front and back of the chassis so you need more pressure from the fans to cram enough air through there to cool the drives. That makes for louder fans or hotter drives and neither of those are great options.
If you find (later) that you need more drive bays, a second 24 bay expansion chassis is not hard to connect externally by means of a second SAS controller or even some cabling adapters and connect it to the existing SAS controller. Once you buy a server like this, you can expand the capacity a massive amount. I have an entire 42U rack at work that is effectively all one server because all the expansion chassis in the rack are all cabled back to one server that runs 128 hard drives.

Thanks, good to know.

That server arrives tomorrow. I guess I need to find some ssds and a battery backup.

 

[xvzacrw]

with a possible firmware update for the HBA.

Was hoping you could shed some light on this.

I see this firmware for the card here: Installer_P20_for_FreeBSD from https://www.broadcom.com/products/storage/host-bus-adapters/sas-9210-8i#downloads

Should I follow the instructions here: I'm still confused. LSI 9220-8i. IT Flashing, DOS & UEFI Errors

Thanks!

 

[Chris Moore]

Was hoping you could shed some light on this.

I see this firmware for the card here: Installer_P20_for_FreeBSD from https://www.broadcom.com/products/storage/host-bus-adapters/sas-9210-8i#downloads

Should I follow the instructions here: I'm still confused. LSI 9220-8i. IT Flashing, DOS & UEFI Errors

Thanks!

Did you look at this guide:

Detailed newcomers' guide to crossflashing LSI 9211 HBA and variants
https://forums.freenas.org/index.ph...o-crossflashing-lsi-9211-hba-and-variants.54/

 

[xvzacrw]

Did you look at this guide:

Detailed newcomers' guide to crossflashing LSI 9211 HBA and variants
https://forums.freenas.org/index.ph...o-crossflashing-lsi-9211-hba-and-variants.54/

Thanks, that's what I need!

 

[xvzacrw]

I guess I just need to update since it's 11 versions behind. Looks like it's already in IT mode.

 

[Chris Moore]

I guess I just need to update since it's 11 versions behind. Looks like it's already in IT mode.

Yes. If I recall correctly, the latest / best version for you to get is the 20.00.07 revision.

Or, it might be 20.00.00.07 ... something with a lot of zeroes, I think they get paid extra for the zeroes.

 

[xvzacrw]

they get paid extra for the zeroes

Haha, that must be it.

Ok, so I finally got it to work.

It took a lot of trial and error because I was making it way more complicated than it needed to be and there are obvious gaps in my knowledge.

I was trying to run both the DOS and UEFI commands from FreeDOS, and neither worked.

The first hurdle was realizing I had to switch my boot mode to UEFI.

I feel like an intelligent person would have known this, but in case anyone like me is reading this looking for help I thought I would leave an update. I am certainly grateful for all of the help I've been given in this thread and everyone that has documented their struggles elsewhere--it made my success possible.

So once I had UEFI selected, I just needed to go to Broadcom's website and find the firmware downloads page for the card I have.
Then download the installer for UEFI and the firmware package, put the 2 files on a usb stick that is formatted as FAT, and run the commands in the manual that came with the downloads.

It was simpler for me because the card was already in IT mode and I just had to do an update, but some of what I learned would also apply to other scenarios.

It sounds like it isn't necessary to update the BIOS since I won't be booting from the disks. I'll use separate SSDs. What about the motherboard BIOS, is that something I should consider updating?

Thanks!

 

[Chris Moore]

I would get the latest BIOS you can find for the system board because they might have important updates.
As for the boot firmware on the SAS controller, you can erase that if you are not using it (I have on mine) and the system will boot faster because it isn't waiting for the SAS controller.

Just be very careful with updates to the system board because if it goes wrong, it will brick the system.

Sent from my SAMSUNG-SGH-I537 using Tapatalk