SOLVED Custom build or IXSYSTEMS FreeNAS mini?

[Fire-Dragon-DoL]

It also gives you flexibility for other projects. For example, I temporarily added 16 drives to my system to do some performance testing. I took them back out later to cut the heat generated and power consumed, but I had the option.

Do keep in touch. There are lots of people here that would be happy to help with anything you run into.

My suggestion would be to use the previous release version of FreeNAS, which is 11.1-U6,3 because there are still some bug popping up in the new 11.2 version which introduced a whole new user interface. You can get that here:
https://download.freenas.org/11/latest/x64/FreeNAS-11.1-U6.3.iso

Once you get the parts, here are some guides that will help you with the setup:

Terminology and Abbreviations Primer
https://forums.freenas.org/index.php?threads/terminology-and-abbreviations-primer.28174/

Building, Burn-In, and Testing your FreeNAS system
https://forums.freenas.org/index.php?resources/building-burn-in-and-testing-your-freenas-system.38/

Github repository for FreeNAS scripts, including disk burnin
https://forums.freenas.org/index.ph...for-freenas-scripts-including-disk-burnin.28/

solnet-array-test (for drive / array speed) non destructive test
https://forums.freenas.org/index.php?resources/solnet-array-test.1/

Useful Commands
https://forums.freenas.org/index.php?threads/useful-commands.30314/#post-195192

Uncle Fester's Basic FreeNAS Configuration Guide
https://www.familybrown.org/dokuwiki/doku.php?id=fester:intro

Disk Price/Performance Analysis Buying Information
https://forums.freenas.org/index.ph...e-performance-analysis-buying-information.62/

Sigh it was too good to be true. The seller for the server doesn't deliver here :(

 

[Constantin]

1. Are the FreeNAS Mini up-to-date? I know they have Xeon processors in there, but are those the latest or am I better served by buying those myself and assembling the computer manually (I'm able to do that)
2. Is the FreeNAS Mini XL overkill for just a family?
3. How much space is available on a 3TB hard drive with ZFS?
4. Having 2 3TB hdds, I wasn't planning on setting up replication, I was going to handle them as two separate drives (different data). Also the first HDD will be mostly "readonly", since it's full and I want to keep the data in there
5. What are the disadvantages of a FreeNAS mini?

1. I'm the owner of a Min XL, which is the 8-bay version of the Mini. The guts are similar, just the case is taller to accommodate 8 hot swap drives. The motherboard was first released circa 2015 - AsRock C2750D4I - but it works and is very well supported. It supports 12 SATA drives, with a mix of SATA2 and SATA3 (use SSDs on the SATA 3 buses). In the Mini XL it came with 32GB of UDIMM ECC RAM. Upgrading to 64GB is possible but the sticks are very particular / hard to find at an inexpensive price point. I got lucky on eBay. I found the performance of the enclosure to be underwhelming re: cooling, as the drives were regularly cooked during scrubs. However, that was also likely due to the use of HGST 7K4000 series 3TB drives that are known to run hot.
2. Depends on your needs. If your aim is to create a Z2 pool, then an enclosure that can hold a minimum of 6 drives is a good match. The Mini has 4 hot-swap bays so your storage vs. parity ratio efficiency will suffer (# of drives available for data vs. # of drives needed for parity)
3. I'd look at it more from the # of drives available for storage vs. parity perspective.
   • For a Mini, a Z2 Pool with 4 drives, results in 2 drives worth of data and 2 drives for parity.
   • A Mini XL allows a 6-drive pool, allowing 4 drives to be used for data, while the number of drives needed for parity remain at 2.
4. 3TB hard drives can be very inexpensive right now. Don't discount used drives either - quality NAS drives from a good vendor can be had with multi-year warranties at low price points because heavy data users are getting rid of them and replacing them with 8 or 10TB spinners to increase capacity and lower electrical bills.
5. As mentioned in 2), the main disadvantage is how few bays remain available for data drives after dedicating one or two for parity disks.

While building your own system is a great idea on paper, there are advantages to ordering from iXsystems. For example, I had zero issues with getting my motherboard replaced when it succumbed to the AVR54 bug (C2xxx-based motherboards made before 2/2018 can have this issue). Other forum users didn't get the same kind of love from ASRock direct.

FWIW, I am running a Z3 pool with 8 drives - 5 for data, 3 for parity. The reason I chose 8 drives is that you are supposed to leave at least 20% space free or FreeNAS performance will plummet. So, to have 4-drives of effective space, you really need 5 drives. Moreover, I do my backups onto RAID 5 external arrays (Oyen Digital Mobius 5, about $200 new), which means I have 4-drives worth of space on them. So the best-performance capacity of the NAS matches that of the backup medium. That way, I can also use the same disks for both the NAS and the backup arrays.

I also strongly encourage you to keep one burned-in spare hard drive always on site, ready to be deployed for either your NAS or the backup. You'll sleep better and the RMA (if the thing is still covered) or replacement process can then play itself out with less risk.

I have yet to sell my original Mini XL case (I use something different now), ditto for the 32BG of OEM RAM that came with the thing.

 

[Chris Moore]

Sigh it was too good to be true. The seller for the server doesn't deliver here :(

you might want to check into this:
https://www.shipito.com/en/

 

[Fire-Dragon-DoL]

Although this is a decent system the way it is, you might find the 2.0 GHz processors a bit slow, and don't forget the existing hardware RAID card would need to be changed out for a SAS HBA.

So, if you start with this for $599:

https://www.ebay.com/itm/Supermicro...-24-x-HDD-Storage-Server-W-Rails/173688157339

Add the SAS HBA for $70:

https://www.ebay.com/itm/HP-H220-6G...0-IT-Mode-for-ZFS-FreeNAS-unRAID/162862201664

Then bump the processor speed up the 2.6 GHz (which works for me) with this kit:

https://www.ebay.com/itm/Matched-Pa...Hz-8-Core-Server-CPU-Socket-2011/153349417309

for an extra $180, that would bring the total to about $849 without counting shipping because I am not sure what it would be to your location.

This might be more than you want to spend for now, but you can always get the processor upgrade later.

Something to keep in mind. This system board supports (if I recall correctly) a terabyte of RAM and you can easily pickup faster processors also. They just need to be the Xeon E5 v2 variety.

Would this: https://www.ebay.ca/itm/SuperMicro-...h=item287284f835:g:miwAAOSwX4VcEQGV:rk:1:pf:0

Be the same as https://www.ebay.com/itm/Supermicro...-24-x-HDD-Storage-Server-W-Rails/173688157339

?

 

[Chris Moore]

Would this: https://www.ebay.ca/itm/SuperMicro-...h=item287284f835:g:miwAAOSwX4VcEQGV:rk:1:pf:0

Be the same as https://www.ebay.com/itm/Supermicro...-24-x-HDD-Storage-Server-W-Rails/173688157339

?

If yes, that saves me a lot on shipping (costs over 250 $USD from U.S.), and I'd save on the processors too (it's already E5 2667)

The details say that there is no RAM installed and it does not specify about the drive trays, but I can see in one of the photos that there are no drive trays in the chassis when photographed. This might cause you some difficulty, but those parts can certainly be purchased separately.
You might want to contact the seller and ask. Also ask if they can include memory, but you might be just as well served to purchase your own.
This is the kind of memory I would suggest:
https://www.ebay.com/itm/Micron-16G...Registered-REG-SERVER-MEMORY-RAM/362509985294
Although, you might run into the tax situation due to import, this is the low cost option for me.
I use Samsung memory in my system, like this:
https://www.ebay.com/itm/Samsung-8G...in-ECC-REG-Registered-Server-RAM/292359825842

As for the drive trays, if you need to get them, this is the kind for that chassis:
These include the blank that is needed when there is no drive installed, to manage airflow:
https://www.ebay.com/itm/SUPERMICRO...rstClass!39529!US!-1:rk:4:pf:1&frcectupt=true
These are the plain, empty, tray:
https://www.ebay.com/itm/Supermicro...rive-Tray-Caddy-Sled-with-screws/332599023771

 

[Fire-Dragon-DoL]

you might want to check into this:
https://www.shipito.com/en/

I checked it, but since it weights a lot (104 lb), the shipping is 210$USD even with that, the overall cost ends up being higher (I didn't run the math, but I'd still need to replace the processor too)

If those two matches, I'd go with the canadian one. As I said, I save on the processor (E5 2667)

The details say that there is no RAM installed and it does not specify about the drive trays, but I can see in one of the photos that there are no drive trays in the chassis when photographed. This might cause you some difficulty, but those parts can certainly be purchased separately.
You might want to contact the seller and ask. Also ask if they can include memory, but you might be just as well served to purchase your own.
This is the kind of memory I would suggest:
https://www.ebay.com/itm/Micron-16G...Registered-REG-SERVER-MEMORY-RAM/362509985294
Although, you might run into the tax situation due to import, this is the low cost option for me.
I use Samsung memory in my system, like this:
https://www.ebay.com/itm/Samsung-8G...in-ECC-REG-Registered-Server-RAM/292359825842

As for the drive trays, if you need to get them, this is the kind for that chassis:
These include the blank that is needed when there is no drive installed, to manage airflow:
https://www.ebay.com/itm/SUPERMICRO...rstClass!39529!US!-1:rk:4:pf:1&frcectupt=true
These are the plain, empty, tray:
https://www.ebay.com/itm/Supermicro...rive-Tray-Caddy-Sled-with-screws/332599023771

OK, that's a lot more stuff, but it might be more interesting since the RAM sticks are now 16GB (I can install plenty of RAM in this way).

I had the damn adblocker on so missed all the details!

Thanks, I'll let you know, if the seller answers I'll go for that.
Do I need 24 trays?

Edit: the 999$CAD has the trays included, it's written in red. OK Great, I'm going with it!

Edit 2: Looks like the E5 2667 is V1, not V2. I'm not sure about the difference

 

[Chris Moore]

Edit: the 999$CAD has the trays included, it's written in red. OK Great, I'm going with it!

Sorry. I don't know how I overlooked that. Any drive bay that doesn't have a drive in it, needs a filler to restrict the airflow so the bays that do have drives get enough air to keep the drives cool.

One of the other forum regulars, @Stux , used packing foam to make drive tray dummy blocks. See his post here:
https://forums.freenas.org/index.ph...ro-x10-sri-f-xeon-e5-1650v4.46262/post-315996

 

[Fire-Dragon-DoL]

Sorry. I don't know how I overlooked that. Any drive bay that doesn't have a drive in it, needs a filler to restrict the airflow so the bays that do have drives get enough air to keep the drives cool.

One of the other forum regulars, @Stux , used packing foam to make drive tray dummy blocks. See his post here:
https://forums.freenas.org/index.ph...ro-x10-sri-f-xeon-e5-1650v4.46262/post-315996

So my last concern is for the CPU. Is the RAM you linked still fine for the V1? And... how bad is the V1 compared to the V2? I noticed it consumes MUCH more (power-wise), which is concerning for a NAS.

Edit: Oh, it looks like it's not that much. My concern is only RAM at this point.

 

[Chris Moore]

Edit 2: Looks like the E5 2667 is V1, not V2. I'm not sure about the difference

Core count, clock speed and power / heat.
The v1 chip was 6 core, 2.9 GHz base, and T-case of 85°C
The v2 chip was 8 core, 3.3 GHz base, and T-case of 74°C
There were some other differences, mostly minor, but either of those should work, although the v2 is better for virtualization due to some additional instructions being available, but I don't remember the details.

 

[Chris Moore]

So my last concern is for the CPU. Is the RAM you linked still fine for the V1? And... how bad is the V1 compared to the V2? I noticed it consumes MUCH more (power-wise), which is concerning for a NAS.

The RAM I suggested should be fine for either CPU because I suggested PC3L-12800R, which is supported by both processors and the system board. If you went with the marginally faster 1866 MHz memory, you would NEED the v2 CPU .

They are both "rated" for 130 watts, but the v1 CPU does draw more current and create more heat. I don't know exactly what the difference is, and it would depend on demand. My NAS sits idle most of the time and the heavy CPU load is when I am running a scrub on the array or copying a lot of data from one pool to another. Plex almost never goes over 10% on CPU utilization.

 

[Fire-Dragon-DoL]

The RAM I suggested should be fine for either CPU because I suggested PC3L-12800R, which is supported by both processors and the system board. If you went with the marginally faster 1866 MHz memory, you would NEED the v2 CPU .

They are both "rated" for 130 watts, but the v1 CPU does draw more current and create more heat. I don't know exactly what the difference is, and it would depend on demand. My NAS sits idle most of the time and the heavy CPU load is when I am running a scrub on the array or copying a lot of data from one pool to another. Plex almost never goes over 10% on CPU utilization.

Ok perfect.

I got the same model from another reseller that delivered to Canada (yours didn't). I checked everything and it matches perfectly the one you linked ( https://www.ebay.ca/itm/Micron-2x-1...e=STRK:MEBIDX:IT&_trksid=p2060353.m2749.l2649 )

All good!

I'll need some cables but I'll figure out when it arrives. Thanks for following me so closely through this. It's definitely more costly than I anticipated, but it does exactly what I was looking for, without costing 2000$CAD and with MUCH MORE room for expansion (24 bays... I can probably rent a few to some friends, lol).

I'll let you know what happens when stuff arrives (30th of January). I'm a bit scared that the post office won't knock and expect me to grab 34kg manually (I don't have a car here), urgh.

 

[Fire-Dragon-DoL]

The RAM I suggested should be fine for either CPU because I suggested PC3L-12800R, which is supported by both processors and the system board. If you went with the marginally faster 1866 MHz memory, you would NEED the v2 CPU .

They are both "rated" for 130 watts, but the v1 CPU does draw more current and create more heat. I don't know exactly what the difference is, and it would depend on demand. My NAS sits idle most of the time and the heavy CPU load is when I am running a scrub on the array or copying a lot of data from one pool to another. Plex almost never goes over 10% on CPU utilization.

Just heads up, I was able to talk to the seller and for additional $100 I got the v2 instead of the v1, pretty happy with the entire thing!

I'm looking for some "cheap" 3tb nas drives (2x), they will be my scapegoats for testing stuff

 

[Fire-Dragon-DoL]

3TB hard drives can be very inexpensive right now. Don't discount used drives either - quality NAS drives from a good vendor can be had with multi-year warranties at low price points because heavy data users are getting rid of them and replacing them with 8 or 10TB spinners to increase capacity and lower electrical bills.

Hey Constantin, any chance you could be more precise regarding this information? I've been looking around for HDDs but the 3TB are more costly on ebay (used drives) than on amazon (brand new)!
I was looking at WD Red drives though, maybe you meant an enterprise drive?

 

[danb35]

The best bang-for-buck in hard drives right now is WD (EasyStore/Elements) external 8TB hard drives for under US$150, shucked.

 

[Fire-Dragon-DoL]

What are my options for the boot device in this machine?
Should I buy 2 USB stick (mirrored) and use those to install freenas on?

Does it have room for an SSD somewhere?

 

[danb35]

Does it have room for an SSD somewhere?

There's always room for a SSD; just Velcro it in if there isn't a bracket. But you should be able to get a mounting bracket; according to https://www.supermicro.com/products/chassis/4U/846/SC846E1-R900.cfm?parts=SHOW#jump it looks like the part number should be MCP-220-84603-0N. Here's one on eBay: https://www.ebay.ca/itm/NEW-Supermi...h=item4b01e898b0:g:nZkAAOxyLiJR0m2a:rk:1:pf:0

 

[Fire-Dragon-DoL]

There's always room for a SSD; just Velcro it in if there isn't a bracket. But you should be able to get a mounting bracket; according to https://www.supermicro.com/products/chassis/4U/846/SC846E1-R900.cfm?parts=SHOW#jump it looks like the part number should be MCP-220-84603-0N. Here's one on eBay: https://www.ebay.ca/itm/NEW-Supermi...h=item4b01e898b0:g:nZkAAOxyLiJR0m2a:rk:1:pf:0

Amazing, thanks I'm buying it! I have a few spare 128gb SSD in my closet (Samsung 840 Pro I believe), that I never use because are too small. Now it can serve a purpose!

Will it take one of the ports for the HDDs though?

 

[Constantin]

Hey Constantin, any chance you could be more precise regarding this information?

yup, don't discount used drives...Here is a vendor selling the same HGST 3TB NAS drives I used previously for $60 a piece and for $5 more, you can get a similar drive with a 3 year warranty. I've had good experience with GoHardrive, i.e. a DOA drive was taken back no question, with pre-paid shipping label from the vendor. Not even the OEMs give you that.

The 10TB drives that @danb35 mentioned offer a better $/TB ratio (I use them presently in my backup arrays), though each of those drives only a 2 year OEM warranty. They also need to be shucked carefully in case you have a failure as you'll likely have to send back a fully assembled drive - the enclosure has the serial # serial-scribed into it...

 

[danb35]

They also need to be shucked carefully in case you have a failure as you'll likely have to send back a fully assembled drive

There are reports of people sending back the shucked drives without issues, though I can't speak from personal experience.

I've had good experience with GoHardrive

...and I've had eight out of six "white label" drives that I bought from them fail. They did replace them promptly, but that failure rate is far too high for me to do that again.

 

[Chris Moore]

Amazing, thanks I'm buying it! I have a few spare 128gb SSD in my closet (Samsung 840 Pro I believe), that I never use because are too small. Now it can serve a purpose!

Will it take one of the ports for the HDDs though?

I think you can get it new for less...
https://www.newegg.com/Product/Product.aspx?Item=N82E16816101828