First TrueNAS Core build

[matthias8422]

I have been planning on building a general purpose home server for some time now and have been researching it for a while. I finally decided to go with TrueNAS Scale as the OS, both due to the community feelings towards the OS as well as its open source nature. (Scale over Core due to the ability o run VMs)

I plan to use this server to run TrueNAS, Plex, multiple different gaming servers, some Windows/linux VM's as well as other random applications i may find and want to use. Based upon this criteria i believe i need a more powerful system than is recommended for TrueNAS just due to all the extra applications that will be running.

I am still unsure if ECC RAM is required or just highly recommended due to the ZFS file system, but just in case i speced out some ECC ram, if its not required i can always find some compatible non-ECC RAM later.

Currently i have the following for my build

Motherboard: SUPERMICRO MBD-X12SCA-5F-O
CPU: Intel Xeon W-1370
RAM: 2x Kingston Server Premier 32GB DDR4 SDRAM Memory Module KSM26ED832HC (i will upgrade to 4x in the future)
Graphics: CPU has built in and i dont currently see a need for a card
Boot: i have an extra 2TB ssd that i will probably use (if VM images are stored on the boot drive, probably buy a 120GB ssd if not)
HDD: Seagate Exos X16 ST14000NM001G 14TB (probably 4-6 to start in raidz1 with 1 as a hot spare in case of failure since they are referb drives, eventually expanding to fill all 10+2 bays of the case)
Case: Sliger Cx4712
Cache drive: im not sure if this is needed, lots of conflicting information on this, any incite would be helpful.
Nic: on board, if i need extra speed (probably not) Intel X520-DA2


I will also eventually get a PCIE sata expansion card similar to this one once i expand past the motherboards sata ports (6)

all in all its about 1100 for everything minus the drives which i currently plan to buy refurbished about $140 per.

What do you guys think, first time building a server like this so any incite would be helpful.

 

[Arwen]

You have posted in the old FreeNAS (aka old Core) forums, not the SCALE forums. But, you are not alone in doing so.

If you love your data, ECC RAM is highly recommended.

Boot drives can not be shared;

I have to waste an entire drive just for booting?

With the advent of TrueNAS Scale, we seem to be getting a large number of people from the Linux community who are not quite grasping what TrueNAS is. TrueNAS is not a "Linux distro" and it is not designed to try to meet your arbitrarily spec'd...

www.truenas.com

Further, boot drives of 16GB to 64GB are fine. Though if the minimum size of a reliable SSD is 120GB, that is fine too.

Graphic card can be used for Plex transcoding, though their are gotchas that I don't know. Of course, if you are using native video formats, then you would not need a graphics card for that purpose.

RAID-Z1 is not recommended for disks larger than 2TB. This is due to the chance that another error would occur during the first disk's replacement, causing data loss.

Spare drives with RAID-Z1 are somewhat silly. Better to have it as on-line redundancy in a RAID-Z2. Unless you want to avoid the overhead of another parity drive.

ZFS RAID-Zx expansion is not yet available and might not be for years to come. Thus, if you start with a 4 to 6 RAID-Zx vDev, you can not just add one disk. You can either replace each disk with larger ones, one at a time. And when complete, your vDev, (and pool), will grow. Or you can add another vDev.

L2ARC Cache drives can be problematic. They take memory for their index, and it is generally recommended to max out memory first. Even then, a maximum of 5 to 1 ratio for size is suggested, though some people have gone higher.

In general, SATA expansion cards are less recommended than the LSI / Broadcom / Avago SAS chip HBA controllers. They usually come with 8 ports too, but can be found with 16, even 24 ports.

Multiply your problems with SATA Port Multipliers and cheap SATA controllers

In the last year or two, we've had a resurgence of users asking about SATA Port Multipliers and cheap SATA controllers. Please, do NOT use port multipliers, and use cheap SATA controllers only after extensive research. SATA controllers and SATA...

www.truenas.com

What's all the noise about HBAs, and why can't I use a RAID controller?

1) An HBA is a Host Bus Adapter. This is a controller that allows SAS and SATA devices to be attached to, and communicate directly with, a server. RAID controllers typically aggregate several disks into a Virtual Disk abstraction of some sort...

www.truenas.com

One last note about disk replacements. ZFS has some features, (that were unique when it came out), that help with disk replacements. For example, a disk that is failing but not yet failed, can be replaced in place. In essence, you install the replacement disk, (but NOT remove the failing disk), cause ZFS to replace the failing disk, which makes a temporary mirror of the failing disk. This allows redundancy from both the failing disk and the rest of the pool to re-build the failing disk. Thus, even on RAID-Z1, you can get better chance of successful replacement.

Also, if during a RAID-Z1 or 2 way Mirror disk replacement, you end up with a bad block(s) on your other disk(s), all is not lost. Meaning ZFS will tell you the impacted file, so the entire pool is not lost. You just restore the bad files from backups.

And, in general, it will be file data not metadata, like directory entries. Regular metadata has 2 copies, regardless of RAID-Z1 or Mirroring involved. Thus, loosing a block of metadata during disk replacement, is generally an automatic repair.

 

[matthias8422]

You have posted in the old FreeNAS (aka old Core) forums, not the SCALE forums. But, you are not alone in doing so.

Is there a way to move this into the TrueNAS Scale thread somehow, i cant find an option to do so perhaps a moderator can?

Boot drives can not be shared;

OK so it sounds like nothing except the OS goes on the boot drive so ill be getting a low cost SSD for that then. As well as using ECC RAM.

Graphic card can be used for Plex transcoding, though their are gotchas that I don't know. Of course, if you are using native video formats, then you would not need a graphics card for that purpose.

I speced out a CPU with intel Quick Sync Video so that the processor can do the transcoding, there will only be 2 people, possible 4 max, using plex at any one time so im not super concerned about offloading the transcoding to a GPU.

RAID-Z1 is not recommended for disks larger than 2TB. This is due to the chance that another error would occur during the first disk's replacement, causing data loss.

Seems as though raidz1 isnt an option due to the drive size i have, with 14TB drives is raidz2 enough or should go to raidz3, im not super concered about utilization and as I understand it, 6 disks in a raidz3 would only give me 50% utilization but if raidz2 would be fine id rather have the extra 14TBs of space.

ZFS RAID-Zx expansion is not yet available and might not be for years to come.

I did read into the zfs expansion stuff and was planning on just adding another pool to the vdev when i need to expand storage. (dont think ill need it for a couple years)

L2ARC Cache drives can be problematic. They take memory for their index, and it is generally recommended to max out memory first.

Ill max out memory first which hopefully i should need to do for a while and then look into cache drive at that point.

In general, SATA expansion cards are less recommended than the LSI / Broadcom / Avago SAS chip HBA controllers. They usually come with 8 ports too, but can be found with 16, even 24 ports.

Thanks for the info and the forum links, i hadnt put much thought into buying one of these yet since i shouldnt need it till i expand drive capacity but i probably would have bough just a generic sata expansion card assuming it would work.

 

[Arwen]

On the subject of RAID-Z2 verses RAID-Z3, depends on your desire for reliability. Ideally you would have another disk or 2, so that the data to parity ratio is higher than 50% when using RAID-Z3. But, if you have backups and can tolerate a window of data loss, (between backups), RAID-Z2 should be fine.


It is helpful to understand both ZFS terminology and how it works. Your wording is a bit off here;

I did read into the zfs expansion stuff and was planning on just adding another pool to the vdev when i need to expand storage. (dont think ill need it for a couple years)

I would word it this way;

I did read into the zfs expansion stuff and was planning on just adding another vDev to the pool when i need to expand storage. (dont think ill need it for a couple years)

Note that I reversed the "pool" and "vDev" words. A ZFS storage unit is a pool, which consists of one or more vDevs, (aka Virtual Devices). ZFS vDevs can be single disks, Mirror pair or RAID-Zx.

Here are some helpful Resources on the subject:

Terminology and Abbreviations Primer

We realize that new users have a lot to learn when they come to FreeNAS. There's a certain amount of confusion added to discussions when users pick random/approximate terms to describe things. I've spent a lot of time quietly trying to...

www.truenas.com

Introduction to ZFS

This is a short introduction to ZFS. It is really only intended to convey the bare minimum knowledge needed to start diving into ZFS and is in no way meant to cut Michael W. Lucas' and Allan Jude's book income. It is a bit of a spiritual...

www.truenas.com

Using the right terminology and understanding enough about ZFS allows you to make a reliable NAS. And when asking questions, get a useful response. Instead of a request for clarification.

Good luck.

 

[Arwen]

Is there a way to move this into the TrueNAS Scale thread somehow, i cant find an option to do so perhaps a moderator can?
...

I remembered the way to capture the attention of a Forum Moderator. Simply use the "Report" link at the bottom of each post, and describe what you want. I've already requested that this thread be moved to the TrueNAS SCALE forums.

 

[Ericloewe]

Not that it makes much of a difference to the recommendations, but are we talking Core or Scale here? It's late and I'm lost.

 

[Arwen]

@Ericloewe - SCALE. It is in the first paragraph of the first post, (but I have been their trying to read a forum thread backwards...);

I have been planning on building a general purpose home server for some time now and have been researching it for a while. I finally decided to go with TrueNAS Scale as the OS, both due to the community feelings towards the OS as well as its open source nature. (Scale over Core due to the ability o run VMs)