FreeNas Build with 3 HDD but one HDD on USB3.0 to SATA3 adapter

[farukhkhan21]

Hello Guys,

Trying to build a freenas setup. I have 3 seagate ironwolf HDD. But the board I am trying to build the freenas onto has only two SATA3 ports. And it also has two USB3.0 ports. Is it possible to plug in two of my seagate drives into the SATA3 ports and use one USB3.0 port to connect the remaining one HDD? Planning to go for a 3 HDD mirrored setup. So, same data mirrored between the 3 disks. If I go for this approach, will freenas have some issues doing the mirroring for any latency problems? Like is there a latency overhead if I go from the USB3.0 interface compared to the direct SATA3 interface? Will this setup work without any problem? Or do I need to take some variables into consideration before proceeding?

Also, another small question is that, can I use one NVMe SSD and have 3 separate partitions in the NVMe SSD for holding the freenas os in one partition, use another partition for L2ARC and the last partition for SLOG/ZIL? Is it a safe approach? Any suggestions regarding this?

Thank You.

 

[Ricko1]

What kind of hardware are you using?
Sata2 ports are fast enough for most hard drives. It has a limit of 300MB/s. For stability it's better then USB any day.

L2arc and slog/zil only have use in specific applications, which I don't you're going to be using with only 3 drives. Why do you think you need them?

 

[farukhkhan21]

I am planning to use a x86 based single board computer called ODROID H2+ from hardkernel. The board only has two SATA 3 ports.

The hardware has two 2.5gbps NICs so I am planning to aggregate the two connections to get a total of 5gbps. For 3 drives which are mirrored, it won't be able to provide 5gbps worth of read and write speeds. That's why I want a read and write cache which will allow me for faster reads and writes @5gbps. That's why was thinking about L2ARC and SLOG/ZIL

 

[jgreco]

I am planning to use a x86 based single board computer called ODROID H2+ from hardkernel. The board only has two SATA 3 ports.

The hardware has two 2.5gbps NICs so I am planning to aggregate the two connections to get a total of 5gbps. For 3 drives which are mirrored, it won't be able to provide 5gbps worth of read and write speeds. That's why I want a read and write cache which will allow me for faster reads and writes @5gbps. That's why was thinking about L2ARC and SLOG/ZIL

Well, I don't know what "thinking about SLOG/ZIL" has to do with "write cache". The ZIL is not a write cache. Please see this link.

Your write cache is system memory, and with only 32GB memory max, we also do not recommend L2ARC, which requires ARC space in order to keep track of what is stored in the L2ARC. The ARC system needs a lot of memory in order to be able to do a good job of identifying blocks that are suitable for eviction to L2ARC.

FreeNAS is not designed to let you share your boot device with anything else. While you might be able to manually hack it up to do an install, this will potentially turn into a problem when doing upgrades or a reinstall.

The characteristics that make for a good SLOG device (power loss protection and high endurance) are generally incompatible with L2ARC (cheap mass flash storage) as well, so that does not work in your favor either.

But I think your main problem is that the Realtek ethernet chipset used on that SBC is not FreeBSD compatible.

 

[farukhkhan21]

@jgreco how can I check the compatible NIC's list for FreeBSD? The other version of this SBC called ODROID H2 with 1GBe NICs worked perfectly fine with FreeNAS. But this version of the SBC added this new network interfaces.

The NAS is 4TB in total size. So, three 4TB Seagate Ironwolf Pro drives in mirroring configuration. That's what I am planning to do. But one part still confused is that how can I connect the third drive without having any latency or stability issues.

The system has total 16GB 2400MHz DDR4 non-ecc memory. Isn't this enough memory for L2ARC on 4TB worth of data?

If I cannot share the boot device, is it possible to use the NVMe for L2ARC only? Also, what will be the use case for implementing SLOG/ZIL? the article is really technical and I am really new to storage servers in general and also freenas.

 

[Constantin]

Given the relatively low cost of server-grade SuperMicro motherboards on ebay, I kindly suggest perusing the the recommended hardware lists and selecting something appropriate rather than trying to kludge together a solution that may work for a while before failing catastrophically. Here are a couple of lists to consider - these resources may feature a orange button in the top right corner for a download:

• Eric Loewe's Guide (has a Downloadable list)
• Chris Moore's recommendations for systems up to 32GB RAM
• Chris Moore's addendum for systems up to 512GB RAM
• Eric Loewe's SuperMicro X10 and X12 Motherboard FAQ

Sticking to known-good hardware may cost you a little up front but it will save you a lot of time and heartache to get it and keep it going. Server-grade boards usually feature better components, better chipsets, etc. ECC memory is also strongly suggested if you want to go the FreeNAS route, as why bother with faultless storage if the data being written to disk is potentially corrupt?

 

[jgreco]

@jgreco how can I check the compatible NIC's list for FreeBSD? The other version of this SBC called ODROID H2 with 1GBe NICs worked perfectly fine with FreeNAS. But this version of the SBC added this new network interfaces.

There are a lot of network interfaces that ""work"" with FreeBSD, but you don't really want that unless you like trying to debug performance issues, mysterious "the damn Realtek dropped offline" problem, etc. The list of network interfaces that work well with FreeBSD and FreeNAS basically boils down to:

Intel

where most Intel chipsets are well-supported with Intel-authored drivers. This is not to say that there are not other ethernet chipsets that work, but in most cases there are particular versions or particular issues or other stuff. Remember, most chipsets are made to work with Windows, and most manufacturers do not write a FreeBSD driver, and most manufacturers keep their design documents protected by an NDA.

We kinda figure it will be exasperating if your NAS doesn't work well. That usually means that your purchase choices need to be dictated by selecting from what is known to work well. You are trying to drive the process the other way, which is often a path to failure.

The NAS is 4TB in total size. So, three 4TB Seagate Ironwolf Pro drives in mirroring configuration. That's what I am planning to do. But one part still confused is that how can I connect the third drive without having any latency or stability issues.

Get a third SATA port, obviously.

The system has total 16GB 2400MHz DDR4 non-ecc memory. Isn't this enough memory for L2ARC on 4TB worth of data?

It's unlikely to have much benefit at only 16GB of RAM. The FreeNAS middleware by itself consumes a bunch of GB, so your entire ARC is probably only going to be able to consider about 8-10GB of content for L2ARC eligibility, which is basically nothing. We normally suggest L2ARC starting at 64GB or more of memory. There's probably some value at 32GB. This all has to do with the specific content, access patterns, and the ability of the algorithm to effectively pick content for eviction to L2ARC.

If I cannot share the boot device, is it possible to use the NVMe for L2ARC only? Also, what will be the use case for implementing SLOG/ZIL? the article is really technical and I am really new to storage servers in general and also freenas.

That article really isn't that technical. I write stuff so that these complex ideas are more accessible to newbies. That doesn't mean you won't need to re-read it a few times so that you can mesh the concepts together in your head though. The short form is "if you turn off sync writes, you are going to go as fast as possible. There is no SLOG that can make your NAS go faster than simply disabling sync writes and having no SLOG."

 

[farukhkhan21]

@jgreco Actually I am trying to build a home server lab with NAS, kubernetes cluster and some other servers. Most of them are based on ARM and I chose this ODROID H2+ board only because of FreeNAS because FreeNAS only works with x86. I also have another board with 2x Intel® I211AT PCIe Gigabit LAN. But this board only has 8GB DDR4 2400MHz memory and cannot be expanded. Also, it has only one SATA interface but with one PCIe gen 2.0 x4 and PCIe gen 2.0 x2 slots.

Not going the route suggested by @Constantin because I need low powered efficient hardware to run in this home environment to reduce the potential electricity cost, heat and other factors.

At this point what should I do with choosing the hardware @jgreco ?

 

[farukhkhan21]

One idea that comes to my mind is to upgrade the ODROID H2+ with 32GB RAM, then attach a PCIe gen 2.0 x4 to SATA3 adapter in the PCIe slot and connect 3 hard drives. But then I have no place left to connect the NVMe so cannot do any L2ARC which as suggested by you, I should skip anyway. But now the only problem is the realtek NICs. Is there no possible way to make these NICs work reliably? If I setup LAGG, isn't it going to be redundant also? Like if one NIC fails for some reason, the other one is still up and running?

Another thing I can do is to connect an Intel NIC to the PCIe gen 2.0 x4 slot. But then I have no place to connect the extra HDD rather than doing some USB3.0 to SATA3 adapter method. :(

 

[jgreco]

FreeNAS is generally optimized for larger systems. If you really want a nice, small, quiet, power-sipping NAS, you are probably better off with a Synology or QNAP desktop unit. These little devices are meant to run on a small embedded CPU.

You can get relatively low-power X64 stuff, but to do so, look at what iXsystems did for their FreeNAS-Mini.

 

[farukhkhan21]

Wanted to build something by myself rather than buying something pre-built. Also, want to play around with FreeNAS to understand ZFS, NAS and other functionalities in general. FreeNAS-Mini are really expensive compared to the 300 dollar budget that I have for this NAS's hardware build.

 

[jgreco]

And I'd like me some $50 2TB SSD's, but it isn't to be, at least not this year. Mmmm.

 

[Evertb1]

Also, want to play around with FreeNAS to understand ZFS, NAS and other functionalities in general.

Playing around with it is possible with almost all hardware that is x86 based. Just take care that you at least have the bare minimum for memory etc. But don't trust it with your precious data. Just consider it as a playing and learning ground not as "production". Then, when you know what you are doing and want to get serious about FreeNAS storage, you have learned enough to look at the needed hardware with a fresh look.

 

[farukhkhan21]

BTW, how much space in OS installation drive is required? is 16GB enough?

 

[Constantin]

I opted for 64GB units because I don't like to run out of space as the OS/Application revisions pile on. However, 16GB is perfectly OK as long as you follow the warnings of FreeNAS as that space starts to get full. 16GB is also what iXsystems shipped with my Mini XL as a SATADOM.

 

[farukhkhan21]

What exactly would take up the OS drive space? shouldn't the OS system only take up the space? What if I get rid of all previous revisions?

 

[Constantin]

Every time FreeNAS is updated, a complete system folder / OS / application / whatever is saved. So you can go back to a previous revision anytime, assuming you haven't altered your pool to make it incompatible with an older revision (TrueNAS vs. FreeNAS, for example)

As long as you stay on top of how full that drive is, you have nothing to worry about. I was simply too lazy to keep up with deleting old revisions of FreeNAS as I updated and so on occasion, I would get a warning that the 16GB drive was 80% full and that junk should get deleted. No drama, easy enough, just had to get off my butt and do it.

 

[farukhkhan21]

@Constantin ah I see. alright then. going for 16GB. Thanks for the suggestion.

 

[Evertb1]

Starting out with 16 GB, it will take some time (and more then a couple of updates of the os) to get at 80% full. Nothing to be worried about.

 

[farukhkhan21]

Got it @Evertb1 Thanks for the suggestions guys.