I'm looking for some advice

[Zofoor]

Hi all!
I am currently planning to move to Freenas for my storage. I'm not 100% new to Freenas (I know very well m0n0wall, and have also done a custom version of it. Also I have seen the born of the Freenas, when it was a form of m0n0wall, but I see that now it's a different project and that the "old freenas" has moved to another project)...
But, before making some bad mistakes, I have a few questions:

My build will be (all new, with the exception of 3 disks):
Supermicro X11SSH-F - becouse of the socket, the 8 SATA ports, IPMI. Also 64Gb max is a nice thing. Also it has two M.2 ports where I can attach boot devices.

2 x 16 Gb Kingston 2133 DDR4 ECC unbuffered - so I start with 32Gb, and can still upgrade with 2 additional modules to 64Gb. I've also found that this memory works well on this forum with that MB.

SSD 64GB Transcend M.2 2280 SATA3 - 64Gb becouse the price was almost the same. I can buy later a second unit for mirror, but as start I think it's enought.

Intel E3-1230V5 - it's more than what I need, but anyway as the consumption should be low anyway when not used, I prefer to have more power that I can use if/when needed. Also it has AES instructions set.

450W Corsair SF450 - 450W PSU should be enought for my needs. It comes with 4 SATA power connections and 4 molex. With 4x Molex2Sata adapters I should able to power 8 HD. Also it's a very efficent PSU. I expect that the CPU will be in idle most of the time...

Case Thermaltake Core x9 - It's a really huge case, has enought space for all drives and more (10 drives max). I hope that the size helps to have a low temperature without the need of additional fans. It comes with 2 fan installed, and can install up to 35 fans (?!?!?!) so if I need I can add some.

5x WD Red 6Tb (new) + 1x WD Red 4 Tb (already owned) + 1x WD Red 3 Tb (already owned) + 1x WD Green 2 Tb (already owned).

APC Smart UPS 1000VA SMT1000I (used, from Ebay, but the battery is new) - sinewave output (needed for the pfc-active PSU), up to 670W, high efficent. When in idle it should be enought for 2 hours, when working at 100% I expect that it lasts 14 minutes.


The server will be placed underground, so it should be pretty cold the whole year. It will be used for:
- stock of personal data (foto, videos, documents, etc)
- backup of a few PC
- storage for movies / movies
- subversion server, where store my code (mostly not used, as now I use an SVN server of the company where I work, so I use it mostly to store just old projects)
- would be nice to install a squid proxy server, to scan the data for virus, and to be able to access to the contents of Netflix of my country when I am outside (using a VPN)
- torrent downloader (+ web downloader if there is a easy plug-in to make that)
- backup to cloud scheduled on night
- samba server
- perhaps Active Directory Server, I'm evalutating...
- plex server, but I think it will never make transcode as currently I usually use players that access directly to the files via samba, and use android phone + chromecast when in holidays (and they usually don't need to transcode)
- owncloud

So, usually the server will just work as proxy server (max 2 PC + 2 phones),

So, I hope that I didn't made any wrong with the components and that my needs are clear :D

My concers are:
- I'd like to spin down drives when not working. Is that a good idea? Here the energy bill is high (euro 0.22/kwh), and I'd like to lower it as much as possible.
- Any other suggestions/recommendations to reduce the power usage?
- To accomplish that, I was thinking to split the drives in that way:
5x 6Tb + 1x 4Tb => RAIDZ2, ZFS Volume 1, one VDEV => main storage
1x 3Tb + 1x 2Tb => RAID1, ZFS Volume 2, one VDEV => dataset + torrent
In that way I think that most of the time the main storage will be idle (I expect just 4 access every day).
Also I will have enought space for my data on the main storage, and as soon as a drive fails I will buy a new 6Tb drive to replace the 4Tb one (and move it to the second ZFS Volume).
I don't care to lose the torrent ZFS volume, but what about the dataset? I've read that it is accessed and written often. If I lose it will have I troubles? A daily backup of the dataset to the main storage is enough?
- about my hardware, any suggestions? do you think that it's all ok?
- is possible to assign to the torrent plug-in one of the two ethernet in exclusive? I ask that becouse id' like to have it separated to another subnet, and separated from the other LAN (thanks of the draytek router). This becouse, as these tcp/udp ports would be the only opened on the router, I feel more secure to have them separated from my LAN. Or can I simply configure the plug-in to denial the access to other LAN IP? The only problem that I have about having a dedicated LAN is that I would need to bring a new cable, becouse I have only two ethernet there (and one will be needed for the IPMI)
- I'd like to encrypt my main data storage. I have read that there are some extra tasks to do when a drive fails in that case. Are there any other concerns about encrypting the volume?
- Something else that I forgot? :D

Thanks for your help, any point of view is wellcome, and sorry if I had written so much, but I think it was necessary to let you know what I have and what I need.

 

[Nick2253]

First off, hardware looks good at first glance.

I'd like to spin down drives when not working. Is that a good idea? Here the energy bill is high (euro 0.22/kwh), and I'd like to lower it as much as possible. Any other suggestions/recommendations to reduce the power usage? ... In that way I think that most of the time the main storage will be idle (I expect just 4 access every day).

Spinning down the drives is usually not recommended. Slowing and starting put the most wear on the drives, and drive spinup draws a not insignificant amount of power. And there's also the possibility that a drive will be slow to spin up, and will be dropped from the array.

If you use your server four times a day, with a fairly random pattern, I'd just leave the drives spinning. If you use the system like clockwork, then it might make more sense.

- To accomplish that, I was thinking to split the drives in that way:
5x 6Tb + 1x 4Tb => RAIDZ2, ZFS Volume 1, one VDEV => main storage
1x 3Tb + 1x 2Tb => RAID1, ZFS Volume 2, one VDEV => dataset + torrent

Make sure you use WDIDLE on the Green drive to prevent head parking issues.

Don't forget, you'll need a dataset where your jails are running. If you want to sleep the main storage, then you'll be putting your jails on your mirror. Don't forget to account for that space.

I don't care to lose the torrent ZFS volume, but what about the dataset? I've read that it is accessed and written often. If I lose it will have I troubles? A daily backup of the dataset to the main storage is enough?

A dataset is a generic unit. I'm assuming you mean the system dataset. Losing that would be bad (https://forums.freenas.org/index.ph...cial-is-the-system-dataset.27680/#post-180506).

Honestly, though, if you don't care if you lose the torrent volume, why are you doing a mirror?

- is possible to assign to the torrent plug-in one of the two ethernet in exclusive? I ask that becouse id' like to have it separated to another subnet, and separated from the other LAN (thanks of the draytek router). This becouse, as these tcp/udp ports would be the only opened on the router, I feel more secure to have them separated from my LAN. Or can I simply configure the plug-in to denial the access to other LAN IP? The only problem that I have about having a dedicated LAN is that I would need to bring a new cable, becouse I have only two ethernet there (and one will be needed for the IPMI)

You can indeed assign a particular NIC to a jail (http://doc.freenas.org/9.10/freenas_jails.html#adding-jails). Instead of running additional cables, you could just use VLANs to segregate the traffic (https://doc.freenas.org/9.3/freenas_network.html#vlans). I believe your router will support that (http://www.abptech.com/draytek-setup-handling-two-separated-lans.html).

- I'd like to encrypt my main data storage. I have read that there are some extra tasks to do when a drive fails in that case. Are there any other concerns about encrypting the volume?

The documentation deals with encrypting, and replacing an encrypted drive. I would argue that encryption is not worth the risk and hassle, and if you need to encrypt only some of your data, use something like VeraCrypt to create an encrypted virtual volume.

 

[Zofoor]

thanks for your detailed reply! :)

Spinning down the drives is usually not recommended. Slowing and starting put the most wear on the drives, and drive spinup draws a not insignificant amount of power. And there's also the possibility that a drive will be slow to spin up, and will be dropped from the array.

If you use your server four times a day, with a fairly random pattern, I'd just leave the drives spinning. If you use the system like clockwork, then it might make more sense.

Ok! So let's them spin!

Make sure you use WDIDLE on the Green drive to prevent head parking issues.

Oh, I was forgotting that... thanks!

A dataset is a generic unit. I'm assuming you mean the system dataset. Losing that would be bad (https://forums.freenas.org/index.ph...cial-is-the-system-dataset.27680/#post-180506).

Honestly, though, if you don't care if you lose the torrent volume, why are you doing a mirror?

Yes I was meaning the system dataset. Ok so better have it on the main zpool RAIDZ2, to be sure to avoid troubles. I was trying to have the system dataset in another zpool let the main zpool in idle.

With these changes of plain, perhaps it's better to just have:
zpool1: 5 x 6tb wd red + 1 x 4tb wd red + 1 x 3tb wd red
zpool2: 1 x 2Tb WD green 2,5" for the torrents

In that way I would have at beginning 10,74 Tb, that is enought at start. When the two older WD will fail and replaced with 6Tb units I would reach 21,48Tb.
I still think that a separated drive for torrents could be a good idea becouse if then I need to move a file from ZPool2 to zpool1 I would probably have all chunks written sequentially, and so increase the read performance.

You can indeed assign a particular NIC to a jail (http://doc.freenas.org/9.10/freenas_jails.html#adding-jails). Instead of running additional cables, you could just use VLANs to segregate the traffic (https://doc.freenas.org/9.3/freenas_network.html#vlans). I believe your router will support that (http://www.abptech.com/draytek-setup-handling-two-separated-lans.html).

Yes my router support it. Currently I use 3 VLANs (without tag, managed by the router), but if I configure the VLAN on freenas using the TAG then I would need to join two ports on VLAN0 and WLAN1 on the router and tag them. And I want to avoid that.
Well I think that I'll bring a new cable, it should be a work of a few hours.

The documentation deals with encrypting, and replacing an encrypted drive. I would argue that encryption is not worth the risk and hassle, and if you need to encrypt only some of your data, use something like VeraCrypt to create an encrypted virtual volume.

Currently I'm using VeraCrypt and it's very annoing. It's true that I need to encrypt only a small amount of data (about 1Tb), but it's also annoing becouse I need to run some services with that data, and I'd like to move that services from my desktop to the NAS (like SVN server). Also, sometimes I need to access to the same encrypted volume from two PC. That's very annoing, and using FreeNAS encryption would resolve all these troubles.

I think that I'll use it if I don't find better solutions.

 

[Nick2253]

zpool1: 5 x 6tb wd red + 1 x 4tb wd red + 1 x 3tb wd red
zpool2: 1 x 2Tb WD green 2,5" for the torrents

Putting that 3tb drive in your first pool actually reduces your available space. RAIDZ2 treats every drive as if they were the smallest. So with that config for pool1, it's like having 7x 3TB drives, equivalent to about 15TB. Without the 3TB drive, you "have" 6x 4TB drive, which is equivalent to about 16TB. If you plan on buying two 6TB drives anytime soon, then it might make sense to put the 7 disks in the RAIDZ2, because as you replace the smaller disks with larger disks, the pool will auto-expand.

FYI: there's no such thing as a zpool. They are just pools. zpool is the command to configure ZFS pools.

Yes my router support it. Currently I use 3 VLANs (without tag, managed by the router), but if I configure the VLAN on freenas using the TAG then I would need to join two ports on VLAN0 and WLAN1 on the router and tag them. And I want to avoid that.
Well I think that I'll bring a new cable, it should be a work of a few hours.

I'm not sure that makes sense. I'm not sure why you would need to join any ports on your router to make this work. That may be a peculiarity of your router, but it'd be one heck of a peculiarity.

In a typical setup, each port is configured with a "default" VLAN, which is the VLAN that is used to tag all untagged traffic, and the port can additionally be configured to pass tagged traffic for certain VLANs. (In Cisco speak, the "native" VLAN is the default.)

With that said, I'd assume that all you'd need to do is identify the port that goes to FreeNAS, and then give that port the ability to pass VLAN traffic for a certain VLAN, and then use that VLAN for the jail. By leaving its "default" or "native" VLAN, you shouldn't have to change anything else.

 

[nojohnny101]

Build looks good.

Make sure you search on the forums for planing the layout of your pool(s) and vdevs. This process should not be rushed and is better heavily researched. You should also look up cyberjock's noob presentation as it contains a detailed and clear explanation of all the different terms that are essential to understanding FreeNAS (pool, vdev, raidz1, raidz2, etc.).

Also, just from glance, your PSU might be slightly undersized and would not give you much room. I didn't crunch your numbers or anything, just a hunch from the hardware you listed (especially considering you are starting off with 8 drives). There is an excellent PSU sizing guide on the forums that should point you in the right direction or at the very least confirm that the PSU you selected will be sufficient.

As I always say, "buy cheap/inadequate, pay twice!"

 

[Zofoor]

Thanks for the tips. Finally all things has arrived, and I have set up the server and started configuring everything. I have also found an 1 year old used Seasonic Platinum 650W for 100 euro with still 6 years guarantee, so I have buyed it.

After a lot of tweaking with bios and Tunables and monitor the result I have reached 35W in idle (with 6 SATA drives) and 49/55W when writing to the NAS (using CIFS share). 105Mb/sec when writing, I'll make later a test from ramdisk to NAS to see where I can get.

Here the strange things:
1. My CPU supports c-states up to C8 (and I have found that also on the bios), but seems that FreeNAS see only C3. What can I do to fully enable C8 states? In the BIOS all seems ok (Supermicro SSH-F).

Code:

[root@freenas] ~# sysctl dev.cpu. | grep -i cx_supported
dev.cpu.7.cx_supported: C1/1/1 C2/2/151 C3/3/256
dev.cpu.6.cx_supported: C1/1/1 C2/2/151 C3/3/256
dev.cpu.5.cx_supported: C1/1/1 C2/2/151 C3/3/256
dev.cpu.4.cx_supported: C1/1/1 C2/2/151 C3/3/256
dev.cpu.3.cx_supported: C1/1/1 C2/2/151 C3/3/256
dev.cpu.2.cx_supported: C1/1/1 C2/2/151 C3/3/256
dev.cpu.1.cx_supported: C1/1/1 C2/2/151 C3/3/256
dev.cpu.0.cx_supported: C1/1/1 C2/2/151 C3/3/256

2. I had configured HDDs Advanced Power Management to disabled, and enabled powerd. The next day I have seen about +6500 spin-ups. Then I have changed Advaced Power Management to 192 and this solved the problem with spin-up. I think that there should be a message that tells the user about that. Perhaps I can suggest it somewhere to be done on the next release. Or perhaps it's a bug?

3. This is the most important problem I have reached.
I have tried to install the plex plug-in but it doesn't boot. Anyway, I didn't lost time to check out why and have created a Jail. Inside that Jail I have installed both Plex and MiniDLNA (following the tutorial in the how-to forum). All works great. Then, I have reached the tricky part.

I have an encrypted EncFS folder. The jail has access to that folder. Now I want to install EncFS to allow to mount that drive inside the jail, and allow Plex to use that encrypted folder. So I logged to the Jail, and execute the following commands:

Code:

cd /usr/ports/sysutils/fusefs-encfs
make install clean

...(cut)...

===>  Extracting for fusefs-libs-2.9.5
=> SHA256 Checksum OK for libfuse-libfuse-fuse_2_9_5_GH0.tar.gz.
===>  Patching for fusefs-libs-2.9.5
===>  Applying FreeBSD patches for fusefs-libs-2.9.5
1 out of 1 hunks failed--saving rejects to configure.ac.rej
=> Patch patch-configure.ac failed to apply cleanly.
*** Error code 1

Stop.
make[3]: stopped in /usr/ports/sysutils/fusefs-libs
*** Error code 1

Stop.
make[2]: stopped in /usr/ports/sysutils/fusefs-libs
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/sysutils/fusefs-encfs
*** Error code 1

Stop.
make: stopped in /usr/ports/sysutils/fusefs-encfs

So, how can I mount that encrypted folder inside the Jail?