Updated shutdown/action on zfs pool inactivity. Also hello :)

Status
Not open for further replies.

Ardacer

Cadet
Joined
Jun 30, 2015
Messages
8
So my NAS is a xeon 1230, 32Gb ecc, intel sever board, in a case with 11 vents (all controlled) and planned ZFS raidz2 pool of 10x3Tb + 1 Hotspare. For now just 4tb sitting inside. It's meant to be a backup machine (all very important stuff is backed up on one other server too), and storage for movies/music/games/general files. I won't be accessing it all so often, so when I WoL-it, I thinks its a good idea to have it power down itself after some time, as it will eat up 60-80W minimum (measured with Amp clamp) at idle (I think at least).

I found a nice code here : https://forums.freenas.org/index.php?threads/helper-script-for-automatic-shutdown-on-zfs-idle.12885/

But it didn't work. So I took some time and upgraded it, added some explanaitons, removed some stuff (I don't think I broke anything, it works fine, but if you have a Zpool names like "0" or some insane names, it might break the script. It could also be modified to do a lot of other stuff too.)

So basically what it does is it monitors your pools for active scrubs and writes/reads. If none of the above are found, it'll turn itself off. Ta-da. Works on 9.3. On my server. You never know..

One thing I'm confused about though, I see a lot of small writes on my non boot pool, are those logs? In that case I'm not sure this script makes a lot of sense if you don't put your logs elsewhere, as they will constantly write on your pool (you could set the script to look for reads only, but imagine the server shutting down in a middle of something backing up on it.. not good). Also the boot pool has hourly writes or so if you're including that, it makes no sense to have the interval longer then 1 hour or so. Anyway, the code is down there, and it can be modified, so .. hope it's worth something to someone.

Code:
#!/bin/bash

#set the time interval in (seconds methinks) which the pools are tested for activity
interval=240

#list your ZFS pools
pool1=freenas-boot
pool2=ZFS

noscrubs=0
inactive=0

while [ true ]; do

#optionaly replace pool2 with the pool you need to monitor, and copy and modify  for other pools if needed, add string2 test2 etc.
string1=$( zpool iostat $pool2 $interval 2 | tail +5 )
test1=$( echo "$string1" | egrep -c "0 *0 *0 *0" )

#do a basic if then if then if then ... for as many pools as you need, simmilar  to the if then pattern in the scrub section below.
if [ $test1 = "1" ]; then inactive=1
  else inactive=0
fi


#copy and modify the lines in this segment to check all pools for scrubs
sctest1=$( echo $(zpool status $pool1 ) | egrep -c "scrub in progress" )
sctest2=$( echo $(zpool status $pool2 ) | egrep -c "scrub in progress" )

#do a basic if then else if then else chain here for all the scrub tests. you're  smart, you'll figure it out.
if [ $sctest1 = "0" ];
  then
  if [ $sctest2 = "0" ];
  then noscrubs=1
  else noscrubs=0
  fi
  else noscrubs=0
fi

if [ $noscrubs = "1" ]; then if [ $inactive = "1" ]; then shutdown -p now
  fi
fi

done




Also, hi all, I'm Benedict from Croatia, first post.

I'm actually a physician, ophthalmologist (just started a week ago, finished studies a year ago!).
Also on doctoral studies in the national central nervous system research center.
I also have around 40 PC-s that passed through my hands (currently 10 active I believe, 2 linux servers and one freenas), not including all the smartphones tablets etc, and also I have a small private electronics lab where I build PCB-s and stuff... I also looove music and am an audiophile, and have HD650's, q701's, O2Odac, custom made speakers and amps, and looking forward to get HD800's.

Why am I a MD?

You tell me...


edit: for simplicity's sake, you could just make a lot of if then lines instead of one big if then chain. just remembered.
 

Ardacer

Cadet
Joined
Jun 30, 2015
Messages
8
Right, to update, the issues with constant writes on my zpool that I thought were logs, well I guess I have this :

https://forums.freenas.org/index.php?threads/continous-background-disk-writes.26593/

https://bugs.freenas.org/issues/4882

https://forums.freenas.org/index.php?threads/continuous-writes-on-zfs-pool.19975/

Tough luck. It appears the nginx and syslog-ng are always writing something on my zpool for some reason, nginx perhaps if I'm using the web interface?? Why would it? And syslog-ng probably uses the zpool for logs, will have to look into that.

Update: apparently nginx does so only when using the web interface, not too weird at all. Syslog still hogs my pool, have to see what's going on.

Someone with normal situation can use the script :)
 
Last edited:

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
Also, hi all, I'm Benedict from Croatia, first post.
Welcome, Benedict.

It's an interesting project, but most people would probably say not to bother when your idle power usage will be so low. Also, leaving FreeNAS always-on has benefits related to things like automated SMART tests and scrubs. Of course, it's your power bill and your choice.
ZFS raidz2 pool of 10x3Tb + 1 Hotspare
Why do RAIDZ2 with hot spare when you could do RAIDZ3?
 

Ardacer

Cadet
Joined
Jun 30, 2015
Messages
8
Thx for the welcome.
Mostly because of the 1gb of ram per 1tb, I'd go with raidz3, but then it would be 33 tb in the zpool plus about 2.5 gb for the system, so 29.5 gb of ram for 33 tb. If that's fine, then sure, yeah, z3 it is.

I calculated it would suck up around 15usd minimum per month, and with my light usage, I actually think the machine will be better off kept on demand. Its not much but I dont like giving money away for nothing.
 

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
Mostly because of the 1gb of ram per 1tb, I'd go with raidz3, but then it would be 33 tb in the zpool plus about 2.5 gb for the system, so 29.5 gb of ram for 33 tb. If that's fine, then sure, yeah, z3 it is.
The 1GB per 1TB rule is just a guide. From what I read in these forums, the more RAM you have the less strict the guide, and with your usage being so light, I highly doubt you will have any RAM issues.
15usd minimum per month
Sounds like your power rates are pretty high if that's what running an 80W device will cost.
 

Ardacer

Cadet
Joined
Jun 30, 2015
Messages
8
Yeah, well that's not a bad idea then, Z3 it is, thanks :)

Did the numbers again, it's more like 13 USD, but with all the drives added it'll go over 15 for sure. But yes, rates here are stupid high. :(
 

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
Did the numbers again, it's more like 13 USD, but with all the drives added it'll go over 15 for sure.
Ideally you'd weigh that against the potential replacement cost of parts that suffer early failure from repeated power cycles, if reliable data were available. Unfortunately I'm not aware of any source for such data, so we're left with lots of hand-wavy reasoning about why it's better to leave systems always-on, but nothing solid to back it up. Oh, how I long for someone to correct me here and point to some reliable data :)
 

Ardacer

Cadet
Joined
Jun 30, 2015
Messages
8
Agreed! It just seemed to me that with my really light use it'd be a good idea, and for normal/heavy use it's best to leave them on. I couldn't find anything smart too, probably cause the disks themselves are too variable :(
 

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
Yes, intuitively it seems like occasional (weekly?) use favors powering down.

The thing about FreeNAS, or any NAS, is that once you have it, you'll probably find more ways to use it, and the more uses you find, the more likely you are to want it to be available all the time.
 

Ardacer

Cadet
Joined
Jun 30, 2015
Messages
8
Couple of times a week at best, but yes, weekly. And regarding the other uses, you're probably right for normal people, but I of course am one special kind of stupid regarding these things.. Soon I'll get a small rasp pi 2 server for a web server (should be plenty plenty plenty for a personal web server, 4 cores @1ghz, 1 gb ram, consumes a wat? 2 wats maybe? For the same money that running the freenas device for 4 months would cost), one rasp pi 2 sever for streaming tv (can't be the same device unfortunately). Currently my ubuntu server for video surveillance and backup of importatnt stuff (different hdd than one that gets all the writes) and this freenas machine are active. One more linux server serves other specific purposes, on demand (electronics lab). So basically, my ubuntu server is already acitve 24/7, and my rasp. pies will be as well, so I don't see the need for the freenas machine to work all the time as well, so unless I find something BSD specific that I need running all the time on powerful hardware, it's just file storage / plex server for now. :)

All other machines (ubuntu server, 2 pfsense boxes (different locations) etc) consume next to no power.
 
Last edited:

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
Well, you probably have enough RAM in your FreeNAS to run a couple of Ubuntu server VMs using the VirtualBox jail...
 

Ardacer

Cadet
Joined
Jun 30, 2015
Messages
8
I probably would, but I already have these boxes so yeah, no real need there. I likely don't even need the nas server at all, but I still have it. :/
Besides, it's not smart to run a "WAN side" device on your "LAN side" NAS anyway, cause I use openvpn on my pfsense boxes... Perhaps someone will find some other use for this script, people have strange needs :)
 
Status
Not open for further replies.
Top