YAN (Yet Another Noob) Build

[Robert Trevellyan]

What I absolutely NEED is a way to connect, locally or remotely, and be able to transfer my movies/tv shows/music. That's why I figured BitVise would be perfect because it's made to sound like I could use Filezilla once the connection was made with BitVise.

I'm confused. Are you trying to get BitVise running on FreeNAS?

BitVise offers SSH server and client for Windows.
Putty is an SSH client for Windows.
FreeNAS has a built-in SSH server and client.
You can use FileZilla (and other programs) to connect as an SFTP client to any system that's running an SSH server, unless that system has deliberately prevented SFTP access.

 

[okgunguy]

To prove how much of a beginner I am, before watching a video on YouTube yesterday, I didn't even know that I could already connect to my NAS without Putty, or BitVise, or Filezilla. Once you create Windows shares, you can go to "my computer" and type in the address bar at the top "\\ip.of.your.NAS" and it will bring up the folders you already created under Storage. Then you can "Map Network Drive" and save Root login credentials and you can create/delete/move files and folders.
Once you turn on the Services of SSH and FTP you can connect with Filezilla and Putty. I still haven't figured out BitVise. But at this point it can wait. I was able to upload 4 movies onto the Windows share and stream 3 of them at the same time on 3 different laptops (after mapping the drives on them all).
It all seems to be making a little sense now. But without any of these pointers in the documentation, it was confusing the hell out of me. My next step is to go watch as many videos on youtube as I can find regarding FreeNAS and jails setups. Mainly as a wide view, idea gathering expedition. Then start narrowing down how I want my system set up. I'm watching a few threads on here with walk-throughs but I'm still missing something.
Once I find the set-up I want, I think I'm going to start over, AGAIN. Blow out all my shares, datasets, and volume. Just restart from initial install. I especially don't like the way 9.3.1 setup Wizard configured my datasets. Something isn't exactly right with my UPS either, it keeps loosing connectivity and then reconnecting. So I'll have to research that as well.
I'm mainly thinking out loud. And I'll do my research and come back with a procedure that I think makes sense to what I want and ask you guys what you think about it. Mainly sharing Movies, Music, TvShows with my TV, Windows laptops, iPhones and iPads. Thanks for the patience!

 

[Robert Trevellyan]

My next step is to go watch as many videos on youtube as I can find regarding FreeNAS and jails setups.

Try to stick to the official channel as much as possible. There's a lot of junk out there.

Once I find the set-up I want, I think I'm going to start over, AGAIN. Blow out all my shares, datasets, and volume. Just restart from initial install.

This is a good plan.

 

[okgunguy]

I know I am completely out of sequence at this point in my install. But after reviewing a few different procedures on installing Plex, Couchpotato, SickRage etc, I have come across a couple "Optional Settings" procedures. These are NOT from IXsystems, or FreeNAS so I wanted to see if this was a common change users make to their servers. And if there are any benefits. Or if it will just catch my motherboard on fire?
running - edit /usr/local/etc/nginx/nginx.conf - and replace worker_processes with the number of cores of your CPU. It not necessary, but recommended to set this to the number of threads your server has.
my E3-1241 is quad-core hyperthreaded to 8 cores. My nginx.conf file lists "1" under "worker_processes"
Is it advisable to change this? Does it provide any improvements? Or is this just another bad idea out there on-the-line?
I've searched through the forums and it appears to be a relatively common file people have changed, but I'm not finding anything specific about the "worker_processes"

 

[Robert Trevellyan]

Does it provide any improvements?

I don't see how it could improve anything except the performance of the GUI. Is that something you're finding unsatisfactory?

Oh, and it will set your server on fire...

;)

 

[okgunguy]

I knew it!! Haters just tryin' to fry my baby!!
My GUI performance is just fine. I just didn't know what it was for. Maybe after you install all the jails your GUI starts slowing down a little? Maybe that's why people change it?
I just thought if it turned on the warp drive function it might be worth checking out.

 

[Robert Trevellyan]

Maybe after you install all the jails your GUI starts slowing down a little?

I don't know why having more jails would slow down the GUI. Unless I'm mistaken, the only thing nginx is doing is delivering the GUI. I can even imagine that giving nginx more threads might have a tiny negative impact on the rest of the system, but that's pure speculation.

 

[okgunguy]

My replacement drive should be shipping back to me any day now. Talked with NewEgg on their chat (first time using that) and the lady is expediting my RMA because I said Please and Thank You.
In my down time, I've been doing more and more reading and came across a thread titled Relocate Jails to SSD helping HDD sleep? Basically someone wanted their HDDs to go to sleep and the jails and sys logs were preventing this from happening. So they migrated all their jails onto SSD's. So, I wanted to ask the question: Is this worth doing? And of course that just raised more questions for me.
- Some people were saying it is not the best thing for your drives to go to sleep in the first place. But is this really the case? Say overnight? Or all day while you're at work? I could see it if they went to sleep for an hour, had to wake up, went back to sleep for 2 hours, had to wake up. But for 8+ hours? If I care nothing about energy savings, only prolonging the life of my drives. Is this a valid argument for not doing it?
- Is it worth the added cost of 2 SSD's? Because if it's worth doing, it's worth parity.
- Is it worth losing 2 SATA ports?
- How much space do the jails and sys logs really take up?
I figured I would ask now. Figuring it would be way easier to set this up initially than to migrate settings and data later.

My down time hasn't been entirely wasted. I have started creating some scripts following this thread: Scripts to report SMART, ZPool and UPS status, HDD/CPU T°, HDD identification and backup the config . I "mkdir scripts" inside my root directory. /root/scripts. And "chmod +x your_script.sh". So Root would be owner and have execute rights. So far I have created: Send config backup to email, Display CPU and HDD temperatures, SMART report email, and ZPool report email. Both of the latter scripts I wrote to be Self-Adaptive since, why not? I haven't added them to Cron yet since I don't have a volume or ZPool to begin with. And all of my drives aren't here yet. But I did test the config backup email script and it works!!
It's been years since I wrote any scripts in Unix and it was pretty fun. Of course back when I used to do it the only way I was taught how was by using the Vi editor. Nano is sooo much nicer. And plus this was just copy/paste and adjusting the small format copy-paste problems. Thankfully Bidule0hm is brilliant with Unix and I can just copy off him in class.

 

[Bidule0hm]

Usually it's a bad idea and FreeNAS isn't optimized for drives sleeping (that's why you need to relocate the jails and .system dataset) let alone the fact the drives will wear faster. It's like pool encryption, unless you have a very good reason to use it and you know exactly what you're doing, it's a bad idea.

You're welcome ;)

 

[okgunguy]

My replacement drive arrived and is currently sitting in badblocks purgatory until it proves itself worthy. 26hrs down, probably only 97 more to go.
In the mean time, I have been trying to research what I need to do if/when Murphy rears his ugly mug. Total Armageddon, my Freenas won't boot. Crashes. Takes a dirt nap.
Here are the steps I understand.
#1) Have a backup plan in place to save the config file (ie. "freenas-v1.db") in a dedicated directory on the pool somewhere. I will probably save one on a local machine as well, often'ish.
#2) Reinstall Freenas. Possibly on new Flashdrives, if they were the problem to begin with.
#3) Auto import the Pool/Volume.
#4) Restore the Config file. <- This is fuzzy to me.
I read a post on this thread by cyberjock saying to restore the config file:
How to restore manually...

-Copy the "old" config file to /data, overwriting the /data/freenas-v1.db that already exists.
- # touch /data/needupdate
- # shutdown -r now

Poof... manually imported. :)

Or... hopefully the GUI will recognize the Pool/Volume and find the latest config I backed up and restore with a couple clicks.
#5) Restore the jails <- This is VERY fuzzy to me. Probably because I've never installed or ran a jail before.
I haven't found a definitive answer to this one. Aren't the jails stored on the Pool/Volume? Wouldn't they restore once I imported the Pool/Volume? Will this become clear to me once I begin installing and using jails?
I don't understand why this process isn't covered AT ALL in the manual. Or why no one has written a definitive guide to performing a complete restore.
Any pointers? Any glaring mistakes? Or could one of the local gurus take it upon himself to write up a "Phisher Price" restore thread? Or has someone already done so and I'm just not finding it in my searches?

 

[Robert Trevellyan]

I have been trying to research what I need to do if/when Murphy rears his ugly mug.

You're over-complicating the process:

1. Recovering the config from the pool is only necessary when you previously failed to download a backup to a local machine. If you get in the habit of doing this every time you make changes in the GUI, plus whenever paranoia takes hold, you have this part covered.
2. Correct.
3. Correct
4. Go to System | General in the GUI. Notice the buttons labelled "Save Config" and "Upload Config". That's all you need to backup and restore the configuration.
5. Jails will be restored when you restore the configuration, assuming your pool is intact.

 

[okgunguy]

Ok. Forgive the length of this post. I've had my first MAJOR problem.
For the past week or so, I've been ripping/converting my movie collection and uploading them to my NAS. I have a little over 100 of them done. I installed Plex in a jail. And have been enjoying it tremendously. Plex is very nice!!
I set up my schedule for Snapshots and Scrubs. My Snapshots are set up to run at Midnight and at Noon. Everyday and to keep for 2 months. But I must have had my volume scrubs set up wrong because when I run "zpool status" the only one that shows that a scrub has run on is the boot volume. I'm pretty sure I figured out what my problem on that was. I have the scrubs set up to run every 7 days. And the "interval" was set at 7 as well. I changed this to 6. So now they should run. I just haven't hit a 7th day yet. (unfortunately, as you'll see if you read on).
So, for a detailed account of what happened:
The evening before last, I had finished uploading more movies, and I uploaded a few songs as well to see how Plex would handle them. Well, Plex didn't pick the music up at all. And just to be sure they weren't corrupted or anything I played a song on one of my laptops directly from the NAS. It was Bruce Springsteen - Born in the USA. (why this may or may not be important will be seen a little later). The song played fine. So about half way through I stopped it and closed down the media player on my laptop. (I know, I should have listened to the whole song, being such a classic work of art)
I watched a couple episodes of one of the TV series I had uploaded. And everything played fine.
So, last night almost bedtime, I was going to watch another episode or two before heading to bed. However, Plex froze up. It wasn't showing any of it's metadata, like episode synopsis, or cover-art, and would not play anything. Episodes, or Movies. My Plex client is a Samsung DVD player. So I figured it was the problem. So I power cycled it. But still nothing. Well, it was close to bedtime, so I went to bed figuring more alert minds could figure it out in the morning.
So, this morning Plex still wasn't working. So I logged into the NAS with Putty and ran "zpool status -v". And got this:

Code:

[root@freenas] ~# zpool status -v
  pool: freenas-boot
state: ONLINE
  scan: scrub repaired 0 in 0h0m with 0 errors on Tue Sep 29 03:45:41 2015
config:

  NAME  STATE  READ WRITE CKSUM
  freenas-boot  ONLINE  0  0  0
  mirror-0  ONLINE  0  0  0
  gptid/2827406f-5756-11e5-9acf-0cc47a696a76  ONLINE  0  0  0
  da1p2  ONLINE  0  0  0

errors: No known data errors

  pool: r2d2
state: DEGRADED
status: One or more devices has experienced an error resulting in data
  corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
  entire pool from backup.
  see: http://illumos.org/msg/ZFS-8000-8A
  scan: none requested
config:

  NAME  STATE  READ WRITE CKSUM
  r2d2  DEGRADED  0  0  46
  raidz2-0  DEGRADED  0  0  92
  gptid/bf4c456e-6094-11e5-8eb7-0cc47a696a76  DEGRADED  0  0  7  too many errors
  gptid/bfc0f8fa-6094-11e5-8eb7-0cc47a696a76  DEGRADED  0  0  2  too many errors
  gptid/c0352be5-6094-11e5-8eb7-0cc47a696a76  DEGRADED  0  0  0  too many errors
  gptid/c0c03f6c-6094-11e5-8eb7-0cc47a696a76  DEGRADED  0  0  5  too many errors
  gptid/c13e984f-6094-11e5-8eb7-0cc47a696a76  DEGRADED  0  0  0  too many errors
  gptid/c1bae329-6094-11e5-8eb7-0cc47a696a76  DEGRADED  0  0  0  too many errors

errors: Permanent errors have been detected in the following files:

  /mnt/r2d2/media/Music/Bruce Springsteen - Born To Run/08 Jungleland.m4a
  /mnt/r2d2/media/Music/Bruce Springsteen - Greatest Hits/08 Born In The U.S.A..m4a
  /mnt/r2d2/pluginjails/plexmediaserver_1/usr/lib/libstdc++.so.6
  /mnt/r2d2/pluginjails/plexmediaserver_1/etc/mail/submit.cf
  /mnt/r2d2/pluginjails/plexmediaserver_1/usr/share/mk/bsd.cpu.mk
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Metadata/Movies/f/e647a24bers/com.plexapp.agents.themoviedb_6a51b5fd15d05471324e4a8c7ef20b323da65ea7
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/15/1
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/d4/d
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/9e/9
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/2c/2
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/45/4
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/f7/f
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/c2/c
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/3e/3
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/b6/b
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Metadata/TV Shows/a/d3d426emes/com.plexapp.agents.plexthememusic_0dbbf6d650fa14caea5145f23ed8dfb4d3ea6401
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/f0/f
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/f8/f
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/6e/6
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/e8/e
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/pkg/vuln.xml
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/d2/d
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/36/3
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/a2/a
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/b0/b
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/d3/d
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/pkg/local.sqlite
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/78/7
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/28/2
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/3e/3
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/38/3
  /mnt/r2d2/pluginjails/plexmediaserver_1/etc/mail.rc
  /mnt/r2d2/pluginjails/plexmediaserver_1/usr/local/etc/periodic/security/410.pkg-audit
  /mnt/r2d2/pluginjails/plexmediaserver_1/etc/syslog.conf
  /mnt/r2d2/pluginjails/plexmediaserver_1/var/db/plexdata/Plex Media Server/Cache/PhotoTranscoder/1a/1

One of my worst nightmares!! Not the worst, that would involve waking up looking like a Ken doll. But that is whole other story.
Easy enough fix I figured. Seeing as the "instructions" say to simply -
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
So in the GUI, since I don't know how in the command line (if anyone would care to give those instructions I would love to have them), From what I could understand, the GUI restore from snapshot is only runnable on the last snapshot created. So, I deleted the last 2 snapshots. Since I didn't know exactly when the problem started, I wanted the snapshot from midnight of the night I knew that everything was fine. And clicked the Restore Snapshot button.
After waiting for it to finish, I ran "zpool status" again. And... no change. I didn't reboot or anything, since I didn't know if I should.
Since a scrub has never ran on the volume. I researched(googled) and found how to run one manually from Putty. "zpool scrub vol". This ran for a little over an hour and then it was time to leave for work. Unfortunately, I had to close Putty down since I had to bring that laptop to the store with me. So I don't know if the scrub finished on it's own. Or even if a scrub is supposed to take that long anyway. Is there hung processes causing the problem? Why is "Born in the USA" listed in the error report?
Any advice out there?
I figured I'd try and reboot, or another manual scrub tonight. If it lets me, I'm going to try and back up all my movies to another external drive I have. Just in case I have to get forceful with it.

 

[rogerh]

I am not going to comment on your problem, because it sounds too difficult for me. But for future reference you can start a scrub from the GUI if you select the root of your pool in Storage/Volumes and press the little paint brush icon (with the 'scrub' tool tip) probably scrolled off the bottom of the screen.

 

[Nick2253]

Make sure you schedule your scrubs and SMART tests. Cyberjock has a great article on that somewhere in the forums.

Worst case, you've lost that data, and you may have to remake your plex configuration, since all the files (other than the two songs) are related to that. However, the bigger issue is why those files screwed up in the first place. Generally speaking, this is due to a bad hard drive (or two), and you'll be able to learn more about your drives if you check their SMART data. I don't remember the command off the top of my head, but if you search the forums, you'll find some examples, and then you can either try to interpret the SMART data yourself (good learning opportunity!) or post here, and we can help you understand it.

EDIT: you might also want to check your RAM for issues. Memory failures and hard drive failures are probably the most common causes of lost data. If you had any ECC warnings, they should have shown up in the IPMI logs.

 

[Robert Trevellyan]

Scrubs run asynchronously, so logging out of your SSH session won't interrupt a scrub.

Rolling back to an earlier snapshot may help you recover your Plex data, but first you need to figure out what's up with your server.

Your zpool status output seems odd to me. It appears to be saying most or all of your drives have had problems at the same time. Therefore, checking all your hardware makes sense, from reseating cables through to SMART extended tests, and memory tests as suggested by @Nick2253.

To check individual drives for any detected problems, run smartctl -a /dev/adaX, where X is replaced by the number of each drive in turn.

 

[cyberjock]

Where did you get your P20 firmware for your LSI card? From LSI or from Supermicro? Are all of those drives attached to the LSI card?

 

[okgunguy]

The P20 came from LSI. None of these drives are attached to the LSI controller.
I have a real problem here. And have no idea how to proceed.
When I came home this evening and tried getting into my NAS, it was totally unresponsive. Putty would ask for UN and PW and then just log into a blank screen. Firefox to FreeNAS IP would do nothing. Firefox to IPMI IP would log in. Remote Console/iKVM did not list an IP - the IP for the FreeNAS webpage (hence not being able to connect through Firefox to FreeNAS), but iKVM would spawn a SHELL. But nothing made any sense.
"smartctl -a /dev/ada0-5" came back with nothing (like the drives didn't exist), "cd mnt/r2d2/media...." came back with a list of all my movies (saying all my files were still there), "zpool status" came back with Degraded (like the above earlier post)
I was out of options. I had to reboot.
I was unable to Reboot. I tried through the iKVM (option 14). I tried on the IPMI main page (Power Down button). Nothing happened. But through the IPMI when I clicked "Reset" it reset the system.
iKVM came back with the IP and I was able to log in with Firefox. I was able to Putty in as well. When I run "smartctl -a /dev/ada0" (or any drive) everything comes back fine. No errors. It's reading all the SMART data. When I try to "cd /mnt/r2d2/media..." it's not there. No such directory. All of my snapshots are gone. Plex is not installed in Jails. When I "zpool status" the only pool listed is Boot.

I'm am fairly certain that my movies are still on my drives. There has to be a way to re-import them. I am thinking the "Detach Volume" in this last screenshot is probably the way forward. But I'm not sure. I don't want to lose my data if I don't absolutely have to. Has anyone done this before?
I am very positive I do NOT have a hardware issue. I am fairly positive something went horribly wrong with Plex, or hung processes, or some kind of software crash. I know it is not the drives. ada0 and ada1 are connected to SATA3 controller, drives ada2-ada5 are on SATA2 controller. My RAM went through close to 3 days of tests with ZERO errors. No drives are connected to LSI. I find it concerning that not only is my volume gone. But all of my snapshots are as well. I had over a weeks worth of snapshots saved and they are all gone.

If there is anything I can copy/paste, or screenshot that can assist in trying to understand. Please let me know. Thanks for any help that can be given.

 

[okgunguy]

I totally screwed up pasting the screenshots on that post. It looks like it duplicated them all. I guess I didn't need to put the {code} remarks in there. First time trying it. Sorry

 

[cyberjock]

All of the behavior (weird WebGUI, etc.) are the result of a zpool with a .system dataset that was unmounted without stopping the services.

I'm not sure what is actually going wrong, but its something that is potentially serious. At this point your zpool is unmounted, so you need to reboot the server and see what happens. There's nothing left you can do with the system before the reboot since the system and its services are now in an unstable state. So reboot it and see what happens.

 

[okgunguy]

Tried the reboot. Came back in the same state. Tried another reboot this time with the Verbose option and it still came back in the same state.
If I do the Detach Volume under Storage/Volumes, what exactly does that button do? Does it delete the volume? Or something else?
If my volume crashed and burned, that's one thing. But where did all of my snapshots go? The gui isn't listing them. What directory are they kept in? I want to see if they are actually gone.