YAN (Yet Another Noob) Build

[okgunguy]

While researching how to see snapshots, I pulled up the iKVM window again and saw the following. This is just system messages, I didn't type any commands to get this output. It looks like it's trying to create a new snapshot but my volume 'r2d2' is not there. Are these ZFS commands anything

that could help?

 

[Robert Trevellyan]

I find it concerning that not only is my volume gone. But all of my snapshots are as well.

It won't be much comfort, but snapshots are part of the volume, so that's only one issue, not two. You may be thinking of them as standalone backups, which they most definitely are not.

If I do the Detach Volume under Storage/Volumes, what exactly does that button do?

In plain ZFS it's purpose is to put the volume into the correct state for importing on a different system. In FreeNAS you also have the option to mark the drives as new, but it won't do anything until you confirm. I'm pretty sure it won't help your situation, because you can't detach a volume that isn't mounted.

If I understand what @cyberjock is saying about the .system dataset, you might be able to make progress by doing a fresh install of FreeNAS on a different boot device, making sure that the .system dataset is either on the boot device or on a new pool, and then attempting to import your existing pool. You might only be able to recover your data, but that would be better than having to start over.

Probably best to wait for a 2nd opinion...

 

[Robert Trevellyan]

It looks like it's trying to create a new snapshot but my volume 'r2d2' is not there.

Right, you still have the snapshot task enabled. Just turn that off for now.

 

[Nick2253]

If I understand what @cyberjock is saying about the .system dataset, you might be able to make progress by doing a fresh install of FreeNAS on a different boot device, making sure that the .system dataset is either on the boot device or on a new pool, and then attempting to import your existing pool. You might only be able to recover your data, but that would be better than having to start over.

Probably best to wait for a 2nd opinion...

This would be my suggestion as well.

Make sure you back up your configuration. However, it would be ideal if you used a completely different device, just to rule out any problems with your current boot device. Theoretically, when you boot the new FreeNAS install, it should automatically import your pool. If it doesn't, you can manually import it. And that will tell us if the pool is the problem.

Also, don't be too quick to rule out hardware failure. Just because something tested as working yesterday does not mean it will also be working today. It's important to go through proper diagnostic steps to rule that out.

 

[Robert Trevellyan]

Make sure you back up your configuration.

In this particular case I'm not sure there's much value in that. I mean, by all means save it in case it might somehow be useful on the original install, but don't plan to import it to the clean install. My reasoning is that if things are as trashed as they seem, reloading the saved configuration would probably be a mistake.

don't be too quick to rule out hardware failure

+1

 

[Nick2253]

In this particular case I'm not sure there's much value in that. I mean, by all means save it in case it might somehow be useful on the original install, but don't plan to import it to the clean install. My reasoning is that if things are as trashed as they seem, reloading the saved configuration would probably be a mistake.

I agree. The reason I recommended backup is that there's more value in having an extra copy of something useless than losing the only copy of something valuable. In this particular case, my thinking was that if it is not due to a misconfiguration issue or corrupt boot device, etc., there's value in having the config. Obviously, worst case, you could simply remake it. If it turns out that a new install of FreeNAS fixes this problem, I too would strongly recommend a manual redo rather than an import.

 

[okgunguy]

Ok. I did a complete new FreeNAS install. After it was done it started the Wizard. Everything looked promising. It asked if I wanted to import my volume 'r2d2' and the wizard finished with no problems.
After it was all done. I checked in Storage but it said 'No Entry Has Been Found'. I tried again by clicking on 'Import Volume' and it pulls up my volume r2d2 as the only option to import but it fails. I captured this:

So I ssh'd in and did a 'zpool status' and it comes with this:

I don't understand. If the GUI sees r2d2 enough to give it as an option to import, why can't I import it? Is there anything I can do from command line to try it, or at least see the errors so we can tell where exactly it is failing?
Are there any commands I can run that would help you guys troubleshoot?
I want to say again a HUGE thank you for all your guys time and effort you've given me on this so far. It is appreciated greatly!!

 

[okgunguy]

From my system emails I've gathered this:
On Sept 29th at 3:45am - starting scrub of pool 'freenas-boot'
On Sept 29th at 10:42pm - The volume r2d2 (ZFS) state is ONLINE: One or more devices has experienced an error resulting in data corruption. Applications may be affected.
On Sept 29th at 11:03pm - The volume r2d2 (ZFS) state is DEGRADED: One or more devices has experienced an error resulting in data corruption. Applications may be affected.
On Sept 30th at 3:01am -
freenas.local kernel log messages:
> pid 88048 (Plex Media Scanner), uid 972: exited on signal 6
> vm_fault: pager read error, pid 4680 (Plex Media Server)
> pid 4680 (Plex Media Server), uid 972: exited on signal 11

-- End of security output --
And then more emails saying the same thing about r2d2 is DEGRADED.
Then on Sept 30th at 9:57pm - The volume r2d2 (ZFS) state is UNKNOWN:
I think this was after I rebooted. Then after that I just keep getting emails over and over saying r2d2 is UNKNOWN.

 

[Robert Trevellyan]

If the GUI sees r2d2 enough to give it as an option to import, why can't I import it?

Presumably because it's sufficiently corrupted.

Are there any commands I can run that would help you guys troubleshoot?

I'm sure there are, but unfortunately I'm getting out of my depth here. However, I will say that it looks like the pool failure caused the Plex crash and not vice versa.

 

[okgunguy]

So is this a 9.3.1 problem? I see all of you are running 9.3
Before I go through all of this again, it doesn't make sense to do it on a version that nobody else is running.
Also, I have a couple of copies of my old config that I had sent to my email. I looked in the manual, but I couldn't find how to import it back into the NAS and restore it.

 

[Vivan]

I just wanted to thank you for so thoroughly documenting your experiences. I'm building an almost identical build (only difference is the case, drives and PSU...) and it is super helpful... even seeing when things go wrong.

 

[Robert Trevellyan]

So is this a 9.3.1 problem? I see all of you are running 9.3

No, it's a "your pool is corrupted" problem. The issues I'm aware of with 9.3.1 relate to replication. EDIT: and when you're on 9.3 STABLE and you do an incremental update, it doesn't change the version number to 9.3.1.

I couldn't find how to import it

System | General | Upload Config

 

[Vivan]

The P20 came from LSI. None of these drives are attached to the LSI controller.

Uhhhh, why? Doesn't that defeat the purpose of the controller?

 

[okgunguy]

Uhhhh, why? Doesn't that defeat the purpose of the controller?

Not if I plan on using the LSI ports for expansion.

 

[okgunguy]

I found an old post from cyberjock that gave me the command

Code:

[root@freenas] ~# zpool import -f r2d2
cannot import 'r2d2': I/O error
  Destroy and re-create the pool from
  a backup source.
[root@freenas] ~#

But I get the above output. I'm not getting a warm fuzzy that anything other than a total rebuild is possible.

 

[Robert Trevellyan]

When I Google "zpool import I/O error" the -F flag comes up in a lot of threads. According to the documentation:

-F Recovery mode for a non-importable pool. Attempt to return
the pool to an importable state by discarding the last few
transactions. Not all damaged pools can be recovered by using
this option. If successful, the data from the discarded
transactions is irretrievably lost. This option is ignored if
the pool is importable or already imported.

So, I don't know if it will help, but it might be worth a try.

EDIT: -F can be combined with -n for a dry run.

 

[okgunguy]

Thank you Robert for your help. None of these ideas have resolved the problem.
I have decided rather than waste any more time trying to recover my data, that I will just start over and keep extra copies of everything that I place on the NAS. Until it can prove itself as reliable.
My next step is to decide on how to reinstall Plex. My initial install was through the GUI as described in this blog. But there are other threads on here showing to do it through the command line. Like this one. I was wanting to put the question out there asking how did you all install your Plex? Was it through the GUI? Or through CL? Has it ever crashed on you? How long has it been operational?
I will reinstall the newest release of FreeNAS and put some media on it. Then I will proceed in creating snapshots, config saves.. etc. Then I will purposely create problems and see if it can be recovered. I am sure that this was not caused by a hardware issue. Zero change in SMART numbers. Zero instances of any ECC errors reported by IPMI. Zero chance of any power issue. Either Plex or FreeNAS or both took a dump and couldn't recover. And my poor data got flushed down the crapper.

 

[nightshade00013]

I install PLEX via command line. It's extremely simple, create a jail. I generally just use the GUI's shell for the Jail in particular. Then it's as simple as the commands

pkg update
pkg upgrade

#for the stable version
pkg install plexmediaserver

#for the latest and greatest version

pkg install plexmediaserver-plexpass

They both work pretty much the same and can both be used with Plex Pass. The difference is that the plexpass version is a slightly later version.

Then later on it's as simple as updating and upgrading around once a month. You can even kinda automate it if you have a bunch of jails https://forums.freenas.org/index.php?threads/how-to-update-all-of-your-jails-at-once.21397/


What part of Oklahoma are you in. I am about 20 miles north of OKC if you ever need help. Still learning but been messing around for a while now learning a thing or two.

 

[okgunguy]

What part of Oklahoma are you in. I am about 20 miles north of OKC if you ever need help. Still learning but been messing around for a while now learning a thing or two.

I'm in Choctaw, East side of OKC. How long has your Plex been running? A month? A year? Has it ever frozen up on you? What are you using for the client?

 

[nightshade00013]

I have actually had the Plex running as it sits for around 6 months. I started off using the preconfigured jail package but once I figured a few things out I started using the port. So far has not frozen up at all. I don't have a plex pass but I use the web interface about as equally as the DLNA server portion at home and have used the web interface from outside my network as well as the Plex app but only to show a few people the features.

Last night I literally just switched the drives over from my testbed to what will eventually be my final system and started using it all there but it's the same install of plex I have had just with more CPU and RAM to work with and still works just fine. I can have multiple clients watching movies at the same time with zero issues.

Ok I know where that is. Hopefully didn't have any problems with the big commotion going on out there over the dogs and such that was posted on OSA