Degraded Plex Volume after power outage!

[whitewhale]

I am running FreeNAS-11.1-U1 on a dell server. We had a power outage yesterday and today when I brought it back on-line freenas had a critical error- "CRITICAL: Sept. 16, 2018, 5:45 p.m. - The volume PlexJail state is DEGRADED: One or more devices has experienced an error resulting in data corruption. Applications may be affected." I have rebooted the system and it doesn't seem to give the error immediately it happens about 15 minutes after boot up.

On the Plex interface it states- "There was an unexpected error loading this library Please visit our forums if you continue to experience problems." I tried from the plex side to optimize the database but this still didn't fix anything.

I am running a two drive mirror system but not sure how to repair or fix the issue.
I read a few forum where they ask for portions of the log files so I have included that below. It seems like this should be fixable but not sure how to do it. Any guidance would be helpful. Thanks.

[root@freenas ~]# zpool status -v
pool: PlexJail
state: DEGRADED
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://illumos.org/msg/ZFS-8000-8A
scan: scrub repaired 0 in 0 days 00:01:15 with 16 errors on Sun Sep 16 18:34:23 2018
config:

NAME STATE READ WRITE CKSUM
PlexJail DEGRADED 0 0 361
gptid/3644a324-19ab-11e8-b8aa-d89ef31b1e18 DEGRADED 0 0 722 too many errors

errors: Permanent errors have been detected in the following files:

/mnt/PlexJail/Jail/plexmediaserver_1/var/db/pkg/vuln.xml
/mnt/PlexJail/Jail/plexmediaserver_1/var/spool/clientmqueue/dfw8EAosRB003638
/mnt/PlexJail/Jail/plexmediaserver_1/var/db/plexdata/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library
.db

pool: freenas-boot
state: ONLINE
scan: scrub repaired 0 in 0 days 00:00:17 with 0 errors on Wed Sep 12 03:45:17 2018
config:

NAME STATE READ WRITE CKSUM
freenas-boot ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
da0p2 ONLINE 0 0 0
da1p2 ONLINE 0 0 0

errors: No known data errors

pool: mediastorage
state: ONLINE
scan: scrub repaired 0 in 0 days 02:38:30 with 0 errors on Sun Aug 5 02:38:30 2018
config:

NAME STATE READ WRITE CKSUM
mediastorage ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
gptid/6ed4edfe-19a0-11e8-b8aa-d89ef31b1e18 ONLINE 0 0 0
gptid/6f8f5289-19a0-11e8-b8aa-d89ef31b1e18 ONLINE 0 0 0

errors: No known data errors
[root@freenas ~]#

 

[IceBoosteR]

Hi,

you are running in a mirrored setup for your media only. The location where you have stored you PLEX library and information has no redundancy, therefore ZFS is correct, you are having a data loss, and it is also aware which files are affected. Without knowing anything about PLEX, from the filelist I can say, the DB itself has a corruption. Before you try to do anything to repair that, restore your PLEX config from backup. our media is still there, so PLEX can discover the media and youre fine. Next time, use redundancy.
-Ice

 

[danb35]

Next time, use redundancy.

...or just put the jail on the mirrored pool you already have. There's no reason to build a separate pool just for one jail.

 

[IceBoosteR]

...or just put the jail on the mirrored pool you already have. There's no reason to build a separate pool just for one jail.

Right;)

 

[Chris Moore]

Sorry, you made a mistake. You can delete the individual files that are damaged and run a scrub. It will not be able to correct the problem because the pool you have for jails is not a mirror, but the error message should be cleared up by the scrub.
You can protect yourself from this in the future by adding a mirror to this pool.

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[whitewhale]

Thank you everyone for the responses. I did what you recommend, deleted the bad files and scrubbed the pool. It worked! I am just a little too new to freenas. The jails/volumes still are a little confusing. I thought that I had covered all the bases because I am set it up with two usb sticks for the freenas OS. I guess I thought that would be where the setting for the PLEX server would exist and so I thought it was mirrored. Maybe someone could explain what I need to do to avoid this situation in the future?

 

[danb35]

Well, jails (which include plug-ins) are never stored on the OS device. They're generally stored on the data pool, and in the most common case, you only have one pool. You've set up a separate pool for your Plex jail, and you don't have any redundancy on that pool. I'd recommend changing the jails path to be on your storage pool, so you can put them on a pool with some redundancy.

 

[Chris Moore]

Thank you everyone for the responses. I did what you recommend, deleted the bad files and scrubbed the pool. It worked! I am just a little too new to freenas. The jails/volumes still are a little confusing. I thought that I had covered all the bases because I am set it up with two usb sticks for the freenas OS. I guess I thought that would be where the setting for the PLEX server would exist and so I thought it was mirrored. Maybe someone could explain what I need to do to avoid this situation in the future?

I am guessing that you put your jails on a SSD? Many people do that thinking it will gain them some speed / responsiveness. As @danb35 said, you can move the jails to your main pool. I would add that you can add a redundant disk (a mirror) to the existing jails pool. Either of those options would take some interaction at the command line.
It might be simpler to destroy the existing jails pool and create a jails dataset in your main pool where you can install a fresh instance of the Plex jail. This would not affect the movies you have stored in your pool, but you would loose any customization you may have done in your Plex. Over the years, I have had to reinstall Plex a couple times and it can be a chore, but it isn't the end of the world.

 

[gabsd]

I am guessing that you put your jails on a SSD? Many people do that thinking it will gain them some speed / responsiveness.

Hi Chris,
I was looking for advice on where to install jails and came upon this older post of yours that almost answers my question. Did you mean that it is wrong to think that you will gain speed from installing a jail in an SSD?

Specifically I am thinking about a Plex jail. I have FreeNAS 11.2 running on a TS440 (E3 1225 v3, 32Gb RAM). I have one pool of 3 x 3TB WD Red. I want to install the Plex plugin and was thinking I'd better add a second pool of 2x 240 Gb SSDs for it. Would this be pointless?

Thanks!!