Unsure if my NAS is failing, help please!

Status
Not open for further replies.
Syner9y

Syner9y

Cadet
Joined
Jan 11, 2016
Messages
5
Hello all. Sorry for the long post, but I wasn't sure what else to do. First of all, I tried reading as much as I could about all of this over the past couple months as I didn't want to clutter up the forum with previously answered questions. However, I am at a point where I am unsure of the best way forward and I could really use some assistance. I will admit I do not know as much about FreeNAS as I should, however I am not afraid to learn so if you just want to provide a resource where I can find the answer, it would be much appreciated.

My specific issue is about 6-9 months ago, I began getting strange messages from my FreeNAS daily log output emails (unfortunately, I did not save these). At the time, I was busy with grad school, so after a quick glance which made it look like a drive was failing, I shut the NAS down until I could spend more time on it. Fast-forward to 2 months ago, when I finally had time to boot it back up. The first thing I had to fix was it would not boot up as the boot drive, an mSATA, was not even on the list of boot devices, so I had to go into the BIOS to fix that. Then, once resolved, there were about 10-12 versions of FreeNAS to choose to boot into, so I picked the most recent and then cleared the older ones once into the GUI.

At this point, everything seemed to work, minus an alert in the GUI. However, on March 19th, I got the following log email:
Code:
Checking status of zfs pools:
NAME		   SIZE  ALLOC   FREE  EXPANDSZ   FRAG	CAP  DEDUP  HEALTH  ALTROOT
freenas-boot  14.9G  3.62G  11.3G		 -	  -	24%  1.00x  ONLINE  -
oldboot	   14.5T  3.69T  10.8T		 -	14%	25%  1.00x  ONLINE  /mnt

  pool: oldboot
state: ONLINE
status: One or more devices has experienced an error resulting in data
		corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
		entire pool from backup.
   see: http://illumos.org/msg/ZFS-8000-8A
  scan: scrub in progress since Sun Mar 19 00:00:00 2017
		3.48T scanned out of 3.69T at 336M/s, 0h10m to go
		0 repaired, 94.40% done
config:

		NAME										  STATE	 READ WRITE CKSUM
		oldboot									   ONLINE	   0	 0	 0
		  gptid/9c95aa50-7d71-11e5-b959-d050996a0ef8  ONLINE	   0	 0	 0
		  gptid/9d504809-7d71-11e5-b959-d050996a0ef8  ONLINE	   0	 0	 0
		  gptid/27c71147-9bae-11e5-8053-d050996a0ef8  ONLINE	   0	 0	 0
		  gptid/28c17a02-9bae-11e5-8053-d050996a0ef8  ONLINE	   0	 0	 0
		spares
		  gptid/6cea195f-b743-11e5-84fe-d050996a0ef8  AVAIL

errors: 5 data errors, use '-v' for a list

-- End of daily output --


I viewed the errors and they were about metadata, which from reading other's similar troubles, looked like I would need to wipe and restore from backup to resolve. However, I did not have backups (as the NAS is primarily for transient files that I don't mind losing) so I wanted to try other possibilities first. I cleared the errors, restarted the NAS and waited as everything still seemed to work.

3 days later, I got this email:
Code:
freenas.local kernel log messages:
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 a0 fb ad 40 67 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 50 cf 74 40 d7 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 b8 d6 fb 40 19 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 78 92 0b 40 1e 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 3d 68 40 1e 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 c8 8d 10 40 1f 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 00 c0 78 40 22 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 40 a0 39 f2 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 c0 06 74 40 a5 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 d0 d6 9c 40 a8 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 f8 64 b8 40 52 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 00 9f d6 40 42 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 30 8d 1c 40 1f 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 98 44 a5 40 1a 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 78 a3 cd 40 65 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 e8 65 ea 40 6c 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 80 67 b6 40 8d 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 00 72 97 40 09 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 50 05 7f 40 73 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 48 75 f3 40 c1 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 c0 b4 26 40 09 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 08 ad 89 40 1b 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 40 20 61 f2 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 98 06 79 40 ab 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 68 ab d9 40 82 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 50 c4 a0 40 6d 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 80 5b c6 40 47 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 98 8f 7e 40 87 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 40 eb d6 40 42 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 30 90 70 f2 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 80 2a 79 40 1d 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 10 54 a3 40 65 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 86 1d 40 08 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 f8 7d cc 40 0b 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 20 06 95 40 0b 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 f0 47 78 40 04 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 20 04 50 40 fa 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 58 87 5e 40 00 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 30 c8 7c f2 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 48 8c d1 40 42 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 f8 7d cc 40 0b 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 20 06 95 40 0b 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 f0 47 78 40 04 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 20 04 50 40 fa 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 58 87 5e 40 00 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 40 d8 7d f2 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 40 a2 40 08 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 48 3e 6d 40 42 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 c0 a8 f3 4d 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 40 a2 40 08 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 48 3e 6d 40 42 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 40 a2 40 08 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 48 3e 6d 40 42 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 48 8f d1 40 42 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 40 a2 40 08 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 48 3e 6d 40 42 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 48 90 d1 40 42 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 5d 5c 40 5e 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 40 fa ae 40 a5 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 48 91 d1 40 42 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 68 fb 9f 40 fa 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 98 6a 82 40 08 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 22 2e 40 67 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 68 fb 9f 40 fa 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 98 6a 82 40 08 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 22 2e 40 67 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 5d 5c 40 5e 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 40 fa ae 40 a5 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 48 91 d1 40 42 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 80 58 7e f2 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 68 fb 9f 40 fa 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 98 6a 82 40 08 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 22 2e 40 67 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 5d 5c 40 5e 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 40 fa ae 40 a5 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 18 50 00 d7 40 42 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 f8 37 1a 40 8f 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 b0 68 12 40 f4 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 50 b5 e1 40 3e 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 40 18 7f f2 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 50 b5 e1 40 3e 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 40 18 7f f2 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 f8 37 1a 40 8f 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 b0 68 12 40 f4 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 f8 37 1a 40 8f 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 b0 68 12 40 f4 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 50 b5 e1 40 3e 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 c0 a8 f4 4d 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 f8 37 1a 40 8f 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 b0 68 12 40 f4 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 50 b5 e1 40 3e 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 e0 88 f4 4d 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 f8 37 1a 40 8f 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Error 5, Retries exhausted
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 b0 68 12 40 f4 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Error 5, Retries exhausted
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 50 b5 e1 40 3e 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Error 5, Retries exhausted
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 48 95 d1 40 42 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 f8 37 1a 40 8f 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 30 58 7f f2 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 d8 29 bf 40 05 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 e0 25 47 40 6a 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 c0 1b 7d 40 05 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 38 d4 b8 40 0b 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 78 5c 2c 40 01 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 f8 c5 c4 40 07 01 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 b0 b9 2c 40 f4 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 68 7d ee 40 8a 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 40 b8 7f f2 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 68 30 d3 40 42 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 c0 58 29 4e 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 68 37 d3 40 42 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 68 38 d3 40 42 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 08 f0 db 40 42 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 50 20 31 fb 40 40 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 20 f0 db 40 42 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 28 50 f0 db 40 42 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 68 3a d3 40 42 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 68 3a d3 40 42 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 68 3a d3 40 42 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 c8 3e d3 40 42 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> tun0: link state changed to DOWN
> tun0: link state changed to UP
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 18 18 a2 64 40 43 00 00 00 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 70 a2 64 40 43 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 78 a3 64 40 43 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command
> (ada2:ahcich2:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 00 90 b0 64 40 43 00 00 01 00 00
> (ada2:ahcich2:0:0:0): CAM status: Uncorrectable parity/CRC error
> (ada2:ahcich2:0:0:0): Retrying command

-- End of security output --


I then found a similar post someone made, where as a part of debugging, someone else wanted the smart report to be ran, which I did (I don't have that report right now but if you feel it's pertinent, I will re-run and post it). However, this did not provide me with anything noticeable in terms of problems so I chose to ignore it as everything else still worked without issue.

Finally, a couple days ago, one of my jails (I have 2, a Transmission one that is problematic and a MySQL one that is working) began acting up when I was attempting to update software. I couldn't do much as everything I tried, even running ls, gave me the same error:
Code:
Shared object "libncurses.so.8" not found, required by "ls"


I saw another post by someone that had a similar issue where restarting the box resolved it, so I did that. However, now I cannot even get into the jail as I get the following when I run jexec 2:
Code:
Shared object "libncurses.so.8" not found, required by "csh"


Fortunately, I can still access all the files in the jail from outside of the jail, and it sounds like from other posts, that my only option now is to delete the jail and re-create it. Sorry for the long post, but I wanted to be thorough. At this point, I have a couple questions I'm hoping you can help with so I don't further muck this up:
  1. Am I correct in my assumption that this jail is hosed and needs to be deleted and then re-installed? If not, what should be done to correct this?
  2. There are a lot of configs in this jail I would like to preserve before deleting. What is the best way to save the contents of the jail so I can restore specific files once it is recreated? Is copying it on the NAS the best approach, trying to ftp it off, plugging in a USB drive, etc? I suppose best in this case means full backup of the jail's files in the quickest/easiest way.
  3. What happened to my NAS? I know I messed around a lot once one thing went wrong and I am perfectly willing to accept I brought this on myself. However, a bigger question I have is is it possible an HD or other hardware component is failing? Or was it simply that one or two files got corrupted and then in trying to fix it, I made it much, much worse?
Again, I am a complete noob with this stuff so I apologize if I did miss something obvious. Also, I would be more than happy to try anything (short of wiping everything) and provide any additional information/logging needed to get to the bottom of this. Thank you so much for reading!

EDIT: Oops, forgot to mention I am running FreeNAS 9.10.2-STABLE
 
Vito Reiter

Vito Reiter

Wise in the Ways of Science
Joined
Jan 18, 2017
Messages
232
It looks like you've got a really weird setup buddy,

You have a FreeNAS-boot volume which looks standard (1-2 drives, a small amount of space). Then you've got this oldboot volume, I don't know why it's named that or what it does but it's not very clear what form of redundancy you have. Obviously, it looks like ada2 is failing, if you have a mirror or a Z1/2/3 setup then you should be able to replace that drive. Jails directories should be stored on that redundant volume so in the case of a failure you can easily replace a drive (i.e ada2) and be back in business and if an error occurs just scrub the volume and see if anything is corrected by ZFS. Now, I'm not sure how much I can recommend other than stop scrubbing the system with a failed drive in it. If you could list some full hardware specs and whether or not you have ECC RAM that would really help. If you don't have ECC you need to not run scrubs until you do, because in the event of a RAM issue this can destroy the whole pool (which could also be the cause of the errors you're experiencing).

You mentioned just having transient files on the system that you can afford to lose. A lot of people on here say that but it's always a headache getting them back and I strongly recommend against a Raid 0 setup even if you can lose the files because it's a headache to get back up and running when a striped array fails. In that case run a RaidZ1 at least so when a drive fails you can replace and most likely get away unharmed all while your NAS is running.

If you're receiving Uncorrectable Sector errors like mentioned above the drive is definitely on its way to dying, especially when it's multiple sectors. Sectors are usually pretty small and almost never larger than 4K so you won't see too much issue until there's a ton of them giving errors. Moral of the story here is, I believe you have a failing drive with the information you've provided. If there's redundancy setup replace it, if not, you need to start logistics on what you're gonna do, backup the important stuff, and rebuild with at least a RaidZ1 to avoid having to go through the whole process in the future.
 
R

rs225

Guru
Joined
Jun 28, 2014
Messages
878
This pool has no redundancy. Backup critical files now.

After that backup the whole thing and recreate from scratch as raidz1.

If you can't do that, you might survive long enough by upgrading the dying drive ada2 to a mirror, if you have an extra drive/port for that. You've nearly finished a scrub, so odds are high you could upgrade to a mirror then swap out the failing drive.
 
Jailer

Jailer

Not strong, but bad
Joined
Sep 12, 2014
Messages
4,977
Please list all your hardware as well as your pool layout.
 
Status
Not open for further replies.

Similar threads

S
  • Locked
WRITE_FPDMA_QUEUED errors
Replies
14
Views
3K
smartyarts
S
rdl
Drive problem apparently.
Replies
5
Views
718
sretalla
sretalla
M
  • Locked
Pretty Sure Drive Is On Its Way Out
Replies
5
Views
1K
droeders
droeders
Joe Fenton
Daily security run output email since upgrading to Freenas 11.1
Replies
7
Views
2K
Joe Fenton
Joe Fenton
nello
  • Locked
SOLVED Disk Going Bad?
Replies
10
Views
3K
joeschmuck
joeschmuck
Top