SOLVED raid3 sync frozen @ 4%

[ThePowerTool]

FreeNAS-9.2.1.9-RELEASE-x86 (2bbba09)

[root@freenas] ~# graid3 status
Name Status Components
raid3/raid3 DEGRADED ada2 (ACTIVE)
ada1 (SYNCHRONIZING, 4%)
ada0 (ACTIVE)

It seems I'm stuck @ 4%. I'm including /var/log/messages, below (at least a snip as it's long).

Can someone please give me an idea what to check or let me know if you want more details on my specific server install.

Thanks!

/var/log/messages

Code:

May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.109518,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.110310,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.110410,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.110479,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.350618,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.351512,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.353508,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.353632,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.609509,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.609751,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.609824,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.609908,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.795434,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.796710,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.797477,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:10 freenas smbd[2362]: [2017/05/20 21:00:10.797990,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:10 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:11 freenas smbd[2362]: [2017/05/20 21:00:11.028022,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:11 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:11 freenas smbd[2362]: [2017/05/20 21:00:11.028289,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:11 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:11 freenas smbd[2362]: [2017/05/20 21:00:11.028350,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:11 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:11 freenas smbd[2362]: [2017/05/20 21:00:11.028412,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:11 freenas smbd[2362]:  unix_filetype: unknown filetype 0
May 20 21:00:11 freenas smbd[2362]: [2017/05/20 21:00:11.263416,  0] ../source3/smbd/trans2.c:1349(unix_filetype)
May 20 21:00:11 freenas smbd[2362]:  unix_filetype: unknown filetype 0

 

[ThePowerTool]

It just bumped to 5%.

I may have posted too quickly.

 

[styno]

Yes, depending on the implementation/age/chipset/disksizes used on the motherboard, these “hardware-assisted software RAID" setups can take a looooong time to resilver. (and thus increasing the risk of a 2nd disk failure during recovery.)

I guess you are using aging hardware and depending on the data you may be better of looking for a hardware replacement that can handle zfs.

 

[ThePowerTool]

Yes, upgrades are high on my priority list but it may be a bit, still. Once I do get new hardware I'll post what I've been using so everyone can get a laugh.

 

[Stux]

Rebuilds can also take a long time if one of the drives is faulty... either reading or writing slowly, or possibly failing for long times between etc... especially if they're not enterprise/NAS drives.

 

[Ericloewe]

I'll take this opportunity to point out that the forum has lost what little UFS+GEOM knowledge it had, in the two and a half years since FreeNAS dropped support for that.

You really should start planning for an update to a modern version and ZFS.

 

[ThePowerTool]

You really should start planning for an update to a modern version and ZFS.

I am running a 14TB raid 3 array on an old IBM IntelliStation Z Pro 6221 Xeon (Intel e7505) 3.2GHz with 4G of memory (which is max memory).

I believe upgrading to ZFS will require 14G of memory to support 14TB of storage :/ -- (a significant upgrade) hence the delay.

 

[danb35]

I believe upgrading to ZFS will require 14G of memory to support 14TB of storage

Well, it will require 8 GB of RAM at a minimum, to be sure. And for most folks, 16 GB is a better bet. You can buy suitable new servers for under US$300 plus some RAM and disks, so the upgrade need not be terribly expensive.

Edit: This HPE Proliant ML10 Gen9 is US$200. Add RAM (it comes with 4 GB, but you'd wantneed more; another 16 GB in 2 x 8 GB sticks would be under $200) and disks (which you could move over from your existing system, though you'd need to back up your data somewhere else first), and you'd be ready to go with a new, modern server, which would also be much more energy-efficient.

As @Ericloewe mentions, there's never been great expertise here with UFS, as FreeNAS has really been focused on ZFS since version 8.something. In the last couple of years, support for UFS has been dropped completely, and therefore what expertise has been here has mostly gone. In short, if you continue with your present configuration, you're going to be pretty much on your own.