For reasons below, we might also add "user has brain problems", but hopefully you will forgive my panicked mistake/not make too much fun of me. Also, I have seen that 4GB is too little RAM for ZFS, so I shall be upgrading that as soon as I am able.
The issue is very similar to this thread: Pool won't import, disks are now missing - except that I have no multipath issues reported and insofar as I can tell multipath has not been enabled.
Also, it's pretty long and I tend not to explain things very well, so thank you to anyone who reads all the way through.
The issues are:
All drives are SAMSUNG, a process of logic tells me that one of the drives not showing up is attached to each of the controllers (ada1 to the Adaptec controller, ada4 to the ASRock mainboard).
I am about 90% certain that two things preceded this:
Initially I saw the following:
The 2 Drives attached to the mainboard didn't really show up. However the ASRock post would show all 4 drives as JBOD disks, configured in Legacy mode. However, the drive attached to port 1 (ada1) showed up as non-configured.
Here's where I did something foolish. In the Adaptec configuration tool I highlighted and hit configure on ada1 - so it now shows up as a Simple Volume and has its status shown as OPTIMAL rather than Legacy or non-configured. I am guessing that this drive is basically dead and gone in terms of its data, so all of my hope rests on recovering ada4 and then forcing a replacement on ada1 and resilvering. I am also fairly sure that this might not be possible.
Diagnostics/Commands/Status
Here are a few things suggested in the similar thread I linked above:
Pastbin:
Full dmesg output
smartctl -a /dev/ada1
smartctl - a /dev/ada4
Hopefully that will provide the various gurus I've been reading the responses from somewhere to start.
The issue is very similar to this thread: Pool won't import, disks are now missing - except that I have no multipath issues reported and insofar as I can tell multipath has not been enabled.
Also, it's pretty long and I tend not to explain things very well, so thank you to anyone who reads all the way through.
The issues are:
- The pool won't mount
- Disks can be seen by the system on lower levels, but not on higher levels
- Core i3
- 4GB RAM
- FreeNAS 9.1 on a USB Stick
- 6 2TB HDDs
- RAIDZ with a single disk of redundancy
All drives are SAMSUNG, a process of logic tells me that one of the drives not showing up is attached to each of the controllers (ada1 to the Adaptec controller, ada4 to the ASRock mainboard).
- 2 Disks attached to the ASRock Motherboard, in AHCI mode
- 4 Disks attached to an Adaptec 1430SA
I am about 90% certain that two things preceded this:
- A too short network cable that caused the box to jolt when it was being slightly moved (a silly idea when it's turned on, I know).
- Heat - it was very hot, and the CPU fan had been blocked by a cable end that had fallen into its housing. The CPU was thus cooling ambiently/passively and indirectly heating the HDDs, idling at around 90C!
Initially I saw the following:
The 2 Drives attached to the mainboard didn't really show up. However the ASRock post would show all 4 drives as JBOD disks, configured in Legacy mode. However, the drive attached to port 1 (ada1) showed up as non-configured.
Here's where I did something foolish. In the Adaptec configuration tool I highlighted and hit configure on ada1 - so it now shows up as a Simple Volume and has its status shown as OPTIMAL rather than Legacy or non-configured. I am guessing that this drive is basically dead and gone in terms of its data, so all of my hope rests on recovering ada4 and then forcing a replacement on ada1 and resilvering. I am also fairly sure that this might not be possible.
Diagnostics/Commands/Status
Code:
# uname -a FreeBSD carbon.dis 9.1-STABLE FreeBSD 9.1-STABLE #0 r+16f6355: Tue Aug 27 00:38:40 PDT 2013 root@build.ixsystems.com:/tank/home/jkh/src/freenas/os-base/amd64/tank/home/jkh/src/freenas/FreeBSD/src/sys/FREENAS.amd64 amd64
Code:
# zpool import pool: Omega id: 15715021659324379333 state: UNAVAIL status: One or more devices are missing from the system. action: The pool cannot be imported. Attach the missing devices and try again. see: http://illumos.org/msg/ZFS-8000-3C config: Omega UNAVAIL insufficient replicas raidz1-0 UNAVAIL insufficient replicas gpt/ada0 ONLINE 9434423201809139189 UNAVAIL cannot open gpt/ada2 ONLINE gpt/ada3 ONLINE 2943002579415553192 UNAVAIL cannot open gpt/ada5 ONLINE
Code:
# camcontrol devlist <SAMSUNG HD204UI 1AQ10001> at scbus0 target 0 lun 0 (ada0,pass0) <SAMSUNG HD204UI 1AQ10001> at scbus1 target 0 lun 0 (ada1,pass1) <SAMSUNG HD204UI 1AQ10001> at scbus2 target 0 lun 0 (ada2,pass2) <SAMSUNG HD204UI 1AQ10001> at scbus3 target 0 lun 0 (ada3,pass3) <SAMSUNG HD204UI 1AQ10001> at scbus7 target 0 lun 0 (ada4,pass4) <SAMSUNG HD204UI 1AQ10001> at scbus8 target 0 lun 0 (ada5,pass5) <Imation Classic PMAP> at scbus9 target 0 lun 0 (pass6,da0)
Code:
# gpart show => 34 3907029101 ada0 GPT (1.8T) 34 94 - free - (47k) 128 4194304 1 freebsd-swap (2.0G) 4194432 3902834703 2 freebsd-zfs (1.8T) => 34 3907029101 ada2 GPT (1.8T) 34 94 - free - (47k) 128 4194304 1 freebsd-swap (2.0G) 4194432 3902834703 2 freebsd-zfs (1.8T) => 34 3907029101 ada3 GPT (1.8T) 34 94 - free - (47k) 128 4194304 1 freebsd-swap (2.0G) 4194432 3902834703 2 freebsd-zfs (1.8T) => 34 3907029101 ada5 GPT (1.8T) 34 94 - free - (47k) 128 4194304 1 freebsd-swap (2.0G) 4194432 3902834703 2 freebsd-zfs (1.8T) => 63 15116673 da0 MBR (7.2G) 63 1930257 1 freebsd [active] (942M) 1930320 63 - free - (31k) 1930383 1930257 2 freebsd (942M) 3860640 3024 3 freebsd (1.5M) 3863664 41328 4 freebsd (20M) 3904992 11211744 - free - (5.4G) => 0 1930257 da0s1 BSD (942M) 0 16 - free - (8.0k) 16 1930241 1 !0 (942M)
Code:
# glabel status Name Status Components gpt/ada0 N/A ada0p2 gpt/ada2 N/A ada2p2 gpt/ada3 N/A ada3p2 gpt/ada5 N/A ada5p2 ufs/FreeNASs3 N/A da0s3 ufs/FreeNASs4 N/A da0s4 ufs/FreeNASs1a N/A da0s1a
Code:
# dmesg | grep ada1 ada1 at mvsch1 bus 0 scbus1 target 0 lun 0 ada1: <SAMSUNG HD204UI 1AQ10001> ATA-8 SATA 2.x device ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada1: quirks=0x1<4K> ada1: Previously was known as ad6 GEOM_RAID: DDF-LE: Disk ada1 state changed from NONE to ACTIVE. GEOM_RAID: DDF-LE: Subdisk SimpleVol:0-ada1 state changed from NONE to ACTIVE.
Code:
# dmesg | grep ada4 ada4 at ahcich4 bus 0 scbus7 target 0 lun 0 ada4: <SAMSUNG HD204UI 1AQ10001> ATA-8 SATA 2.x device ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada4: Command Queueing enabled ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada4: quirks=0x1<4K> ada4: Previously was known as ad18
Here are a few things suggested in the similar thread I linked above:
Code:
# gpart recover /dev/ada1 gpart: arg0 'ada1': Invalid argument # gpart recover /dev/ada4 gpart: arg0 'ada4': Invalid argument
Code:
# gmultipath list #
Pastbin:
Full dmesg output
smartctl -a /dev/ada1
smartctl - a /dev/ada4
Hopefully that will provide the various gurus I've been reading the responses from somewhere to start.