Hard Drive: "Lost Device"

[Kory Fierstine]

OK, I'm basically a noob. I've been running a ZFS raid (3 x 2TB) without issue for almost a year, but I am not a command line dude, nor a software engineer. I just read the instructions and dinked around until I got the thing to function.

My issue is that I recently ran out of space and tried to add a 4th 2TB HDD. In the BIOS on my Dell Vostro 220, I could/can see the 4th HDD (I know, I know, crappy foundation on which to lay freenas, but it's what I have). After my initial reboot, I saw the device in the GUI under "View Disks." I then used the ZFS volume manager to extend the volume and when it finished, under "View Disks" the HDD is gone. I'm not a command line kind of guy, but using some gouge from a previous complainer, I saw to do the following so I post the output for you to decipher:

[root@freenas ~]# dmesg | grep -i ata
atapci0: <Intel ICH9 SATA300 controller> port 0xf700-0xf707,0xf600-0xf603,0xf500
-0xf507,0xf400-0xf403,0xf300-0xf30f,0xf200-0xf20f irq 19 at device 31.2 on pci0
ata2: <ATA channel> at channel 0 on atapci0
ata3: <ATA channel> at channel 1 on atapci0
atapci1: <Intel ICH9 SATA300 controller> port 0xf000-0xf007,0xef00-0xef03,0xee00
-0xee07,0xed00-0xed03,0xec00-0xec0f,0xeb00-0xeb0f irq 19 at device 31.5 on pci0
ata4: <ATA channel> at channel 0 on atapci1
ata5: <ATA channel> at channel 1 on atapci1
ada0 at ata2 bus 0 scbus0 target 0 lun 0
ada0: <WDC WD20EZRX-00D8PB0 80.00A80> ATA-9 SATA 3.x device
ada0: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada1 at ata3 bus 0 scbus1 target 0 lun 0
ada1: <WDC WD20EZRX-00D8PB0 80.00A80> ATA-9 SATA 3.x device
ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada2 at ata4 bus 0 scbus2 target 0 lun 0
ada2: <WDC WD20EZRX-00D8PB0 80.00A80> ATA-9 SATA 3.x device
ada2: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada3 at ata5 bus 0 scbus3 target 0 lun 0
ada3: <WDC WD20EZRX-00D8PB0 80.00A80> ATA-9 SATA 3.x device
ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
(ada3:ata5:0:0:0): lost device
(ada3:ata5:0:0:0): removing device entry

You can see at the end here, the device is detected, but freenas boots it out for some reason. What can I do to a) get it back, and b) add it to my existing volume.

Thanks.

 

[Dennis K.]

Please post the output of "zpool status" It might be the case, that you do not want to extend your pool that way in the first place.

 

[danb35]

It is most definitely the case that you don't want to extend your pool that way, but I'm concerned that you might have anyway. zpool status will show what happened.

Kory, take a careful look through the powerpoint posted here: http://forums.freenas.org/index.php...ning-vdev-zpool-zil-and-l2arc-for-noobs.7775/. If you were successful in extending the pool this way, you've really messed up your data redundancy, and that will explain why.

 

[Kory Fierstine]

Here's my spool status. I'll take a look at the PPT. Thanks.

[root@freenas ~]# zpool status
pool: fierpool1
state: ONLINE
status: The pool is formatted using a legacy on-disk format. The pool can
still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'. Once this is done, the
pool will no longer be accessible on software that does not support feature flags.
scan: scrub repaired 0 in 6h36m with 0 errors on Sun Aug 3 06:36:25 2014
config:

NAME STATE READ WRITE CKSUM
fierpool1 ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
ada0 ONLINE 0 0 0
ada1 ONLINE 0 0 0
ada2 ONLINE 0 0 0

errors: No known data errors
[root@freenas ~]#

 

[danb35]

Ok, so you haven't messed up your redundancy. Good. So the question remains, what happened to that disk? Do you have a different SATA port you could plug it in to?

Don't worry about the "legacy on-disk format" for now, it won't cause a problem.

 

[Kory Fierstine]

I am maxed out on SATA ports (4). I thought about buying a SATA card, but compatibility is an issue I think (the only cards available locally are knock offs - IO, Vantec, Siig). I suppose it would be bad for me to swap SATA ports between drives?

 

[danb35]

Moving the disks around shouldn't hurt anything. To be on the safe side, though, go into the GUI and detach your existing volume (make sure you don't check the box to mark the disks as new), then power down and unplug the drives. Then try connecting the new drive to a different port, boot FreeNAS, create a volume, etc.

For controller cards, the most widely-recommended here seems to be the IBM M1015, which is available from eBay for around $100US and will give 8 SATA ports.

I doubt it's going to have an effect here, but what's the rest of your setup like? FreeNAS version, CPU, RAM (how much, is it ECC), etc.?

 

[Dennis K.]

Moving the disk around could hurt you, if the port where you tried to attach your new disk is damaged. If you now attach a drive from your exiting pool to that potential damaged port, you could lose that drive and would end up with a degraded pool.

 

[Kory Fierstine]

My system is an older Dell Vostro 220. Like I mentioned before, I know I "should" be running ECC and all the rest, but this started as the answer to requirement to store a lot of junk that I didn't want on my primary computer's SDD and it grew from there.

Build FreeNAS-9.2.0-RELEASE-x64 (ab098f4)
Platform Intel(R) Core(TM)2 Duo CPU E6550 @ 2.33GHz
Memory 3296MB (I actually have 4GB installed, but the BIOS maxed out at 3GB)
Non-ECC (system doesn't support it)
SATA 2, (4) ports on the Mainboard
4 x WD 2TB Green
Gigabit Ethernet
USB 2.0 <---Freenas boots from this

Dennis K., the BIOS sees all four HDDs, and Freenas sees all four too, although kicks the 4th out for some reason (didn't the first time I booted the system - only failed after I tried to expand the volume. I suppose a bad port would result in this current issue, but I would think a bad port would make it hard for the BIOS to see the drive.

I'll head down to the dungeon where I keep my NAS to swap SATA ports and see if the nasty beast shows me something different.

 

[Kory Fierstine]

Alright! It showed back up after swapping ports (in the following print out, it's ada0).

[root@freenas ~]# dmesg | grep -i ata
atapci0: <Intel ICH9 SATA300 controller> port 0xf700-0xf707,0xf600-0xf603,0xf500
-0xf507,0xf400-0xf403,0xf300-0xf30f,0xf200-0xf20f irq 19 at device 31.2 on pci0
ata2: <ATA channel> at channel 0 on atapci0
ata3: <ATA channel> at channel 1 on atapci0
atapci1: <Intel ICH9 SATA300 controller> port 0xf000-0xf007,0xef00-0xef03,0xee00
-0xee07,0xed00-0xed03,0xec00-0xec0f,0xeb00-0xeb0f irq 19 at device 31.5 on pci0
ata4: <ATA channel> at channel 0 on atapci1
ata5: <ATA channel> at channel 1 on atapci1
ada0 at ata2 bus 0 scbus0 target 0 lun 0
ada0: <WDC WD20EZRX-00D8PB0 80.00A80> ATA-9 SATA 3.x device
ada0: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada1 at ata3 bus 0 scbus1 target 0 lun 0
ada1: <WDC WD20EZRX-00D8PB0 80.00A80> ATA-9 SATA 3.x device
ada1: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada2 at ata4 bus 0 scbus2 target 0 lun 0
ada2: <WDC WD20EZRX-00D8PB0 80.00A80> ATA-9 SATA 3.x device
ada2: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
ada3 at ata5 bus 0 scbus3 target 0 lun 0
ada3: <WDC WD20EZRX-00D8PB0 80.00A80> ATA-9 SATA 3.x device
ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes)
[root@freenas ~]#

So, can I add this to my existing raid to increase my capacity or do I have to set this up as a separate disk?

 

[Kory Fierstine]

OK, been reading other posts about trying to add drives and, while I am struggling to understand the reason why adding single drives to a zpool is not an option, at least I am not so dull that I can't get the basic point - adding single drives to a zpool is not an option. Part of the issue is that with some solutions it is. I believe if I were independently wealthy and bought a synology box, I could add drives one by one and they get incorporated into the pool without any issue. This may be why noobs like me are a little dense when told that expanding our systems drive by drive is not an option (or I could be completely off kilter about the synology thing too).

 

[Dennis K.]

Your pool now looks like this:

Code:

fierpool1
    raidz1-0
        ada0
        ada1
        ada2

If you would add the new drive to your pool it would look like this:

Code:

fierpool1
    stripe0
        raidz1-0
            ada0
            ada1
            ada2
        ada3

Your new drive would be outside of your raidz1. If that new drive at ada3 failes at some point, your whole pool is gone.

 

[Kory Fierstine]

So, if I have the option, would it be better to nuke the entire thing and start over with a 4-drive raidz1 or 2?

 

[Dennis K.]

Thats definitely possible. Raidz1 or raidz2 is your choice to make. But if you choose raidz2 with 4 disks, you won't gain any space in comparison to your 3-disk raidz1 setup.

 

[gpsguy]

STOP STOP STOP

You need to address the (lack of) memory issue before you proceed. ZFS on FreeNAS needs a minimum of 8Gb of RAM. With just 3Gb you are gambling with your data.

 

[SweetAndLow]

STOP STOP STOP

You need to address the (lack of) memory issue before you proceed. ZFS on FreeNAS needs a minimum of 8Gb of RAM. With just 3Gb you are gambling with your data.

This is the reason you have unexplained behavior. You're hardware is not even close to the minimum required.