michaelrcolton
Cadet
- Joined
- Sep 29, 2017
- Messages
- 1
Hi everyone,
First let me say I really appreciate any help you can offer and I’ll provide any additional information you need. I apologize if this is posted in the wrong place or I’ve otherwise made any forum faux pax.
I've got a home file server that's been running FreeNAS for at least a year now without any serious problems. I decided to add some additional hard drives, which required the addition of a SATA PCI-Express card. Now, every zpool I try to create with these drives on this card appears to work, but is in an “unknown state” when I reboot.
The system information is as follows:
Running on FreeNAS-11-STABLE (I update whenever it tells me there is an update)
Motherboard is an Asus P5N-D
CPU is an Intel Core 2 Quad Q8400
It has 10GB of DDR3 (2x4GB and 2x1GB) non ECC, don’t be mad.
Up until now it had 4x2TB SATA drives connected to the motherboard SATA ports
2 are WD Enterprise Drives (Yellows?)
2 are WD Reds
The drives are set up as a pair of mirrors, one yellow and one red in each mirror and those two pairs are in a single zpool that acts as the main file storage for the house.
There is a single 160GB PATA drive that is used for storage on a security camera and is non-critical but works fine.
PSU is about 450-500 Watts, I forget. Seems to run fine even with 9 drives connected.
Boot drives are a mirrored pair of 16GB flash drives.
Other than the occasional hiccup, performance has been totally satisfactory.
I’m trying to add two more 2TB drives (HGSTs with TLER turned on). I also have an additional 2x2TB drives (WD Red Pros) that I am borrowing in case I need temporary storage while I am shuffling things around. I only have those for a few more days. They should also allow me to rule out the HGSTs being the problem.
So I added a Syba SY-PEX40008 (based on the SIL3124, which is supported by FreeBSD) I thought I had done my homework and found a card that was specifically known to work well with FreeNAS, subsequent searching makes me wonder if I actually got that right.
I have been making new, temporary zpools with these new drives (usually as mirrored or striped pairs, but I’ve also tried a RAIDZ2) to make sure everything is working before I make any changes to may main pool.
No matter what I’ve tried so far, they always up as “state unknown” after a reboot.
I have tried exporting and reimporting the test pools, they always fail, with a message to check zpool status, which reports something like an I/O error (I’ll need to do it again to get the actual message)
I have tried wiping the drives between some attempts. I also tried the full zero wipe once (using the command line, so I could wipe two drives at once (which, btw, crashed the web server, but I could still be managed by the console on the actual machine))
I have read that fake raid cards will overwrite the last sector or more of the drive, so I tried partitioning myself, with additional space at the end, and manually creating the zpool from the command line and then importing that into the FreeNAS GUI, it works up to the GUI import step, that fails.
I found what I believe to be the current firmware for the Syba card. If I am reading it right, there is a dumb firmware (just connects the sata ports to the computer) and a RAID firmware. Most of the tests were done with the raid firmware before I installed the update (that was version 6.3.18 of the raid firmware). I tried with the drives unconfigured in the BIOS (FreeNAS could see them just fine, but the reboot issues persisted) and in a “Concatenated” mode, which, I think, is their word for JBOD. This time the problem was a little different (I can’t remember how) but it still didn’t work.
Then I tried updating the card (thankfully it is of the vintage where this is possible, apparently newer cards can’t be updated) to the dumb firmware (6.6.00, the newest dumb firmware I have found, the raid firmware goes up to to 6.6.03). It no longer shows anything during boot up (the raid firmware has a BIOS setup screen) and FreeNAS still sees the drives. But the reboot problem persists.
I tried only using two of the ports on the Syba card, but the issues persists.
I am wary to connect the new drives to the motherboard ports because I don’t want to do anything to the main pool. If it’s safe to take that pool offline and hook in the new drives, I could at least eliminate the Syba card as an issue, or prove that it is one.
I have found other people that seems to be using the same SATA card (or at least other cards based on the same SIL3124) who aren’t complaining of issues.
I have found at least one person that had almost the exact same problem, but he wasn’t able to figure it out, he ended up buying a new motherboard. I would prefer not to do that unless I have to.
Once this starts working, I’m either going to RAIDZ2 or 3xMirror the 6 drives. I was leaning toward RAIDZ2, but this article ( http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/ ) makes me hesitate.
I have the extra 2TB drives as a place to store the data during the change (if it’s the mirror then I don’t need to do that, but if I switch to RAIDZ2, then I’ll have to transfer the data, then delete and reconfigure the zpool) I only have those extra 2tb drives for a little longer so I am trying to get this done. Also it would be nice to move on with my life. I have been fighting this for two weeks!
First let me say I really appreciate any help you can offer and I’ll provide any additional information you need. I apologize if this is posted in the wrong place or I’ve otherwise made any forum faux pax.
I've got a home file server that's been running FreeNAS for at least a year now without any serious problems. I decided to add some additional hard drives, which required the addition of a SATA PCI-Express card. Now, every zpool I try to create with these drives on this card appears to work, but is in an “unknown state” when I reboot.
The system information is as follows:
Running on FreeNAS-11-STABLE (I update whenever it tells me there is an update)
Motherboard is an Asus P5N-D
CPU is an Intel Core 2 Quad Q8400
It has 10GB of DDR3 (2x4GB and 2x1GB) non ECC, don’t be mad.
Up until now it had 4x2TB SATA drives connected to the motherboard SATA ports
2 are WD Enterprise Drives (Yellows?)
2 are WD Reds
The drives are set up as a pair of mirrors, one yellow and one red in each mirror and those two pairs are in a single zpool that acts as the main file storage for the house.
There is a single 160GB PATA drive that is used for storage on a security camera and is non-critical but works fine.
PSU is about 450-500 Watts, I forget. Seems to run fine even with 9 drives connected.
Boot drives are a mirrored pair of 16GB flash drives.
Other than the occasional hiccup, performance has been totally satisfactory.
I’m trying to add two more 2TB drives (HGSTs with TLER turned on). I also have an additional 2x2TB drives (WD Red Pros) that I am borrowing in case I need temporary storage while I am shuffling things around. I only have those for a few more days. They should also allow me to rule out the HGSTs being the problem.
So I added a Syba SY-PEX40008 (based on the SIL3124, which is supported by FreeBSD) I thought I had done my homework and found a card that was specifically known to work well with FreeNAS, subsequent searching makes me wonder if I actually got that right.
I have been making new, temporary zpools with these new drives (usually as mirrored or striped pairs, but I’ve also tried a RAIDZ2) to make sure everything is working before I make any changes to may main pool.
No matter what I’ve tried so far, they always up as “state unknown” after a reboot.
I have tried exporting and reimporting the test pools, they always fail, with a message to check zpool status, which reports something like an I/O error (I’ll need to do it again to get the actual message)
I have tried wiping the drives between some attempts. I also tried the full zero wipe once (using the command line, so I could wipe two drives at once (which, btw, crashed the web server, but I could still be managed by the console on the actual machine))
I have read that fake raid cards will overwrite the last sector or more of the drive, so I tried partitioning myself, with additional space at the end, and manually creating the zpool from the command line and then importing that into the FreeNAS GUI, it works up to the GUI import step, that fails.
I found what I believe to be the current firmware for the Syba card. If I am reading it right, there is a dumb firmware (just connects the sata ports to the computer) and a RAID firmware. Most of the tests were done with the raid firmware before I installed the update (that was version 6.3.18 of the raid firmware). I tried with the drives unconfigured in the BIOS (FreeNAS could see them just fine, but the reboot issues persisted) and in a “Concatenated” mode, which, I think, is their word for JBOD. This time the problem was a little different (I can’t remember how) but it still didn’t work.
Then I tried updating the card (thankfully it is of the vintage where this is possible, apparently newer cards can’t be updated) to the dumb firmware (6.6.00, the newest dumb firmware I have found, the raid firmware goes up to to 6.6.03). It no longer shows anything during boot up (the raid firmware has a BIOS setup screen) and FreeNAS still sees the drives. But the reboot problem persists.
I tried only using two of the ports on the Syba card, but the issues persists.
I am wary to connect the new drives to the motherboard ports because I don’t want to do anything to the main pool. If it’s safe to take that pool offline and hook in the new drives, I could at least eliminate the Syba card as an issue, or prove that it is one.
I have found other people that seems to be using the same SATA card (or at least other cards based on the same SIL3124) who aren’t complaining of issues.
I have found at least one person that had almost the exact same problem, but he wasn’t able to figure it out, he ended up buying a new motherboard. I would prefer not to do that unless I have to.
Once this starts working, I’m either going to RAIDZ2 or 3xMirror the 6 drives. I was leaning toward RAIDZ2, but this article ( http://jrs-s.net/2015/02/06/zfs-you-should-use-mirror-vdevs-not-raidz/ ) makes me hesitate.
I have the extra 2TB drives as a place to store the data during the change (if it’s the mirror then I don’t need to do that, but if I switch to RAIDZ2, then I’ll have to transfer the data, then delete and reconfigure the zpool) I only have those extra 2tb drives for a little longer so I am trying to get this done. Also it would be nice to move on with my life. I have been fighting this for two weeks!
Last edited: