Dealing with sub-optimal configuration

[mwheeler1982]

Dealing with sub-optimal configuration - Unexpected Sense Code

I need to set up a temporary FreeNAS box to support a migration between 2 other NAS boxes. Hopefully, if FreeNAS works well in this temporary situation, I can push to use it in production with better hardware, but that's down the road.

I have a Dell 2950 with a PERC 5/i card and 6x1.5TB SATA drives. This card cannot do JBOD mode. Instead of configuring the card with 6xRAID0 with single drives, I opted for a RAID50 and just use FreeNAS to export the storage (since this is just temporary). However, FreeNAS seems to think that something is wrong and there's a constant rebuild in progress.

The drive sled lights on the front of the server are all green, so I don't think there's an actual drive failure. Is FreeNAS just passing through what it's getting from the PERC card, or is it misreading what's going on under the covers?

Here's what I see in the logs:
May 1 13:07:56 freenas kernel: mfi0: 27978 (389210797s/0x0002/WARN) - Predictive failure: PD 02(e1/s2)
May 1 13:07:56 freenas kernel: mfi0: 27979 (389210797s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 03 00 00 00 40 00, Sense: f0 00 00 00 00 00 00 0a 00 00 00 00 5d 10 00 00 00 00
May 1 15:59:17 freenas kernel: mfi0: 28050 (389221052s/0x0002/WARN) - Removed: PD 02(e1/s2)
May 1 15:59:17 freenas kernel: mfi0: 28051 (389221052s/0x0002/info) - Removed: PD 02(e1/s2) Info: enclPd=08, scsiType=0, portMap=04, sasAddr=1221000002000000,0000000000000000
May 1 15:59:17 freenas kernel: mfi0: 28052 (389221052s/0x0002/info) - State change on PD 02(e1/s2) from REBUILD(14) to FAILED(11)
May 1 15:59:17 freenas kernel: mfi0: 28053 (389221052s/0x0002info) - State change on P 02(e1/s) from FAILED(1) to UNCONFIGURD_BAD(1)
May 1 15:59:17 freenas kernel:
May 1 15:59:17 freenas kernel: mfi0: 28054 (389221052s/0x0002/CRIT) - Rebuild failed on PD 02(e1/s2) due to target drive error
May 1 15:59:32 freenas kernel: mfi0: 28055 (389221067s/0x0002/info) - Inserted: PD 02(e1/s2)
May 1 15:59:33 freenas kernel: mfi0: 28056 (389221067s/0x0002/info) - Inserted: PD 02(e1/s2) Info: enclPd=08, scsiType=0, portMap=04, sasAddr=1221000002000000,0000000000000000
May 1 15:59:33 freenas kernel: mfi0: 28057 (389221067s/0x0002/WARN) - PD 02(e1/s2) is not a certified drive
May 1 15:59:33 freenas kernel: mfi0: 28058 (389221067s/0x0002/info) - State change on PD 02(e1/s2) from UNCONFIGURED_BAD(1) to UNCONFIGURED_GOOD(0)
May 1 15:59:33 freenas kernel: mfi0: 28059 (389221067s/0x0002/info) - State change on PD 02(e1/s2) from UNCONFIGURED_GOOD(0) to OFFLINE(10)
May 1 15:59:33 freenas kernel: mfi0: 28060 (389221068s/0x0002/info) - Rebuild automatically started on PD 02(e1/s2)
May 1 15:59:33 freenas kernel: mfi0: 28061 (389221068s/0x0002/info) - State change on PD 02(e1/s2) from OFFLINE(10) to REBUILD(14)
May 1 18:54:07 freenas kernel: mfi0: 28135 (389231516s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d f3 d9 00 00 01 00 00, Sense: 70 00 06 00 00 00 00 0a 00 00 00 00 29 00 00 00 00 0
May 1 18:54:08 freenas kernel: mfi0: 28136 (389231516s/0x0002/WARN) - Predictive failure: PD 02(e1/s2)
May 1 18:54:08 freenas kernel: mfi0: 28137 (389231516s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 03 00 00 00 40 00, Sense: f0 00 00 00 00 00 00 0a 00 00 00 00 5d 10 00 00 00 00
May 1 18:54:08 freenas kernel: mfi0: 28138 (389231516s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 15 11 00 00 10 00, Sense: 70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00 00 00
May 1 18:54:08 freenas kernel: mfi0: 28139 (389231516s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 15 11 00 00 10 00, Sense: 70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00 00 00
May 1 18:54:51 freenas kernel: mfi0: 28140 (389231560s/0x0002/WARN) - Removed: PD 02(e1/s2)
May 1 18:54:51 freenas kernel: mfi0: 28141 (389231560s/0x0002/info) - Removed: PD 02(e1/s2) Info: enclPd=08, scsiType=0, portMap=04, sasAddr=1221000002000000,0000000000000000
May 1 18:54:51 freenas kernel: mfi0: 28142 (389231560s/0x0002/info) - State change on PD 02(e1/s2) from REBUILD(14) to FAILED(11)
May 1 18:54:51 freenas kernel: mfi0:
May 1 18:54:51 freenas kernel: 28143 (389231560s/0x0002/info) - State change on PD 02(e1/s2) from FAILED(11) to UNCGURED_BAD(1)
May 1 18:54:52 freenas kernel:
May 1 18:54:52 freenas kernel: mfi0: 28144 (389231560s/0x0002/CRIT) - Rebuild failed on PD 02(e1/s2) due to target drive error
May 1 18:55:02 freenas kernel: mfi0: 28145 (389231571s/0x0002/info) - Inserted: PD 02(e1/s2)
May 1 18:55:03 freenas kernel: mfi0: 28146 (389231571s/0x0002/info) - Inserted: PD 02(e1/s2) Info: enclPd=08, scsiType=0, portMap=04, sasAddr=1221000002000000,0000000000000000
May 1 18:55:03 freenas kernel: mfi0: 28147 (389231571s/0x0002/WARN) - PD 02(e1/s2) is not a certified drive
May 1 18:55:03 freenas kernel: mfi0: 28148 (389231571s/0x0002/info) - State change on PD 02(e1/s2) from UNCONFIGURED_BAD(1) to UNCONFIGURED_GOOD(0)
May 1 18:55:03 freenas kernel: mfi0: 28149 (389231571s/0x0002/info) - State change on PD 02(e1/s2) from UNCONFIGURED_GOOD(0) to OFFLINE(10)
May 1 18:55:03 freenas kernel: mfi0: 28150 (389231571s/0x0002/info) - Rebuild automatically started on PD 02(e1/s2)
May 1 18:55:03 freenas kernel: mfi0: 28151 (389231571s/0x0002/info) - State change on PD 02(e1/s2) from OFFLINE(10) to REBUILD(14)
May 1 21:48:14 freenas kernel: mfi0: 28225 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d4 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 44 00 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28226 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d5 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28227 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d5 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28228 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d6 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28229 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d4 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28230 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d7 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28231 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d7 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28232 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d8 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28233 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d4 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28234 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d9 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28235 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d6 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28236 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 da 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28237 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d5 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28238 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 23 fe 34 82 00 00 01 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28239 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 23 fe 34 82 00 00 01 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28240 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d5 00 00 01 00 00, Sense: 70 00 04 00 00 00
May 1 21:48:15 freenas kernel: 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel:
May 1 21:48:15 freenas kernel: mfi0: 28241 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 da 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28242 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d6 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28243 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d9 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28244 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d4 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28245 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d8 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28246 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d7 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28247 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d7 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28248 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d8 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28249 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d4 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28250 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d9 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28251 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d6 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28252 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 da 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28253 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d5 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28254 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 23 fe 34 82 00 00 01 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28255 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 23 fe 34 82 00 00 01 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28256 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d5 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28257 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 da 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28258 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d6 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28259 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d9 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28260 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d4 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28261 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d8 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28262 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d7 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28263 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d8 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28264 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d9 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28265 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d6 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28266 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 da 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28267 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 23 fe 34 82 00 00 01 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28268 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 23 fe 34 82 00 00 01 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28269 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 da 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28270 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d9 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28271 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d8 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28272 (389241937s/0x0002/info) - Unexpected sense: PD 02(e1/s2), CDB: 2a 00 7d b9 d7 00 00 01 00 00, Sense: 70 00 04 00 00 00 00 0a 00 00 00 00 3e 01 00 00 00 0
May 1 21:48:15 freenas kernel: mfi0: 28273 (389241937s/0x0002/WARN) - Error on PD 02(e1/s2) (Error 02)
May 1 21:48:15 freenas kernel: mfi0: 28274 (389241937s/0x0002/info) - State change on PD 02(e1/s2) from REBUILD(14) to FAILED(11)
May 1 21:48:15 freenas kernel: mfi0: 28275 (389241938s/0x0002/CRIT) - Rebuild failed on PD 02(e1/s2) due to target drive error

 

[cyberjock]

From personal experience with using a RAID controller's RAID functions for FreeNAS, you can expect to have some issues(some borderline show stoppers) using RAID with FreeNAS. The manual says "don't do RAID with FreeNAS" and the short and dirty answer is "do what the manual says and don't try to outsmart the manual".

In my configuration, a hard drive was simulated failed by disconnecting power to the drive. FreeNAS was completely unaware of the "failed" drive and simply kept on going along working. No warnings or anything. Of course, as I started to read/write a large number of errors accumulated with the device. The only way FreeNAS would ultimately acknowledge that the pool was not healthy was to reboot the system and let FreeNAS figure out that one device was actually missing. Not the optimal configuration at all. Basically this means that I MUST monitor for errors in the devices on the pool to ensure the pool is healthy.