Harddrive space not recognized correctly

Status
Not open for further replies.
M

mzsys

Cadet
Joined
Feb 27, 2017
Messages
6
So we are using FreeNAS as a data storage solution (two servers, one replicated to the other), but I've run into an issue that I haven't been able to find any solutions/ideas on how to fix. For some reason, the 10TB drives are being seen by FreeNAS 9.3 (as well as FreeBSD 11) as 1.2 TB drives. I booted into a live cd of Ubuntu, and it's able to see all of the drives at the proper size.

I have to use BIOS instead of UEFI due to some bug that should be resolved in the next major version of FreeNas (https://bugs.freenas.org/issues/18224), but even in UEFI mode, FreeBSD 11 shows 1.2TB.

Server Specs:
Dell PowerEdge R730xd
Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz (2400.04-MHz K8-class CPU)
256 GB Ram

PERC H730 Raid Controller (HBA mode)
2 200GB SSD (FreeNAS installed)
11 10TB HUH721010AL4200 Harddrives
PERC H830 Raid Controller (HBA mode)
11 10TB HUH721010AL4200 Harddrives

I checked with MegaCli, and they all show as 9TB (as expected), but when I list them in dmesg, they show as 1.2TB. The GUI lists them as 1.2TB as well.

Here's an excerpt from MegaCli
Code:
freenas# MegaCli -PDList -aAll

Adapter #0

Enclosure Device ID: 32
Slot Number: 0
Enclosure position: 1
Device Id: 0
WWN: 5000CCA25120816B
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS

Raw Size: 9.032 TB [0x90840000 Sectors]
Non Coerced Size: 9.031 TB [0x90820000 Sectors]
Coerced Size: 9.031 TB [0x90820000 Sectors]
Sector Size:  4096
Logical Sector Size:  4096
Physical Sector Size:  4096
Firmware state: JBOD
Device Firmware Level: LS03
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x5000cca251208169
SAS Address(1): 0x0
Connected Port Number: 0(path0)
Inquiry Data: HGST  HUH721010AL4200 LS037PGKW5PG
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 12.0Gb/s
Link Speed: 12.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :31C (87.80 F)
PI Eligibility:  No
Drive is formatted for PI information:  Yes
PI: PI with type 2
Port-0 :
Port status: Active
Port's Linkspeed: 12.0Gb/s
Port-1 :
Port status: Active
Port's Linkspeed: 12.0Gb/s
Drive has flagged a S.M.A.R.T alert : No



Enclosure Device ID: 32
Slot Number: 1
Enclosure position: 1
Device Id: 1
WWN: 5000CCA25120851B
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS

Raw Size: 9.032 TB [0x90840000 Sectors]
Non Coerced Size: 9.031 TB [0x90820000 Sectors]
Coerced Size: 9.031 TB [0x90820000 Sectors]
Sector Size:  4096
Logical Sector Size:  4096
Physical Sector Size:  4096
Firmware state: JBOD
Device Firmware Level: LS03
Shield Counter: 0
Successful diagnostics completion on :  N/A
SAS Address(0): 0x5000cca251208519
SAS Address(1): 0x0
Connected Port Number: 0(path0)
Inquiry Data: HGST  HUH721010AL4200 LS037PGKWE9G
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 12.0Gb/s
Link Speed: 12.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :32C (89.60 F)
PI Eligibility:  No
Drive is formatted for PI information:  Yes
PI: PI with type 2
Port-0 :
Port status: Active
Port's Linkspeed: 12.0Gb/s
Port-1 :
Port status: Active
Port's Linkspeed: 12.0Gb/s
Drive has flagged a S.M.A.R.T alert : No


Code:
freenas# dmesg

mfisyspd0: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 0)
mfisyspd1: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 1)
mfisyspd2: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 2)
mfisyspd3: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 3)
mfisyspd4: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 4)
mfisyspd5: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 5)
mfisyspd6: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 6)
mfisyspd7: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 7)
mfisyspd8: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 8)
mfisyspd9: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 9)
mfisyspd10: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 10)
mfisyspd11: 190782MB (390721968 sectors) SYSPD volume (deviceid: 12)
mfisyspd12: 190782MB (390721968 sectors) SYSPD volume (deviceid: 13)
mfisyspd13: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 1)
mfisyspd14: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 2)
mfisyspd15: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 3)
mfisyspd16: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 4)
mfisyspd17: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 5)
mfisyspd18: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 6)
mfisyspd19: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 7)
mfisyspd20: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 8)
mfisyspd21: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 9)
mfisyspd22: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 10)
mfisyspd23: 1183872MB (2424569856 sectors) SYSPD volume (deviceid: 11)


If there's anything missing that I should add, please let me know. I've tried googling this problem but I haven't been able to find anything relevant. Any help would be much appreciated.

Thanks!
 
M

mzsys

Cadet
Joined
Feb 27, 2017
Messages
6
Yes, they are in IT mode (no flash, they are able to switch between HBA/IT and RAID mode in the controller settings through the bios menus).
 
R

Robert Trevellyan

Pony Wrangler
Joined
May 16, 2014
Messages
3,778
mzsys said:
they are able to switch between HBA/IT and RAID mode in the controller settings through the bios menus
Click to expand...
Not sure if that really counts. Mostly the recommendation is to flash to IT mode, or if that's not an option, pick a different controller.
 
M

mzsys

Cadet
Joined
Feb 27, 2017
Messages
6
I went and took another look in a different way. My coworker reminded me that I had said something about smartd, as I had seen an error show up, but forgot as I've been trying to understand what's going on with this sizing issue. Turns out it's not passing the smartd (possibly other?) information through as /dev/mfisyspd##, which is what FreeNAS is seeing as disks. Looks like it's being passed through as /dev/pass##, which FreeNAS is not registering as disks for the GUI. I learned this from trying to find the smart status of the devices.

Code:
smartctl -i /dev/mfisyspd1
smartctl 6.5 2016-05-07 r4318 [FreeBSD 10.3-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

/dev/mfisyspd1: Unable to detect device type
Please specify device type with the -d option.

Use smartctl -h to get a usage summary


So I decided to do a smartd search, and looks like it presents the passed through devices as another device outside of /dev/mfisyspd##, as /dev/pass##:

Code:
smartctl --scan
/dev/pass0 -d scsi # /dev/pass0, SCSI device
/dev/pass1 -d scsi # /dev/pass1, SCSI device
/dev/pass2 -d scsi # /dev/pass2, SCSI device
/dev/pass3 -d scsi # /dev/pass3, SCSI device
/dev/pass4 -d scsi # /dev/pass4, SCSI device
/dev/pass5 -d scsi # /dev/pass5, SCSI device
/dev/pass6 -d scsi # /dev/pass6, SCSI device
/dev/pass7 -d scsi # /dev/pass7, SCSI device
/dev/pass8 -d scsi # /dev/pass8, SCSI device
/dev/pass9 -d scsi # /dev/pass9, SCSI device
/dev/pass10 -d scsi # /dev/pass10, SCSI device
/dev/pass11 -d scsi # /dev/pass11, SCSI device
/dev/pass12 -d scsi # /dev/pass12, SCSI device
/dev/ses0 -d scsi # /dev/ses0, SCSI device
/dev/ses1 -d scsi # /dev/ses1, SCSI device
/dev/pass15 -d scsi # /dev/pass15, SCSI device
/dev/pass16 -d scsi # /dev/pass16, SCSI device
/dev/pass17 -d scsi # /dev/pass17, SCSI device
/dev/pass18 -d scsi # /dev/pass18, SCSI device
/dev/pass19 -d scsi # /dev/pass19, SCSI device
/dev/pass20 -d scsi # /dev/pass20, SCSI device
/dev/pass21 -d scsi # /dev/pass21, SCSI device
/dev/pass22 -d scsi # /dev/pass22, SCSI device
/dev/pass23 -d scsi # /dev/pass23, SCSI device
/dev/pass24 -d scsi # /dev/pass24, SCSI device
/dev/pass25 -d scsi # /dev/pass25, SCSI device

smartctl -a /dev/pass0
smartctl 6.5 2016-05-07 r4318 [FreeBSD 10.3-STABLE amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:  HGST
Product:  HUH721010AL4200
Revision:  LS03
Compliance:  SPC-4
User Capacity:  9,931,038,130,176 bytes [9.93 TB]
Logical block size:  4096 bytes
Formatted with type 2 protection
LU is fully provisioned
Rotation Rate:  7200 rpm
Form Factor:  3.5 inches
Logical Unit id:  0x5000cca251208168
Serial number:  7PGKW5PG
Device type:  disk
Transport protocol:  SAS (SPL-3)
Local Time is:  Wed Mar  1 12:25:09 2017 CST
SMART support is:  Available - device has SMART capability.
SMART support is:  Enabled
Temperature Warning:  Disabled or Not Supported

=== START OF READ SMART DATA SECTION ===
SMART Health Status: OK

Current Drive Temperature:  29 C
Drive Trip Temperature:  50 C

Manufactured in week 35 of year 2016
Specified cycle count over device lifetime:  50000
Accumulated start-stop cycles:  214
Specified load-unload count over device lifetime:  600000
Accumulated load-unload cycles:  226
Elements in grown defect list: 0

Vendor (Seagate) cache information
  Blocks sent to initiator = 0

Error counter log:
  Errors Corrected by  Total  Correction  Gigabytes  Total
  ECC  rereads/  errors  algorithm  processed  uncorrected
  fast | delayed  rewrites  corrected  invocations  [10^9 bytes]  errors
read:  0  0  0  0  10  0.233  0
write:  0  0  0  0  0  0.000  0
verify:  0  0  0  0  1286  0.835  0

Non-medium error count:  0

SMART Self-test log
Num  Test  Status  segment  LifeTime  LBA_first_err [SK ASC ASQ]
  Description  number  (hours)
# 1  Background short  Completed  96  12  - [-  -  -]
# 2  Reserved(7)  Completed  64  12  - [-  -  -]

Long (extended) Self Test duration: 63831 seconds [1063.8 minutes]


I'll dig in this direction then... Any chance you've seen something along similar lines?

Thanks again. I'm relatively new to FreeNAS and FreeBSD so I've been playing catchup trying to see how to debug things.
 
M

mzsys

Cadet
Joined
Feb 27, 2017
Messages
6
Ok, Thanks for the new direction to look into. Looks like the issue is that FreeNAS uses the MFI driver as a default, and doesn't use the MRSAS drivers after installing and configuration. Thanks to new google terms, I was able to find a similar thread:

https://forums.freenas.org/index.ph...dell-servers-with-perc-h730-controller.46631/

This showed that to be able to use it properly, you need to use the hw.mri.mrsas_enable="1" hint in the /boot/device.hints

Hopefully this helps someone in the future if they run into this and are having an issue trying to debug this.

Thanks again for the help Robert.
 
Status
Not open for further replies.

Similar threads

X
  • Locked
Chelsio T580-LP-CR
Replies
10
Views
4K
xlameee
X
E
Issue Upgrading to Larger Disks
Replies
1
Views
1K
morganL
M
H
Don't see drive status LED after connecting it HP H220 HBA
2
Replies
21
Views
6K
vampirebyte
vampirebyte
S
Strange Bay Mapping with 9211-8i
Replies
1
Views
770
jgreco
jgreco
S
Pool Created from CLI, RMA replaced via GUI, extra 2G Linux swap created on replacement drive, the larger partition is smaller than other pool members
Replies
3
Views
873
Sgt_Bizkit
S
Top