SMART model# mismatch?

[Ender117]

So I bought some SAS HDDs from eBay for dirt cheap, and first thing I did was to look at its SMART reading. Here is an example:

Code:

smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:			   HGST
Product:			  H7240B520SUN4.0T
Revision:			 M548
Compliance:		   SPC-4
User Capacity:		4,000,787,030,016 bytes [4.00 TB]
Logical block size:   512 bytes
Physical block size:  4096 bytes
Formatted with type 1 protection
LU is fully provisioned
Rotation Rate:		7200 rpm
Form Factor:		  3.5 inches
Logical Unit id:	  0x5000cca24400c050
Serial number:		001447M0DU1J		N8G0DU1J
Device type:		  disk
Transport protocol:   SAS (SPL-3)
Local Time is:		Mon Sep 10 21:28:09 2018 EDT
SMART support is:	 Available - device has SMART capability.
SMART support is:	 Enabled
Temperature Warning:  Enabled

=== START OF READ SMART DATA SECTION ===
SMART Health Status: OK

Current Drive Temperature:	 34 C
Drive Trip Temperature:		85 C

Manufactured in week 47 of year 2014
Specified cycle count over device lifetime:  50000
Accumulated start-stop cycles:  6
Specified load-unload count over device lifetime:  600000
Accumulated load-unload cycles:  8
Elements in grown defect list: 0

Vendor (Seagate) cache information
  Blocks sent to initiator = 37877920563200

Error counter log:
		   Errors Corrected by		   Total	 Correction	 Gigabytes	Total
			   ECC		  rereads/	errors	   algorithm	  processed	uncorrected
		   fast | delayed   rewrites  corrected  invocations   [10^9 bytes]  errors
read:		  0		4		 0		 4			1705			  1351.695		  0
write:		 0	   13		 0		13		   4652			 4508.933		   0
verify:		0		0		 0		 0			 2644			   0.000			  0

Non-medium error count:		0

SMART Self-test log
Num  Test				   Status				 segment  LifeTime  LBA_first_err [SK ASC ASQ]
	 Description										number   (hours)
# 1  Background long   Completed				   -	  54				 - [-   -	-]
# 2  Background short  Completed				   -	  45				 - [-   -	-]
# 3  Background short  Completed				   -	  43				 - [-   -	-]

Long (extended) Self Test duration: 34237 seconds [570.6 minutes]

Very different from SATA drive's reading. Manufactured years ago but with few hours, looks like new old stock. Got a few ECC corrected errors, but no unrecoverable ones (I believe consumer drives only report the latter?). All looks good to me.

What is bugging me is that the model# H7240B520SUN4.0T is different from the physical lable, which is HUS726040AL5210. Maybe this was made by HGST for SUN, but SUN was bought well before the drive was made. Also it feels wired to have different #s on the label and burned in.

What's your thoughts on these drives?

 

[Chris Moore]

What is bugging me is that the model# H7240B520SUN4.0T is different from the physical lable, which is HUS726040AL5210. Maybe this was made by HGST for SUN, but SUN was bought well before the drive was made. Also it feels wired to have different #s on the label and burned in.

I imagine that the reason for the number difference is that these drives were flashed with a custom firmware by Sun.
Were you able to create a pool and load any test data on the drives?

 

[rvassar]

I imagine that the reason for the number difference is that these drives were flashed with a custom firmware by Sun.

Sun was acquired in 2010... And there was something of a culture clash when Oracle acquired it... Labels like this result from entrenched processes, coupled with a little bit of bitter clinging, etc... Oracle wasn't a hardware company before this, so their playbook didn't cover crushing subtle firmware level branding like this.

 

[Chris Moore]

Sun was acquired in 2010... And there was something of a culture clash when Oracle acquired it... Labels like this result from entrenched processes, coupled with a little bit of bitter clinging, etc... Oracle wasn't a hardware company before this, so their playbook didn't cover crushing subtle firmware level branding like this.

Oracle may have bought Sun, but only two years ago, when they sent people out to install our new 'central site' SAN, where I work, they still talk about themselves as working for Sun, not Oracle, and Sun is listed first in the name on the side of the racks. It doesn't shock me at all that Sun is the only name listed in the firmware. The guy that is updating that code has probably been with the organization for 30 years and isn't about to change the way things are done. Just because.

 

[Ender117]

I imagine that the reason for the number difference is that these drives were flashed with a custom firmware by Sun.
Were you able to create a pool and load any test data on the drives?

that could be it. I have created a pool w/o problem using the wizard, but have not done any testing on that pool. Should I do that before testing each individual drives?

 

[Chris Moore]

that could be it. I have created a pool w/o problem using the wizard, but have not done any testing on that pool. Should I do that before testing each individual drives?

I would, just for testing, then you can go on to any burn-in that you like before getting back to setup of the storage for real.


Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[Chris Moore]

How many drives did you get?

Sent from my SAMSUNG-SGH-I537 using Tapatalk

 

[Ender117]

I would, just for testing, then you can go on to any burn-in that you like before getting back to setup of the storage for real.


Sent from my SAMSUNG-SGH-I537 using Tapatalk

How would you test a pool? ZFS is a whole new thing to me. I have 12 of these

 

[Ericloewe]

Sun brings the Bonwick Youth to mind. Oracle brings despair. I'm still convinced that Oracle is actually a social experiment to see how masochistic companies actually are.

 

[Chris Moore]

How would you test a pool? ZFS is a whole new thing to me. I have 12 of these

Well, you could start by telling us what hardware you have, including the drive controller, chassis, etc. Then, what do you plan to do with it? I like to set my pool up as two vdevs of six drives each with a redundancy factor of RAIDz2. To test it, I would copy some data into the pool and look at the iostat data to see what the pool performance was. Understand that this is just a little initial testing and this data will be obliterated during the drive burn-in.
How much do you know about the drive burn-in that is suggested on the forum?

 

[Ender117]

Well, you could start by telling us what hardware you have, including the drive controller, chassis, etc. Then, what do you plan to do with it? I like to set my pool up as two vdevs of six drives each with a redundancy factor of RAIDz2. To test it, I would copy some data into the pool and look at the iostat data to see what the pool performance was. Understand that this is just a little initial testing and this data will be obliterated during the drive burn-in.
How much do you know about the drive burn-in that is suggested on the forum?

OK, here is my current system

Dell R620 as head unit:
Dual E5 2690 v2
128GB RAM
H710P
LSI 9207 8e
Dell network card with 2 x540 2 i350

NetAPP DS4243 as DAS:
24 3.5 Bay
swapped in HB-SBB2-E601-COMP IO module because of price on minisas to qsfp cable

I know H710p as a hardware raid card should not be use with ZFS, but I am only use it for boot drive, and maybe L2ARC if I need it in the future. AFAIK you can loose these drives w/o loose any data on your main pool. If there is any gotcha please let me know.

Will be used for ESXi datastore and general filer (most of them should be media files), but unsure of details yet. I personally use badblocks for drive test, I have read people use dd for the test but that looks more like a performance benchmark to me.

 

[Chris Moore]

Dell R620 as head unit:

I don't like the skinny (1U / 2U) systems because there is no room to add much to them. I have a couple of 1U servers myself and even though there are technically two expansion slots in them, I can not fit anything useful in the second slot because it is not long enough. So they just have a 10Gb network card in the one slot and I use them for ESXi host systems. Similar problem with this chassis you have, there is almost no room in it to add things you will need.
For example:

Will be used for ESXi datastore

Is that going to be a NFS datastore? If yes, you will want a SLOG device to accelerate sync writes and the best SLOG is a PCIe card but you have no room for it, unless I am thinking of the wrong chassis. Personally, I like 4U servers because they usually have more room to add the controller cards required for options.
Please don't think I am criticizing your selections, but you may need to make some changes to get the desired performance out of your hardware.

H710P

Is that integrated on the system board or some kind of add-in card? It might be able to be pulled and a SLOG device might go there?

Also, what pool layout had you thought to use?

 

[Ender117]

I don't like the skinny (1U / 2U) systems because there is no room to add much to them. I have a couple of 1U servers myself and even though there are technically two expansion slots in them, I can not fit anything useful in the second slot because it is not long enough. So they just have a 10Gb network card in the one slot and I use them for ESXi host systems. Similar problem with this chassis you have, there is almost no room in it to add things you will need.
For example:

Is that going to be a NFS datastore? If yes, you will want a SLOG device to accelerate sync writes and the best SLOG is a PCIe card but you have no room for it, unless I am thinking of the wrong chassis. Personally, I like 4U servers because they usually have more room to add the controller cards required for options.
Please don't think I am criticizing your selections, but you may need to make some changes to get the desired performance out of your hardware.

Is that integrated on the system board or some kind of add-in card? It might be able to be pulled and a SLOG device might go there?

Also, what pool layout had you thought to use?

Yeah they are not very expandable but that's what I have. On the other hand they are not that bad, I got 3 low profile half length pcie slots. One of them was taken by 9207-8e another will be for a p3700 that's on the way. So at least I have the basics covered, I hope.

As for the H710p, it's a mini card that cannot take standard pcie cards. The best you can put in is a H310 mini that can do un-raid mode, but still not IT mode. Unfortunately you cannot crossflash these mini cards or you would brick them. My current plan is to use it for boot drives and potentially L2ARC, if run into problem will try a H310 mini.

Another interesting idea I plan to test was use the H710P + raid1 HDD as a SLOG. It's on-board cache should have very low latency and non-volatile, the backing HDD array have decent seq write performance and almost unlimited write endurance. But now since I got myself a P3700 this would just be a test.

 

[Ender117]

Now on pool layouts, since the differences in IOPS demand, I think I will be have a least 2 pools. I have think about:
1. 4x stripped mirror + 2x 10 disk raidz2
2. 8x stripped mirror + 2x 8 disk raidz2
Now the first option might be weak on raw IOPS. I have learnt that if you have plenty of free space ZFS will turn random write into seq write, so SLOG will become bottleneck for writes. And with my RAM I might be able to toss in a L2ARC as large as 1T. After these the apparent performance shouldn't be bad.

 

[Chris Moore]

another will be for a p3700 that's on the way.

You didn't tell me that. Sounds like you have it covered then.

 

[Chris Moore]

As for the H710p, it's a mini card that cannot take standard pcie cards.

Does it have a HBA mode where you can pass a drive through or must it be setup as a "container" or some thing like that?

My current plan is to use it for boot drives and potentially L2ARC, if run into problem will try a H310 mini.

For the way you plan to use it, it might not be a problem and you have a backup plan. It is true that if you have a backup of the config, loosing the boot volume is not a big issue. You can do a fresh install to new media, load the config DB and be back up in a matter of half an hour. If you have drives handy.

Another interesting idea I plan to test was use the H710P + raid1 HDD as a SLOG. It's on-board cache should have very low latency and non-volatile, the backing HDD array have decent seq write performance and almost unlimited write endurance. But now since I got myself a P3700 this would just be a test.

It might be an interesting test, but the P3700 should be much better.
Here are some interesting threads about that topic:

Testing the benefits of SLOG using a RAM disk!
https://forums.freenas.org/index.ph...s-of-slog-using-a-ram-disk.56561/#post-396630

Testing the benefits of SLOG
https://forums.freenas.org/index.php?threads/testing-the-benefits-of-slog-using-a-ram-disk.56561

SLOG benchmarking and finding the best SLOG
https://forums.freenas.org/index.ph...-and-finding-the-best-slog.63521/#post-454773

If you want to, you could contribute your test results to that last one.

8x stripped mirror + 2x 8 disk raidz2
Now the first option might be weak on raw IOPS. I have learnt that if you have plenty of free space ZFS will turn random write into seq write, so SLOG will become bottleneck for writes. And with my RAM I might be able to toss in a L2ARC as large as 1T. After these the apparent performance shouldn't be bad.

It sounds like you have a pretty decent plan going there.
You might want to take a look at these scripts for use with your system:

Github repository for FreeNAS scripts, including disk burnin
https://forums.freenas.org/index.ph...for-freenas-scripts-including-disk-burnin.28/

solnet-array-test (for drive / array speed) non destructive test
https://forums.freenas.org/index.php?resources/solnet-array-test.1/

 

[HoneyBadger]

Hey, I recognize you @Ender117 - sorry I never got back to you on your last post in your previous thread, got distracted and guess it slipped my mind. I'll review here:

I (and I think many others) will only find use in 2.5 drives for:
1. boot drive
2. L2ARC
3. SLOG

Now skipping the last one. Is there any tangible benefit to let FreeNAS to have raw access of the first 2?

Boot drive: potentially, you can do a mirrored ZFS boot environment which means scrubbing and guaranteed integrity of the actual system/boot volume.

L2ARC: Until persistent L2ARC becomes a thing, L2ARC is transient data and it being lost is pretty much only a performance impact. 1T is a bit big though, even with 128GB of RAM. You can always just add more, so start small and ramp up.

You covered the SLOG by opting for a P3700 (although did you look at trying to nab an Optane 900p card instead?) so we don't need to address #3 directly. Re: your idea of abusing the NVRAM on your H710p; it's been done in the past for testing, I haven't heard of anyone doing it recently since a "proper SLOG" in the form of NVMe devices are now commonly available. You'd want to set your RAID card to heavily favor caching writes.

Now that said, if the H710p has the same battery-test cycle as the full-size card, that will absolutely ruin your fun and performance, because it disables the cache entirely during this test. While that won't affect a boot drive or L2ARC that much, your SLOG volume will become a pair of spinning disks.

If you do this as a test, please post the results in the SLOG thread in my signature. Curious how well it works.

 

[Ender117]

You didn't tell me that. Sounds like you have it covered then.

Sorry about that:p, only included what is currently running.

Does it have a HBA mode where you can pass a drive through or must it be setup as a "container" or some thing like that?

For the way you plan to use it, it might not be a problem and you have a backup plan. It is true that if you have a backup of the config, loosing the boot volume is not a big issue. You can do a fresh install to new media, load the config DB and be back up in a matter of half an hour. If you have drives handy.

Unfortunately the H710p is a true raid card and the best you can do I believe is to put drives into single drive raid0. The slot can also take a H310 mini, which can do non-raid mode which reportedly will pass through SMART info and stuff, but still not a true HBA (IT mode). Given the limited improvement and my use case, I doubt there are any tangible benefit for switching to H310 mini. But like a said if I run into problems I will give it a whirl.

It might be an interesting test, but the P3700 should be much better.
Here are some interesting threads about that topic:

Testing the benefits of SLOG using a RAM disk!
https://forums.freenas.org/index.ph...s-of-slog-using-a-ram-disk.56561/#post-396630

Testing the benefits of SLOG
https://forums.freenas.org/index.php?threads/testing-the-benefits-of-slog-using-a-ram-disk.56561

SLOG benchmarking and finding the best SLOG
https://forums.freenas.org/index.ph...-and-finding-the-best-slog.63521/#post-454773

If you want to, you could contribute your test results to that last one.

It sounds like you have a pretty decent plan going there.
You might want to take a look at these scripts for use with your system:

Github repository for FreeNAS scripts, including disk burnin
https://forums.freenas.org/index.ph...for-freenas-scripts-including-disk-burnin.28/

solnet-array-test (for drive / array speed) non destructive test
https://forums.freenas.org/index.php?resources/solnet-array-test.1/

Thanks for the links I will give them a read. Just to confirm, do you recommend against "4x stripped mirror + 2x 10 disk raidz2" setup or just happened to not included it in the quote?


Now back to the original topic, do you think is it possible that these SUN firmware (assume they are) will give troubles with FreeNAS? If so, how? And how do I test against it? Maybe I can do a pool performance test but am unsure what to watch for.

 

[Ender117]

Hey, I recognize you @Ender117 - sorry I never got back to you on your last post in your previous thread, got distracted and guess it slipped my mind. I'll review here:

Hey HoneyBadger thanks for recognize me. Wish you are doing well.

Boot drive: potentially, you can do a mirrored ZFS boot environment which means scrubbing and guaranteed integrity of the actual system/boot volume.

Correct me if I am wrong but I believe ZFS can work on top of raid card presented LUN, you just loose SMART monitoring and other optimizations. If I do 2 single drive raid0 and mirror them in FreeNAS, I suppose redundancy and checksuming/scrubbing/integrity should still work? In the end I feel this is not much worse than dual USB boot stick setup.

L2ARC: Until persistent L2ARC becomes a thing, L2ARC is transient data and it being lost is pretty much only a performance impact. 1T is a bit big though, even with 128GB of RAM. You can always just add more, so start small and ramp up.

Sorry if I wasn't clear but I mentioned 1T as the upper limit. I for sure will start from a smaller SSDs and add more along the way, as they are still quite expensive in TB range:(. With that said, I still have 16 DIMM slots, so I can add more RAM if situation calls for it.

You covered the SLOG by opting for a P3700 (although did you look at trying to nab an Optane 900p card instead?) so we don't need to address #3 directly. Re: your idea of abusing the NVRAM on your H710p; it's been done in the past for testing, I haven't heard of anyone doing it recently since a "proper SLOG" in the form of NVMe devices are now commonly available. You'd want to set your RAID card to heavily favor caching writes.

I have been debating on P3700 vs 900p. I know that 900p is of even higher performance than p3700. What holds me back was its resistance to power losses. I know that 900p was initially listed by intel with PLP but now without, as well as it should persist data through power losses architecturally. However w/o official documentation and seen nobody ever tested this, I decided to error on the safe side because I have no idea how to test this either. Besides, I found a good deal on a used P3700;)

Now that said, if the H710p has the same battery-test cycle as the full-size card, that will absolutely ruin your fun and performance, because it disables the cache entirely during this test. While that won't affect a boot drive or L2ARC that much, your SLOG volume will become a pair of spinning disks.

Thank you, that is very useful information. Glad I got myself a proper SLOG

If you do this as a test, please post the results in the SLOG thread in my signature. Curious how well it works.

Yeah I am going to test it and record it, for science [read: fun]!

 

[Ericloewe]

Correct me if I am wrong but I believe ZFS can work on top of raid card presented LUN, you just loose SMART monitoring and other optimizations.

It's not that simple. For starters, SMART is essential to keeping things under control. There's also the problem of all the trickery, to put it charitably, that HW RAID controllers are known to do. You can end up with data corruption out of the blue because of that man in the middle screwing things up.