Resource icon

SLOG benchmarking and finding the best SLOG

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,087
Intel S3700 100GB
Intel S3700 (100G ==> 8G):

Code:
Sector Size:	  512 bytes logical/physical

Random thought - I was graphing out the results here, and I noticed that both of you have your DC S3700 set to 512 byte physical sectors rather than 4K. (Bonus edit: Spearfoot's S3500 120GB is also 512b physical.) Given that you'll virtually never have a pool with an ashift value under 12 these days, why not set the drive to use the bigger sector size? You can change that with the ISDCT tool under Windows, provided your firmware is new enough.

Intel KB article: https://www.intel.com/content/www/us/en/support/articles/000006392/memory-and-storage.html
 
Last edited:

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,087
Speaking of graphing this out, I'm seeing a few oddities.

We might want to include more hardware specs in the posts, such as SATA/SAS controller used, whether it's behind an expander or in an external JBOD, or any other bottlenecks

@Spearfoot 's S3500 240GB gets dusted by @Elliot Dierksen 's S3500 120GB at low block sizes. Generally speaking, that shouldn't happen - the bigger drive with more NAND should be faster.

@Chris Moore - that HGST SLC drive might put down huge numbers in bigger blocks, but up to recordsize=16K it's actually slower than my old Intel 320 80GB according to this benchmark.
 

tazinblack

Explorer
Joined
Apr 12, 2013
Messages
77
my 900p benchmarks:

Code:
smartctl -a /dev/nvme0
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:					   INTEL SSDPED1D280GA
Serial Number:					  PHMB739200C7280CGN
Firmware Version:				   E2010325
PCI Vendor/Subsystem ID:			0x8086
IEEE OUI Identifier:				0x5cd2e4
Controller ID:					  0
Number of Namespaces:			   1
Namespace 1 Size/Capacity:		  280,065,171,456 [280 GB]
Namespace 1 Formatted LBA Size:	 512
Local Time is:					  Tue Aug 14 16:26:34 2018 CEST
Firmware Updates (0x02):			1 Slot
Optional Admin Commands (0x0007):   Security Format Frmw_DL
Optional NVM Commands (0x0006):	 Wr_Unc DS_Mngmt
Maximum Data Transfer Size:		 32 Pages

Supported Power States
St Op	 Max   Active	 Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +	18.00W	   -		-	0  0  0  0		0	   0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +	 512	   0		 2

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff)
Critical Warning:				   0x00
Temperature:						47 Celsius
Available Spare:					100%
Available Spare Threshold:		  0%
Percentage Used:					0%
Data Units Read:					44 [22.5 MB]
Data Units Written:				 134,530 [68.8 GB]
Host Read Commands:				 538
Host Write Commands:				2,162,750
Controller Busy Time:			   0
Power Cycles:					   14
Power On Hours:					 26
Unsafe Shutdowns:				   0
Media and Data Integrity Errors:	0
Error Information Log Entries:	  0

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged



Code:
diskinfo -wS /dev/nvd0
/dev/nvd0
		512			 # sectorsize
		280065171456	# mediasize in bytes (261G)
		547002288	   # mediasize in sectors
		0			   # stripesize
		0			   # stripeoffset
		INTEL SSDPED1D280GA	 # Disk descr.
		PHMB739200C7280CGN	  # Disk ident.

Synchronous random writes:
		 0.5 kbytes:	 15.6 usec/IO =	 31.4 Mbytes/s
		   1 kbytes:	 15.4 usec/IO =	 63.6 Mbytes/s
		   2 kbytes:	 15.2 usec/IO =	128.6 Mbytes/s
		   4 kbytes:	 12.5 usec/IO =	311.5 Mbytes/s
		   8 kbytes:	 14.4 usec/IO =	540.9 Mbytes/s
		  16 kbytes:	 19.6 usec/IO =	798.4 Mbytes/s
		  32 kbytes:	 29.6 usec/IO =   1054.9 Mbytes/s
		  64 kbytes:	 48.4 usec/IO =   1292.1 Mbytes/s
		 128 kbytes:	 87.1 usec/IO =   1435.9 Mbytes/s
		 256 kbytes:	166.8 usec/IO =   1498.5 Mbytes/s
		 512 kbytes:	322.2 usec/IO =   1551.7 Mbytes/s
		1024 kbytes:	625.4 usec/IO =   1599.0 Mbytes/s
		2048 kbytes:   1225.5 usec/IO =   1632.0 Mbytes/s
		4096 kbytes:   2424.6 usec/IO =   1649.8 Mbytes/s
		8192 kbytes:   4748.2 usec/IO =   1684.9 Mbytes/s


This is bare metal without hypervisor. Doesn't look as impressive as I saw in this thread above with pass through, but where is the handbrake on?
 

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,087
This is bare metal without hypervisor. Doesn't look as impressive as I saw in this thread above with pass through, but where is the handbrake on?

You're running out of steam at the high end, but your small-block is actually the highest of the benchmarks so far.

upload_2018-8-14_11-17-22.png


You're in the blue marked with a T, you're leading the pack until you hit the really large (512K+) record sizes

What's the hardware platform? PCIe version, slot width, are you tied to the correct processor if it's an MP system?
 

tazinblack

Explorer
Joined
Apr 12, 2013
Messages
77
You're running out of steam at the high end, but your small-block is actually the highest of the benchmarks so far.

...

You're in the blue marked with a T, you're leading the pack until you hit the really large (512K+) record sizes

What's the hardware platform? PCIe version, slot width, are you tied to the correct processor if it's an MP system?

Good questions. It's a supermicro box with X10SRi-F board with a xeon-e5-1620v4 CPU.
The hardware is described here https://forums.freenas.org/index.php?threads/24-bay-freenas.62137/page-2
Let me take a look inside. Will take some time...
 

tazinblack

Explorer
Joined
Apr 12, 2013
Messages
77
ok, first thing I tried it to fiddle around with some bios values in the pcie section.
Result:
Code:
diskinfo -wS /dev/nvd0
/dev/nvd0
		512			 # sectorsize
		280065171456	# mediasize in bytes (261G)
		547002288	   # mediasize in sectors
		0			   # stripesize
		0			   # stripeoffset
		INTEL SSDPED1D280GA	 # Disk descr.
		PHMB739200C7280CGN	  # Disk ident.

Synchronous random writes:
		 0.5 kbytes:	 18.6 usec/IO =	 26.2 Mbytes/s
		   1 kbytes:	 15.9 usec/IO =	 61.6 Mbytes/s
		   2 kbytes:	 15.3 usec/IO =	127.7 Mbytes/s
		   4 kbytes:	 12.5 usec/IO =	313.6 Mbytes/s
		   8 kbytes:	 14.5 usec/IO =	540.2 Mbytes/s
		  16 kbytes:	 19.3 usec/IO =	811.2 Mbytes/s
		  32 kbytes:	 27.7 usec/IO =   1128.5 Mbytes/s
		  64 kbytes:	 46.1 usec/IO =   1356.2 Mbytes/s
		 128 kbytes:	 80.4 usec/IO =   1554.3 Mbytes/s
		 256 kbytes:	142.3 usec/IO =   1757.0 Mbytes/s
		 512 kbytes:	266.7 usec/IO =   1874.8 Mbytes/s
		1024 kbytes:	534.5 usec/IO =   1870.8 Mbytes/s
		2048 kbytes:   1068.8 usec/IO =   1871.2 Mbytes/s
		4096 kbytes:   2115.0 usec/IO =   1891.3 Mbytes/s
		8192 kbytes:   4182.0 usec/IO =   1913.0 Mbytes/s


So better at high block area now but worst in the small block area.
So would be interesting to know which is the blocksize we try to acquire the best values for.:confused:

A second run brings some different values:

Code:
diskinfo -wS /dev/nvd0
/dev/nvd0
		512			 # sectorsize
		280065171456	# mediasize in bytes (261G)
		547002288	   # mediasize in sectors
		0			   # stripesize
		0			   # stripeoffset
		INTEL SSDPED1D280GA	 # Disk descr.
		PHMB739200C7280CGN	  # Disk ident.

Synchronous random writes:
		 0.5 kbytes:	 18.5 usec/IO =	 26.4 Mbytes/s
		   1 kbytes:	 15.2 usec/IO =	 64.4 Mbytes/s
		   2 kbytes:	 14.8 usec/IO =	131.9 Mbytes/s
		   4 kbytes:	 12.4 usec/IO =	315.0 Mbytes/s
		   8 kbytes:	 14.3 usec/IO =	545.9 Mbytes/s
		  16 kbytes:	 19.0 usec/IO =	824.1 Mbytes/s
		  32 kbytes:	 27.2 usec/IO =   1150.2 Mbytes/s
		  64 kbytes:	 45.2 usec/IO =   1383.6 Mbytes/s
		 128 kbytes:	 79.1 usec/IO =   1579.6 Mbytes/s
		 256 kbytes:	145.3 usec/IO =   1721.0 Mbytes/s
		 512 kbytes:	277.9 usec/IO =   1799.1 Mbytes/s
		1024 kbytes:	547.8 usec/IO =   1825.3 Mbytes/s
		2048 kbytes:   1069.4 usec/IO =   1870.2 Mbytes/s
		4096 kbytes:   2107.7 usec/IO =   1897.8 Mbytes/s
		8192 kbytes:   4173.2 usec/IO =   1917.0 Mbytes/s


So maybe we are moving inside a normal tollerance.
 

tazinblack

Explorer
Joined
Apr 12, 2013
Messages
77
What's the hardware platform? PCIe version, slot width, are you tied to the correct processor if it's an MP system?

It's connected to PCIe 3.0 X8 Slot and the system has only one processor socket. According to the datasheet this should be more than sufficient (3.0 X4)
 

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,087
Looks like whatever bits you twiddled in the BIOS helped, especially for bigger records. Do you remember what you adjusted for future users?

So would be interesting to know which is the blocksize we try to acquire the best values for.:confused:

Most users who add an SLOG are trying to improve performance of VMs or DBs, which tend to use a small recordsize (4K, 8K, 16K) - there are some cases where you want or get sync writes at larger recordsizes (your NFS storage might be one) but generally speaking SLOGs tend to be put into play when the small-block write can't keep up.
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
Samba itself has no sync write function, so you would have to handle that on the ZFS layer.
I was just reading something yesterday on the forum that said a new version of SMB was being used that does support sync writes. Is that a change to this?
 

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,087
I was just reading something yesterday on the forum that said a new version of SMB was being used that does support sync writes. Is that a change to this?

FreeNAS 11.1 merged Samba 4.7.3 which changed the default to strict sync = yes

https://forums.freenas.org/index.php?threads/smb-share-slow-and-capped-at-30mbps.63748/#post-456938

Strict sync is discussed here:
https://www.samba.org/samba/docs/current/man-html/smb.conf.5.html

however

Strict sync will only perform a sync write if requested by the client; Windows generally doesn't request sync writes but there seems to be a number of OSX users reporting that it will request them. This is opposed to setting sync=always on the dataset where ZFS will treat everything as a sync write even if the client requests async.
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
The command to test:
We got a new server at work with a couple of Optane drives.

The first is: Intel Optane DC P4800X Series 375 GB - SSDPE21K375GA
Like this: 20180912_121213.jpg
Code:
root@freenas:/dev # smartctl -a /dev/nvme0
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:					   INTEL SSDPE21K375GA
Serial Number:					  PHKE7495005G375AGN
Firmware Version:				   E2010423
PCI Vendor/Subsystem ID:			0x8086
IEEE OUI Identifier:				0x5cd2e4
Controller ID:					  0
Number of Namespaces:			   1
Namespace 1 Size/Capacity:		  375,083,606,016 [375 GB]
Namespace 1 Formatted LBA Size:	 512
Namespace 1 IEEE EUI-64:			5cd2e4 8b99040100
Local Time is:					  Wed Sep 12 13:55:39 2018 CDT
Firmware Updates (0x02):			1 Slot
Optional Admin Commands (0x0007):   Security Format Frmw_DL
Optional NVM Commands (0x0006):	 Wr_Unc DS_Mngmt
Maximum Data Transfer Size:		 32 Pages

Supported Power States
St Op	 Max   Active	 Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +	18.00W	   -		-	0  0  0  0		0	   0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +	 512	   0		 2
 1 -	 512	   8		 2
 2 -	 512	  16		 2
 3 -	4096	   0		 0
 4 -	4096	   8		 0
 5 -	4096	  64		 0
 6 -	4096	 128		 0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff)
Critical Warning:				   0x00
Temperature:						31 Celsius
Available Spare:					100%
Available Spare Threshold:		  0%
Percentage Used:					0%
Data Units Read:					2,188,477 [1.12 TB]
Data Units Written:				 2,191,008 [1.12 TB]
Host Read Commands:				 269,590,135
Host Write Commands:				270,035,281
Controller Busy Time:			   44
Power Cycles:					   24
Power On Hours:					 76
Unsafe Shutdowns:				   15
Media and Data Integrity Errors:	0
Error Information Log Entries:	  0

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged

Code:
root@freenas:~ # diskinfo -wS /dev/nvd0
/dev/nvd0
   512			 # sectorsize
   375083606016   # mediasize in bytes (349G)
   732585168	   # mediasize in sectors
   0			   # stripesize
   0			   # stripeoffset
   INTEL SSDPE21K375GA   # Disk descr.
   PHKE7495005G375AGN   # Disk ident.

Synchronous random writes:
	0.5 kbytes:	 15.1 usec/IO =	 32.4 Mbytes/s
	  1 kbytes:	 15.5 usec/IO =	 62.8 Mbytes/s
	  2 kbytes:	 15.7 usec/IO =	124.3 Mbytes/s
	  4 kbytes:	 12.8 usec/IO =	305.5 Mbytes/s
	  8 kbytes:	 14.9 usec/IO =	523.2 Mbytes/s
	 16 kbytes:	 20.0 usec/IO =	782.2 Mbytes/s
	 32 kbytes:	 29.7 usec/IO =   1053.0 Mbytes/s
	 64 kbytes:	 49.4 usec/IO =   1266.3 Mbytes/s
	128 kbytes:	 85.5 usec/IO =   1461.2 Mbytes/s
	256 kbytes:	152.7 usec/IO =   1637.2 Mbytes/s
	512 kbytes:	287.0 usec/IO =   1742.0 Mbytes/s
   1024 kbytes:	548.0 usec/IO =   1824.7 Mbytes/s
   2048 kbytes:   1069.4 usec/IO =   1870.3 Mbytes/s
   4096 kbytes:   2117.8 usec/IO =   1888.8 Mbytes/s
   8192 kbytes:   4240.6 usec/IO =   1886.5 Mbytes/s
root@freenas:~ #

The second one is: Intel DC P4600 2TB NVMe PCIe3.0x4 3D TLC SSD - SSDPEDKE020T7
Like this: 20180912_121225.jpg
Code:
root@freenas:/dev # smartctl -a /dev/nvme1
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:					   INTEL SSDPEDKE020T7
Serial Number:					  PHLE746200WK2P0IGN
Firmware Version:				   QDV10190
PCI Vendor/Subsystem ID:			0x8086
IEEE OUI Identifier:				0x5cd2e4
Total NVM Capacity:				 2,000,398,934,016 [2.00 TB]
Unallocated NVM Capacity:		   0
Controller ID:					  0
Number of Namespaces:			   1
Namespace 1 Size/Capacity:		  2,000,398,934,016 [2.00 TB]
Namespace 1 Formatted LBA Size:	 512
Namespace 1 IEEE EUI-64:			5cd2e4 58003d0100
Local Time is:					  Wed Sep 12 13:55:48 2018 CDT
Firmware Updates (0x02):			1 Slot
Optional Admin Commands (0x0006):   Format Frmw_DL
Optional NVM Commands (0x0006):	 Wr_Unc DS_Mngmt
Maximum Data Transfer Size:		 32 Pages
Warning  Comp. Temp. Threshold:	 70 Celsius
Critical Comp. Temp. Threshold:	 77 Celsius

Supported Power States
St Op	 Max   Active	 Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +	25.00W	   -		-	0  0  0  0		0	   0

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +	 512	   0		 2
 1 -	4096	   0		 0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff)
Critical Warning:				   0x00
Temperature:						13 Celsius
Available Spare:					100%
Available Spare Threshold:		  10%
Percentage Used:					0%
Data Units Read:					1,557,745 [797 GB]
Data Units Written:				 1,574,602 [806 GB]
Host Read Commands:				 192,450,353
Host Write Commands:				193,015,734
Controller Busy Time:			   9
Power Cycles:					   23
Power On Hours:					 76
Unsafe Shutdowns:				   15
Media and Data Integrity Errors:	0
Error Information Log Entries:	  0
Warning  Comp. Temperature Time:	0
Critical Comp. Temperature Time:	0

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged

Code:
root@freenas:~ #diskinfo -wS /dev/nvd1
/dev/nvd1
   512			 # sectorsize
   2000398934016   # mediasize in bytes (1.8T)
   3907029168	  # mediasize in sectors
   0			   # stripesize
   0			   # stripeoffset
   INTEL SSDPEDKE020T7   # Disk descr.
   PHLE746200WK2P0IGN   # Disk ident.

Synchronous random writes:
	0.5 kbytes:	 15.5 usec/IO =	 31.5 Mbytes/s
	  1 kbytes:	 13.2 usec/IO =	 74.0 Mbytes/s
	  2 kbytes:	 11.7 usec/IO =	166.9 Mbytes/s
	  4 kbytes:	 10.6 usec/IO =	370.3 Mbytes/s
	  8 kbytes:	 12.5 usec/IO =	626.7 Mbytes/s
	 16 kbytes:	 16.7 usec/IO =	936.7 Mbytes/s
	 32 kbytes:	 25.8 usec/IO =   1209.8 Mbytes/s
	 64 kbytes:	 44.4 usec/IO =   1407.3 Mbytes/s
	128 kbytes:	 86.7 usec/IO =   1442.1 Mbytes/s
	256 kbytes:	165.0 usec/IO =   1515.3 Mbytes/s
	512 kbytes:	330.8 usec/IO =   1511.7 Mbytes/s
   1024 kbytes:	653.8 usec/IO =   1529.4 Mbytes/s
   2048 kbytes:   1276.4 usec/IO =   1566.9 Mbytes/s
   4096 kbytes:   2593.0 usec/IO =   1542.6 Mbytes/s
   8192 kbytes:   5148.3 usec/IO =   1553.9 Mbytes/s
root@freenas:~ #
 
Last edited:

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,087
Congratulations, that P4600 is the new leader for small-block writes (<=32K) - it can't keep up with Optane at the high end and I imagine its endurance rating is a lot lower as well.

(Edit- Actually it's not that bad at all. The P4800x is rated for 20.5PBW and the P4600 for 11.08PBW. Good job on the high-endurance TLC, Intel.)

Regarding the Optane P4800X, could I trouble you to change the physical sector size to 4K and run the tests again?

https://www.intel.com/content/www/u...6238/memory-and-storage/data-center-ssds.html
 

HoneyBadger

actually does care
Administrator
Moderator
iXsystems
Joined
Feb 6, 2014
Messages
5,087
Optane 900p drives labeled for their owner:

E - Elliot Dierksen
S - svtkobra7 (20GB vdisk)
T3 - tazinblack (3rd run)
Chris Moore has the P4800X, P4600, and the 144GB HPA'd P3700

upload_2018-9-12_15-10-10.png


The improvement between the P3700 and P4600 alone is impressive; mind you, the latter is a 2TB drive so it has a lot more NAND to worth with.
 

Chris Moore

Hall of Famer
Joined
May 2, 2015
Messages
10,080
Regarding the Optane P4800X, could I trouble you to change the physical sector size to 4K and run the tests again?

https://www.intel.com/content/www/u...6238/memory-and-storage/data-center-ssds.html
I just created the pool and was going to throw some data on it to see how it works, but I don't have a 10Gb connection at my workbench, so I can't really tell that much. I might as well do this. The tool is for either Linux or Windows though. OK. I will need to fiddle with it a bit and get back to you.
 

Stux

MVP
Joined
Jun 2, 2016
Messages
4,292
P3700 is still a good performer at least :)

And the 800GB versions were always faster too.
 

Brezlord

Contributor
Joined
Jan 7, 2017
Messages
189
Code:
root@nas1:~ # smartctl -a /dev/nvme0
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.2-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:					   SAMSUNG MZ1LV480HCHP-000MU
Serial Number:					  S2C1NAAH600033
Firmware Version:				   BXV76M8Q
PCI Vendor/Subsystem ID:			0x144d
IEEE OUI Identifier:				0x382500
Controller ID:					  1
Number of Namespaces:			   1
Namespace 1 Size/Capacity:		  480,103,981,056 [480 GB]
Namespace 1 Utilization:			3,177,054,208 [3.17 GB]
Namespace 1 Formatted LBA Size:	 512
Namespace 1 IEEE EUI-64:			002538 2266200033
Local Time is:					  Mon Sep 17 21:22:15 2018 AWST
Firmware Updates (0x07):			3 Slots, Slot 1 R/O
Optional Admin Commands (0x000e):   Format Frmw_DL NS_Mngmt
Optional NVM Commands (0x005f):	 Comp Wr_Unc DS_Mngmt Wr_Zero Sav/Sel_Feat Timestmp

Supported Power States
St Op	 Max   Active	 Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +	 8.00W	   -		-	0  0  0  0	   30	  30

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +	 512	   0		 0

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff)
Critical Warning:				   0x00
Temperature:						27 Celsius
Available Spare:					100%
Available Spare Threshold:		  10%
Percentage Used:					0%
Data Units Read:					12,872 [6.59 GB]
Data Units Written:				 67,696,497 [34.6 TB]
Host Read Commands:				 188,906
Host Write Commands:				934,361,473
Controller Busy Time:			   533
Power Cycles:					   37
Power On Hours:					 4,466
Unsafe Shutdowns:				   21
Media and Data Integrity Errors:	0
Error Information Log Entries:	  2
Warning  Comp. Temperature Time:	95
Critical Comp. Temperature Time:	69

Error Information (NVMe Log 0x01, max 64 entries)
Num   ErrCount  SQId   CmdId  Status  PELoc		  LBA  NSID	VS
  0		  2	 0  0x001b  0x4004  0x000			0	 0	 -
  1		  1	 0  0x001b  0x4004  0x000			0	 0	 -


Code:
root@nas1:~ # diskinfo -wS /dev/nvd0
/dev/nvd0
		512			 # sectorsize
		480103981056	# mediasize in bytes (447G)
		937703088	   # mediasize in sectors
		0			   # stripesize
		0			   # stripeoffset
		SAMSUNG MZ1LV480HCHP-000MU	  # Disk descr.
		S2C1NAAH600033  # Disk ident.
		Yes			 # TRIM/UNMAP support
		0			   # Rotation rate in RPM

Synchronous random writes:
		 0.5 kbytes:	 24.9 usec/IO =	 19.6 Mbytes/s
		   1 kbytes:	 25.0 usec/IO =	 39.1 Mbytes/s
		   2 kbytes:	 26.5 usec/IO =	 73.8 Mbytes/s
		   4 kbytes:	 26.7 usec/IO =	146.5 Mbytes/s
		   8 kbytes:	 28.1 usec/IO =	278.0 Mbytes/s
		  16 kbytes:	 32.7 usec/IO =	477.4 Mbytes/s
		  32 kbytes:	 57.6 usec/IO =	542.5 Mbytes/s
		  64 kbytes:	115.0 usec/IO =	543.3 Mbytes/s
		 128 kbytes:	229.3 usec/IO =	545.2 Mbytes/s
		 256 kbytes:	458.9 usec/IO =	544.8 Mbytes/s
		 512 kbytes:	918.2 usec/IO =	544.5 Mbytes/s
		1024 kbytes:   1837.8 usec/IO =	544.1 Mbytes/s
		2048 kbytes:   3681.4 usec/IO =	543.3 Mbytes/s
		4096 kbytes:   7345.4 usec/IO =	544.6 Mbytes/s
		8192 kbytes:  14784.1 usec/IO =	541.1 Mbytes/s
 
Last edited:
Joined
Dec 29, 2014
Messages
1,135
In case anybody is interested, Amazon seems is having a special on the 280GB Optane 900P's for $299.00. I think you have to be a prime member to get that, but I couldn't help myself at that price. I ordered one for my backup FreeNAS.
 

Ender117

Patron
Joined
Aug 20, 2018
Messages
219
In case anybody is interested, Amazon seems is having a special on the 280GB Optane 900P's for $299.00. I think you have to be a prime member to get that, but I couldn't help myself at that price. I ordered one for my backup FreeNAS.
not a few days ago newegg have it for 260...
 

mjt5282

Contributor
Joined
Mar 19, 2013
Messages
139
Here is an interesting example of a affordable homelab (read: NOT PRODUCTION SUITABLE!!) SLOG
I bought an Intel NVME M.10 Optane 32GB and a PCIE 3.0 adapter board. My X11SSM motherboard did not have a NVME slot. The memory card was about $60 and the adapter board was $16. I had to install Freenas 11.1.6 in UEFI mode and reload my configuration in order to recognize the storage device installed on the PCIE adapter.

Code:
diskinfo -wS /dev/nvd0 
/dev/nvd0 

512 		  # sectorsize 

29260513280 # mediasize in bytes (27G) 
57149440  	# mediasize in sectors 

0 			# stripesize 
0 			# stripeoffset 

INTEL MEMPEK1J032GA # Disk descr. 

PHBT75XXXXXXXX # Disk ident. 
Synchronous random writes: 

0.5 kbytes:	  9.0 usec/IO =	 54.4 Mbytes/s
  1 kbytes:	  8.8 usec/IO =	110.6 Mbytes/s
  2 kbytes:	 11.4 usec/IO =	171.1 Mbytes/s
  4 kbytes:	 16.8 usec/IO =	232.1 Mbytes/s
  8 kbytes:	 28.2 usec/IO =	277.2 Mbytes/s
  16 kbytes:	 51.4 usec/IO =	303.8 Mbytes/s
  32 kbytes:	 97.0 usec/IO =	322.1 Mbytes/s
  64 kbytes:	189.8 usec/IO =	329.3 Mbytes/s
128 kbytes:	399.1 usec/IO =	313.2 Mbytes/s
256 kbytes:	779.4 usec/IO =	320.7 Mbytes/s
512 kbytes:   1494.1 usec/IO =	334.6 Mbytes/s
1024 kbytes:   2946.2 usec/IO =	339.4 Mbytes/s
2048 kbytes:   5866.6 usec/IO =	340.9 Mbytes/s
4096 kbytes:  11719.2 usec/IO =	341.3 Mbytes/s
8192 kbytes:  23423.3 usec/IO =	341.5 Mbytes/s


Code:
smartctl -a /dev/nvme0
smartctl 6.6 2017-11-05 r4594 [FreeBSD 11.1-STABLE amd64] (local build)
Copyright (C) 2002-17, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Number:					   INTEL MEMPEK1J032GA
Serial Number:					  PHBT75XXXXXXXXX
Firmware Version:				   K4110400
PCI Vendor/Subsystem ID:			0x8086
IEEE OUI Identifier:				0x5cd2e4
Controller ID:					  0
Number of Namespaces:			   1
Namespace 1 Size/Capacity:		  29,260,513,280 [29.2 GB]
Namespace 1 Formatted LBA Size:	 512
Namespace 1 IEEE EUI-64:			5cd2e4 1925060100
Local Time is:					  Mon Sep 24 21:51:48 2018 EDT
Firmware Updates (0x02):			1 Slot
Optional Admin Commands (0x0006):   Format Frmw_DL
Optional NVM Commands (0x0046):	 Wr_Unc DS_Mngmt Timestmp
Maximum Data Transfer Size:		 32 Pages

Supported Power States
St Op	 Max   Active	 Idle   RL RT WL WT  Ent_Lat  Ex_Lat
 0 +	 2.80W	   -		-	0  0  0  0  1000000   30000
 1 +	 2.20W	   -		-	0  1  0  1  1000000   30000
 2 +	 1.80W	   -		-	0  2  0  2  1000000   30000
 3 -   0.0080W	   -		-	0  0  0  0  1150000   30000

Supported LBA Sizes (NSID 0x1)
Id Fmt  Data  Metadt  Rel_Perf
 0 +	 512	   0		 2

=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

SMART/Health Information (NVMe Log 0x02, NSID 0xffffffff)
Critical Warning:				   0x00
Temperature:						42 Celsius
Available Spare:					100%
Available Spare Threshold:		  0%
Percentage Used:					0%
Data Units Read:					559 [286 MB]
Data Units Written:				 1,094,439 [560 GB]
Host Read Commands:				 5,743
Host Write Commands:				8,777,991
Controller Busy Time:			   0
Power Cycles:					   13
Power On Hours:					 269
Unsafe Shutdowns:				   4
Media and Data Integrity Errors:	0
Error Information Log Entries:	  0

Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged



Happily I carved it into two (I have two pools ...) and configured it to be my SLOG(s). I took out the Intel SSDs that have been soldiering on & on. A Side note: SLOG(s) are really only used for NFS writes and things like database sync writes. It might be a benefit or it might not, use:
Code:
gstat -p -I 5s

and see if your new slog is even used at all during your important tests. Results may surprise you.
 
Top