16GB optane, for posteriority:
root@freenas[/mnt/SAS]# smartctl -a /dev/nvme0
smartctl 7.1 2019-12-30 r5022 [FreeBSD 12.1-STABLE amd64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke,
www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Number: INTEL MEMPEK1W016GA
Serial Number: PHBT813403S5016D
Firmware Version: K3110310
PCI Vendor/Subsystem ID: 0x8086
IEEE OUI Identifier: 0x5cd2e4
Controller ID: 0
Number of Namespaces: 1
Namespace 1 Size/Capacity: 14,403,239,936 [14.4 GB]
Namespace 1 Formatted LBA Size: 512
Local Time is: Fri Aug 28 10:35:37 2020 CEST
Firmware Updates (0x02): 1 Slot
Optional Admin Commands (0x0006): Format Frmw_DL
Optional NVM Commands (0x0046): Wr_Unc DS_Mngmt Timestmp
Maximum Data Transfer Size: 32 Pages
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 4.50W - - 0 0 0 0 0 0
Supported LBA Sizes (NSID 0x1)
Id Fmt Data Metadt Rel_Perf
0 + 512 0 2
=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02)
Critical Warning: 0x00
Temperature: 57 Celsius
Available Spare: 100%
Available Spare Threshold: 0%
Percentage Used: 0%
Data Units Read: 271 [138 MB]
Data Units Written: 2,919,304 [1.49 TB]
Host Read Commands: 6,427
Host Write Commands: 25,410,239
Controller Busy Time: 0
Power Cycles: 18
Power On Hours: 263
Unsafe Shutdowns: 1
Media and Data Integrity Errors: 0
Error Information Log Entries: 0
Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged
root@freenas[/mnt/SAS]# diskinfo -wS /dev/nvd0
/dev/nvd0
512 # sectorsize
14403239936 # mediasize in bytes (13G)
28131328 # mediasize in sectors
0 # stripesize
0 # stripeoffset
INTEL MEMPEK1W016GA # Disk descr.
PHBT813403S5016D # Disk ident.
Yes # TRIM/UNMAP support
0 # Rotation rate in RPM
Synchronous random writes:
0.5 kbytes: 13.7 usec/IO = 35.7 Mbytes/s
1 kbytes: 13.1 usec/IO = 74.4 Mbytes/s
2 kbytes: 19.2 usec/IO = 101.9 Mbytes/s
4 kbytes: 32.6 usec/IO = 120.0 Mbytes/s
8 kbytes: 59.7 usec/IO = 130.8 Mbytes/s
16 kbytes: 114.0 usec/IO = 137.0 Mbytes/s
32 kbytes: 224.6 usec/IO = 139.1 Mbytes/s
64 kbytes: 445.1 usec/IO = 140.4 Mbytes/s
128 kbytes: 897.9 usec/IO = 139.2 Mbytes/s
256 kbytes: 1770.4 usec/IO = 141.2 Mbytes/s
512 kbytes: 3513.3 usec/IO = 142.3 Mbytes/s
1024 kbytes: 6974.2 usec/IO = 143.4 Mbytes/s
2048 kbytes: 13872.6 usec/IO = 144.2 Mbytes/s
4096 kbytes: 27682.9 usec/IO = 144.5 Mbytes/s
8192 kbytes: 55192.2 usec/IO = 144.9 Mbytes/s
Optane connected in pcix 2.0 X1:
root@freenas[~]# diskinfo -wS /dev/nvd1
/dev/nvd1
512 # sectorsize
14403239936 # mediasize in bytes (13G)
28131328 # mediasize in sectors
0 # stripesize
0 # stripeoffset
INTEL MEMPEK1W016GA # Disk descr.
PHBT813403YH016D # Disk ident.
Yes # TRIM/UNMAP support
0 # Rotation rate in RPM
Synchronous random writes:
0.5 kbytes: 18.0 usec/IO = 27.1 Mbytes/s
1 kbytes: 17.9 usec/IO = 54.5 Mbytes/s
2 kbytes: 24.2 usec/IO = 80.8 Mbytes/s
4 kbytes: 38.2 usec/IO = 102.2 Mbytes/s
8 kbytes: 67.4 usec/IO = 115.8 Mbytes/s
16 kbytes: 127.6 usec/IO = 122.4 Mbytes/s
32 kbytes: 249.0 usec/IO = 125.5 Mbytes/s
64 kbytes: 496.9 usec/IO = 125.8 Mbytes/s
128 kbytes: 988.9 usec/IO = 126.4 Mbytes/s
256 kbytes: 1952.2 usec/IO = 128.1 Mbytes/s
512 kbytes: 3837.4 usec/IO = 130.3 Mbytes/s
1024 kbytes: 7572.2 usec/IO = 132.1 Mbytes/s
2048 kbytes: 15198.8 usec/IO = 131.6 Mbytes/s
4096 kbytes: 29953.4 usec/IO = 133.5 Mbytes/s
8192 kbytes: 59712.3 usec/IO = 134.0 Mbytes/s
root@freenas[~]# diskinfo -wS /dev/nvd2
/dev/nvd2
512 # sectorsize
14403239936 # mediasize in bytes (13G)
28131328 # mediasize in sectors
0 # stripesize
0 # stripeoffset
INTEL MEMPEK1W016GA # Disk descr.
PHBT813404B2016D # Disk ident.
Yes # TRIM/UNMAP support
0 # Rotation rate in RPM
Synchronous random writes:
0.5 kbytes: 17.2 usec/IO = 28.4 Mbytes/s
1 kbytes: 17.7 usec/IO = 55.1 Mbytes/s
2 kbytes: 24.0 usec/IO = 81.4 Mbytes/s
4 kbytes: 37.7 usec/IO = 103.6 Mbytes/s
8 kbytes: 66.3 usec/IO = 117.9 Mbytes/s
16 kbytes: 124.7 usec/IO = 125.3 Mbytes/s
32 kbytes: 243.5 usec/IO = 128.3 Mbytes/s
64 kbytes: 482.8 usec/IO = 129.4 Mbytes/s
128 kbytes: 963.2 usec/IO = 129.8 Mbytes/s
256 kbytes: 1868.3 usec/IO = 133.8 Mbytes/s
512 kbytes: 3721.1 usec/IO = 134.4 Mbytes/s
1024 kbytes: 7407.1 usec/IO = 135.0 Mbytes/s
2048 kbytes: 14707.9 usec/IO = 136.0 Mbytes/s
4096 kbytes: 29137.1 usec/IO = 137.3 Mbytes/s
8192 kbytes: 57974.8 usec/IO = 138.0 Mbytes/s
root@freenas[~]#
root@freenas[~]# smartctl -d nvme -a /dev/nvme1
smartctl 7.1 2019-12-30 r5022 [FreeBSD 12.1-STABLE amd64] (local build)
Copyright (C) 2002-19, Bruce Allen, Christian Franke,
www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Number: INTEL MEMPEK1W016GA
Serial Number: PHBT813403YH016D
Firmware Version: K3110310
PCI Vendor/Subsystem ID: 0x8086
IEEE OUI Identifier: 0x5cd2e4
Controller ID: 0
Number of Namespaces: 1
Namespace 1 Size/Capacity: 14,403,239,936 [14.4 GB]
Namespace 1 Formatted LBA Size: 512
Local Time is: Fri Sep 11 14:50:40 2020 CEST
Firmware Updates (0x02): 1 Slot
Optional Admin Commands (0x0006): Format Frmw_DL
Optional NVM Commands (0x0046): Wr_Unc DS_Mngmt Timestmp
Maximum Data Transfer Size: 32 Pages
Supported Power States
St Op Max Active Idle RL RT WL WT Ent_Lat Ex_Lat
0 + 4.50W - - 0 0 0 0 0 0
Supported LBA Sizes (NSID 0x1)
Id Fmt Data Metadt Rel_Perf
0 + 512 0 2
=== START OF SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
SMART/Health Information (NVMe Log 0x02)
Critical Warning: 0x00
Temperature: 48 Celsius
Available Spare: 100%
Available Spare Threshold: 0%
Percentage Used: 0%
Data Units Read: 11 [5.63 MB]
Data Units Written: 0
Host Read Commands: 1,248
Host Write Commands: 0
Controller Busy Time: 0
Power Cycles: 14
Power On Hours: 0
Unsafe Shutdowns: 0
Media and Data Integrity Errors: 0
Error Information Log Entries: 0
Error Information (NVMe Log 0x01, max 64 entries)
No Errors Logged
4x16GB optane in raid 10 produce roghly 300MB/s of sync writes
Will soon buy some 112G ones to see the difference, guess 4x112G will reach roughly 1GB/s, so al little beyond what a 10G link can carry
This is what I get in a VM(it´s not the only VM active so real number are a little better, total bandwidth of VM is caped at roughly 800MB/s)