TrueNAS Scale on esxi w/USB3.1(USB-C) passthrough, NO DISKS SHOWING

[bigjiggity]

Hardware involved:
Mac Pro 5,1 - ESXi 7
VM assigned 4 cores/32GB RAM
PCIE passthrough: ASM2142 USB 3.1 Host Controller
Drives: x5 - HGST 10TB

So Im sure I am comiting Heresy by Virtualizing SCALE and doing a hardware pass through of a USB-C/3.1 controller to a 5 disk bay, but the
Host Sees drives, has individual serial numbers for each disk, was able to fdisk and add zfs partition map to drives.
see output below:

Code:

=== START OF INFORMATION SECTION ===
Device Model:     HUH721010ALE601
Serial Number:    2TG08V7D
LU WWN Device Id: 5 000cca 26ac02135
Firmware Version: LHGL0003
User Capacity:    10,000,831,348,736 bytes [10.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Apr 12 05:23:24 2023 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF INFORMATION SECTION ===
Device Model:     HUH721010ALE601
Serial Number:    7JKUREGC
LU WWN Device Id: 5 000cca 266f5f801
Firmware Version: LHGL0003
User Capacity:    10,000,831,348,736 bytes [10.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Apr 12 05:24:09 2023 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF INFORMATION SECTION ===
Device Model:     HUH721010ALE601
Serial Number:    7JKA2XJC
LU WWN Device Id: 5 000cca 266eedd2f
Firmware Version: LHGL0003
User Capacity:    10,000,831,348,736 bytes [10.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Apr 12 05:24:17 2023 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF INFORMATION SECTION ===
Device Model:     HUH721010ALE601
Serial Number:    7JJPYUXE
LU WWN Device Id: 5 000cca 266e62acd
Firmware Version: LHGL0003
User Capacity:    10,000,831,348,736 bytes [10.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Apr 12 05:24:33 2023 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF INFORMATION SECTION ===
Device Model:     HUH721010ALE601
Serial Number:    7JKT51SC
LU WWN Device Id: 5 000cca 266f54266
Firmware Version: LHGL0003
User Capacity:    10,000,831,348,736 bytes [10.0 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-2, ATA8-ACS T13/1699-D revision 4
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Wed Apr 12 05:24:35 2023 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

Was able to fdisk all drives and add ZFS partition map to them via CLI, but SCALE will not recognize them:

at a loss... any and all feedback welcome

 

[Arwen]

Why did you manually fdisk and add a ZFS partition?

TrueNAS does that for you with blank disks. Now you have remove that partitioning information and start over. Of course, I can not guarantee your USB enclosure will work. Nor that your virtualization will work.

You also don't list your boot device. Their have been cases where someone assigned all their disks to the boot-pool, thus no disks were available for data purposes. They did not understand that TrueNAS, (BOTH Core & SCALE), use dedicated boot devices;

I have to waste an entire drive just for booting?

With the advent of TrueNAS Scale, we seem to be getting a large number of people from the Linux community who are not quite grasping what TrueNAS is. TrueNAS is not a "Linux distro" and it is not designed to try to meet your arbitrarily spec'd...

www.truenas.com

On the subject of USB data drives;

Why you should avoid USB attached drives for data pool disks

This subject has been coming up quite often in the last year or 2. Perhaps because of TrueNAS SCALE has brought forth some attention to TrueNAS in general and ZFS in specific. Please note that this is about USB attached storage for ZFS data...

www.truenas.com

On virtualizing TrueNAS:

Resource - "Absolutely must virtualize TrueNAS!" ... a guide to not completely losing your data.

[---- 2024/01/16: Still relevant. Virtualization really doesn't change much. Updates made as appropriate. ----] [---- 2018/02/27: This is still as relevant as ever. As PCIe-Passthru has matured, fewer problems are reported. I've updated some...

www.truenas.com

 

[bigjiggity]

Why did you manually fdisk and add a ZFS partition?

TrueNAS does that for you with blank disks. Now you have remove that partitioning information and start over. Of course, I can not guarantee your USB enclosure will work. Nor that your virtualization will work.

You also don't list your boot device. Their have been cases where someone assigned all their disks to the boot-pool, thus no disks were available for data purposes. They did not understand that TrueNAS, (BOTH Core & SCALE), use dedicated boot devices;

I have to waste an entire drive just for booting?

With the advent of TrueNAS Scale, we seem to be getting a large number of people from the Linux community who are not quite grasping what TrueNAS is. TrueNAS is not a "Linux distro" and it is not designed to try to meet your arbitrarily spec'd...

www.truenas.com

On the subject of USB data drives;

Why you should avoid USB attached drives for data pool disks

This subject has been coming up quite often in the last year or 2. Perhaps because of TrueNAS SCALE has brought forth some attention to TrueNAS in general and ZFS in specific. Please note that this is about USB attached storage for ZFS data...

www.truenas.com

On virtualizing TrueNAS:

Resource - "Absolutely must virtualize TrueNAS!" ... a guide to not completely losing your data.

[---- 2024/01/16: Still relevant. Virtualization really doesn't change much. Updates made as appropriate. ----] [---- 2018/02/27: This is still as relevant as ever. As PCIe-Passthru has matured, fewer problems are reported. I've updated some...

www.truenas.com

Why did you manually fdisk and add a ZFS partition?

TrueNAS does that for you with blank disks. Now you have remove that partitioning information and start over. Of course, I can not guarantee your USB enclosure will work. Nor that your virtualization will work.

You also don't list your boot device. Their have been cases where someone assigned all their disks to the boot-pool, thus no disks were available for data purposes. They did not understand that TrueNAS, (BOTH Core & SCALE), use dedicated boot devices;

I have to waste an entire drive just for booting?

With the advent of TrueNAS Scale, we seem to be getting a large number of people from the Linux community who are not quite grasping what TrueNAS is. TrueNAS is not a "Linux distro" and it is not designed to try to meet your arbitrarily spec'd...

www.truenas.com

On the subject of USB data drives;

Why you should avoid USB attached drives for data pool disks

This subject has been coming up quite often in the last year or 2. Perhaps because of TrueNAS SCALE has brought forth some attention to TrueNAS in general and ZFS in specific. Please note that this is about USB attached storage for ZFS data...

www.truenas.com

On virtualizing TrueNAS:

Resource - "Absolutely must virtualize TrueNAS!" ... a guide to not completely losing your data.

[---- 2024/01/16: Still relevant. Virtualization really doesn't change much. Updates made as appropriate. ----] [---- 2018/02/27: This is still as relevant as ever. As PCIe-Passthru has matured, fewer problems are reported. I've updated some...

www.truenas.com

.
To prove that the underlying OS saw the drives and could partition etc… so the application layer (truenas) *should* be able to interact/manipulate the drives.

Boot devices is a VMDK on a SSD, and the other disks aren’t attached. Not my first rodeo with truenas

Again… all issues with usb are mitigated. Controller is supported in ESXi, is direct pci pass through so no virtual layer, the the drive enclosure is brain dead, no seperate chip, it’s just pure bus to disk. The controller/enclosure are on a 20Gbps usb-c bus… more than enough through put.

Covered all the precautions on virtualiszing and hardware is up to snuff. Still doesn’t explain why the underlying OS of core can see/manipulate drives, the TrueNAS can’t. Doesn’t make sense.

 

[Arwen]

Excellent. I did not mean to imply you were a complete novice at TrueNAS.

To be clear, you did remove the partition table from the disks and reboot the NAS?


What does the GUI screen below show?
Storage tab on right -> Disks button on upper right

It should show the all the devices, used by a ZFS pool or not. In theory, if your 5 USB attached disks do show up, you can use the pull down menu on each disk to "wipe" them. Which should then allow them to be made into a ZFS pool.


One last comment, just because the Linux OS in SCALE can detect the serial numbers of the 5 USB attached drives, does not mean SCALE middleware can.