SMB blocked during veracrypt container creation

[Benni.blanko]

Hi there,
I'm trying to create a veracrypt container file on a Truenas Core 13.0-U5.2.
While the creation is running, my Windows 10 PC is writing with 500MBit/s to the device (over the 1GB Ethernet interface).
Also ALL other SMB connections are timing out/throwing errors on the Truenas box during this process. SMB seems to be completely unusable.
And this happens even on creating a small 20GB veracrypt container. Don't think a 3TB one ...

Is that a known behaviour? One connection can block everything else?
Isn't SMB here capable of handling multiple (even high loading) connections?
Should it even run faster than 500Mbit/s on SMB?
Replication operations to another Truenas Box are capable of fully utilizing the 1GB interface.

Truenas CPU-utilization is not alarming during this operation. All other services are also running fine (NFS, webgui, SSH).

top show:
last pid: 2754; load averages: 1.22, 0.94, 0.55 up 0+00:12:11 16:52:33
58 processes: 2 running, 56 sleeping
CPU: 8.8% user, 0.0% nice, 19.6% system, 0.0% interrupt, 71.5% idle
Mem: 1178M Active, 193M Inact, 1103M Wired, 29G Free
ARC: 266M Total, 72M MFU, 158M MRU, 11M Anon, 2973K Header, 17M Other
141M Compressed, 400M Uncompressed, 2.83:1 Ratio
Swap: 8192M Total, 8192M Free

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
2623 root 1 93 0 187M 161M CPU2 2 3:18 67.55% smbd
993 root 31 21 0 426M 287M kqread 1 0:45 0.38% python3.9
2754 root 1 21 0 14M 4160K CPU1 1 0:00 0.25% top
2740 root 1 20 0 14M 4168K select 3 0:00 0.07% top
1765 root 4 20 0 34M 6876K select 2 0:01 0.06% vmtoolsd
959 root 1 20 0 11M 2012K select 1 0:00 0.01% devd
2208 www 1 20 0 38M 11M kqread 0 0:00 0.01% nginx
2741 root 1 20 0 20M 9124K select 2 0:00 0.01% sshd
2046 root 1 20 0 28M 2880K select 2 0:00 0.01% mountd
2110 root 8 20 0 49M 13M select 2 0:01 0.01% rrdcached
2081 ntpd 1 20 0 17M 6884K select 0 0:00 0.01% ntpd
975 root 1 20 0 19M 6576K select 2 0:00 0.00% zfsd
2055 root 16 52 0 12M 2672K rpcsvc 1 0:00 0.00% nfsd

Configuration:
HPE DL380 with 10Core Xeon E5-2640V4 CPU (10C/20T 2,4Ghz/3,2Ghz) Total 144GB RAM, 1GB/s Ethernet port used
Truenas running as VM within ESXi 6.5.0U3, configured for 4vCPU (normally running fine with 2vCPU, increased to 4 now but no change in behaviour), 32GB RAM
LSI 2008 per PCI devicethru given to Truenas VM; several HDD and SDD attached
This setup runs well for >1,5 years now and I just fell into this problem while creating the veracrypt container.

Regards
BB

 

[Benni.blanko]

Correction ...
looks like only the SMB connections from my veracrypt PC are failing while the container creates; others PCs still can use SMB shares on the Truenas box; maybe it's Windows then?

 

[WI_Hedgehog]

Could it be writes are bottle-necking? I'm not really sure what you're running there, but there are several factors that can cause "slow" writes.

 

[Benni.blanko]

simply copying a 34GB file to the Truenas SMB also tops out at 500MBit/s, but while that is running, other SMB functions (browsing, reading/writing files) is running fine. It seems something special in the kind of sparse file veracrypt or Windows are creating.
Ethernet speed over SMB is something to be looked at later as well.

 

[Benni.blanko]

copying two files in parallel from my Windows PC to Truenas give 850-930MBit/s. Okay, Truenas can handle that speed and it's Windows and/or SMB which is unable to speed up more that 500MBit/s per write operation.
And during both copy jobs maxing out at >900MBit/s together, I'm still able to browse the shares and do other read/write operations.
So two things learned:
1. SMB in my configuration max. out at 500MBit/s
2. only the veracrypt container creation is killing all other SMB operations FROM THE SAME PC

Well...then I can handle the situation by doing the veracrypt container creation with a dedicated PC, which can be blocked for a couple of hours.

Thanks.

 

[winnielinnie]

Is it out of the picture to create a dedicated encrypted dataset that you can lock/unlock with a passphrase?

You won't be limited to using VeraCrypt's software, nor limited to any arbitrary capacity, nor deal with the copy-on-write consequences of ZFS over SMB for a single massively large file which will be modified frequently.

 

[Benni.blanko]

@winnielinnie: I'm using encrypted ZFS datasets on other occasions, but in this case I'd like to stay with veracrypt.

But I have solution to trick windows or veracrypt on the container creation:
1. start the 3TB veracrypt container creation process
2. let it run for some minutes, then abort and restart windows (as the abort is not stoping the data transfer from Windows to the container file)
3. after restart mount the container file with veracrypt; this does work, but the drive letter cannot be used, as the format of the presented raw device was aborted through the reboot
4. do a manual quick format (format z: /q /fs:ntfs) on the mounted drive letter
Done!
Now there is a 3TB veracrypt container file available, which is "thin provisioned". It's shown as 3TB in the file system, but is consuming only a few hundred MB in the Truenas file system.
Filling the veracrypt container is then letting the Truenas file growing in the dataset.
Perfect for my requirements!

Problem solved.

 

[WI_Hedgehog]

@winnielinnie: I'm using encrypted ZFS datasets on other occasions, but in this case I'd like to stay with veracrypt.

But I have solution to trick windows or veracrypt on the container creation:
1. start the 3TB veracrypt container creation process
2. let it run for some minutes, then abort and restart windows (as the abort is not stoping the data transfer from Windows to the container file)
3. after restart mount the container file with veracrypt; this does work, but the drive letter cannot be used, as the format of the presented raw device was aborted through the reboot
4. do a manual quick format (format z: /q /fs:ntfs) on the mounted drive letter
Done!
Now there is a 3TB veracrypt container file available, which is "thin provisioned". It's shown as 3TB in the file system, but is consuming only a few hundred MB in the Truenas file system.
Filling the veracrypt container is then letting the Truenas file growing in the dataset.
Perfect for my requirements!

Problem solved.

For me (who's not necessarily lazy but also not overly-motivated to do more work than necessary) that's a lot of unnecessary work. Again, it's just me, but I'd select Quick Format and Dynamic to accomplish the same result, but...you do you; I'm good with it.

For a normal fat container (not sparsely provisioned) I'd definitely use Quick Format as TrueNAS will handle the rest.

(This is probably why I have so much time to stare out the window and listen to music "at work.")

 

[winnielinnie]

What about "Elaine"?

 

[WI_Hedgehog]

What about "Elaine"?

I'd have never guessed her name is "Elaine." Generally that's reserved for someone's aunt's name, isn't it? Update here:

TrueNAS might not be for you, if you are home user.

"cmd.exe" is behind-the-times; a relic from the days of DOS. Agreed, but if you prefer it for some reason, you can still SSH from there.

www.truenas.com