Frequent SMB disconnects, help troubleshooting please.

[DenisInternet]

Hello,

I am lately experiencing frequent but seemingly random SMB disconnects. I tried swapping out the NIC on my NAS for the same model "Supermicro AOC-S25G-B2S" and for a while everything worked well but now the SMB disconnects have returned. I tried swapping fiber cables, but that didn't work either. I tried restarting MacOS and occasionally this solves the issue but doesn't always solve the issue. My network switch is up to date, and isn't showing any red flags/warning signs, and the same is true for TrueNAS. Any ideas on how to trouble shoot what might be causing this?

Things I haven't tried yet:

1. Replacing the Sonnet Twin25G thunderbolt adapter for my Mac (this part is expensive so would like to avoid buying another just for troubleshooting if possible).
2. Re-installing TrueNAS Cobia from scratch vs upgrading.
3. Connecting on another Mac system, I did however connect via my PopOS box, and that had issues as well (and it wasn't using the thunderbolt adapter.

Things I have changed recently on TrueNAS:
I added a new 'child' Dataset for Apple's TimeMachine, for which I also created a separate User and SMB share.
So my main edit share I connect via "smb://Post_Main:*@192.168.1.251" and login with one set of credentials,
and connect to my Time Machine share via "smb://mac01:*@192.168.1.251" and login using a second set of credentials.

The disconnects have become more frequent since I made this change, but I am not sure they are the cause of it, or a just exacerbating the issue.

Any advice, ideas, pointers, to how best to troubleshoot this, would as always be greatly appreciated. (Hardware is in my signature)

Thank you!!

sudo less /var/log/messages:

Code:

 
Mar 22 02:47:53 truenas systemd-journald[1045]: Data hash table of /var/log/journal/9f39f538868e46d0a429492f6a262deb/system.journal has a fill level at 75.0 (8533 of 11377 items, 6553600 file size, 768 bytes per hash table item), suggesting rotation.
Mar 22 02:47:53 truenas systemd-journald[1045]: /var/log/journal/9f39f538868e46d0a429492f6a262deb/system.journal: Journal header limits reached or header out-of-date, rotating.
Mar 23 23:59:16 truenas systemd-journald[1045]: Data hash table of /var/log/journal/9f39f538868e46d0a429492f6a262deb/system.journal has a fill level at 75.0 (8533 of 11377 items, 6553600 file size, 768 bytes per hash table item), suggesting rotation.
Mar 23 23:59:16 truenas systemd-journald[1045]: /var/log/journal/9f39f538868e46d0a429492f6a262deb/system.journal: Journal header limits reached or header out-of-date, rotating.
Mar 24 00:00:00 truenas syslog-ng[3753]: Configuration reload request received, reloading configuration;
Mar 24 00:00:00 truenas syslog-ng[3753]: Configuration reload finished;
Mar 25 11:55:49 truenas kernel: loop0: detected capacity change from 0 to 2526536
Mar 25 11:55:49 truenas kernel: squashfs: version 4.0 (2009/01/31) Phillip Lougher
Mar 25 11:55:49 truenas kernel: loop0: Can't mount, would change RO state
Mar 25 11:56:01 truenas kernel: loop0: detected capacity change from 0 to 2526536
Mar 25 18:26:46 truenas systemd-journald[1045]: Data hash table of /var/log/journal/9f39f538868e46d0a429492f6a262deb/system.journal has a fill level at 75.0 (8536 of 11377 items, 6553600 file size, 767 bytes per hash table item), suggesting rotation.
Mar 25 18:26:46 truenas systemd-journald[1045]: /var/log/journal/9f39f538868e46d0a429492f6a262deb/system.journal: Journal header limits reached or header out-of-date, rotating.

log show --predicate 'processImagePath contains "smbd" OR subsystem == "com.apple.smb"' --info --last 24h :

Code:

 
Log      - Default:         22, Info:                1, Debug:             0, Error:          2, Fault:          0
Activity - Create:          10, Transition:          0, Actions:           0
❯ log show --predicate 'processImagePath contains "smbd" OR subsystem == "com.apple.smb"' --info --last 24h
Filtering the log data using "processImagePath CONTAINS "smbd" OR subsystem == "com.apple.smb""
Skipping debug messages, pass --debug to include.
Timestamp                       Thread     Type        Activity             PID    TTL  
2024-03-24 21:52:31.768657-0400 0x30f1dd   Default     0x0                  27983  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-24 21:52:31.768750-0400 0x30f1dd   Default     0x0                  27983  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-24 22:08:54.611649-0400 0x30f91e   Default     0x0                  28014  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-24 22:08:54.611891-0400 0x30f91e   Default     0x0                  28014  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-24 22:27:24.589139-0400 0x3104e2   Default     0x0                  28090  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-24 22:27:24.589352-0400 0x3104e2   Default     0x0                  28090  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-24 22:45:43.812873-0400 0x3110a5   Default     0x0                  28180  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-24 22:45:43.813005-0400 0x3110a5   Default     0x0                  28180  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-24 23:02:26.517381-0400 0x3118aa   Default     0x0                  28216  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-24 23:02:26.517554-0400 0x3118aa   Default     0x0                  28216  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-24 23:20:47.289650-0400 0x312610   Default     0x0                  28295  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-24 23:20:47.289763-0400 0x312610   Default     0x0                  28295  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-24 23:37:42.884900-0400 0x312e53   Default     0x0                  28336  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-24 23:37:42.885201-0400 0x312e53   Default     0x0                  28336  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-24 23:50:35.565527-0400 0x313cbf   Default     0x0                  28464  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-24 23:50:35.565762-0400 0x313cbf   Default     0x0                  28464  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 00:07:29.447887-0400 0x3144c3   Default     0x0                  28516  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 00:07:29.448049-0400 0x3144c3   Default     0x0                  28516  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 00:24:30.746918-0400 0x314e3d   Default     0x0                  28562  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 00:24:30.746982-0400 0x314e3d   Default     0x0                  28562  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 00:41:26.214123-0400 0x3155ca   Default     0x0                  28597  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 00:41:26.214241-0400 0x3155ca   Default     0x0                  28597  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 01:03:53.334432-0400 0x3174b9   Default     0x0                  28899  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 01:03:53.334760-0400 0x3174b9   Default     0x0                  28899  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 01:23:14.914112-0400 0x318386   Default     0x0                  29026  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 01:23:14.914236-0400 0x318386   Default     0x0                  29026  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 01:42:08.829139-0400 0x3190ce   Default     0x0                  29122  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 01:42:08.829290-0400 0x3190ce   Default     0x0                  29122  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 02:19:16.622282-0400 0x31d6bf   Default     0x0                  30239  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 02:19:16.622372-0400 0x31d6bf   Default     0x0                  30239  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 02:36:10.304021-0400 0x31e001   Default     0x0                  30323  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 02:36:10.304137-0400 0x31e001   Default     0x0                  30323  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 03:39:57.852374-0400 0x326fe4   Default     0x0                  31853  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 03:39:57.852487-0400 0x326fe4   Default     0x0                  31853  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 03:58:28.084839-0400 0x327bd2   Default     0x0                  31940  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 03:58:28.084921-0400 0x327bd2   Default     0x0                  31940  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 04:16:12.593413-0400 0x3286fe   Default     0x0                  32019  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 04:16:12.593856-0400 0x3286fe   Default     0x0                  32019  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 04:34:29.630867-0400 0x3291af   Default     0x0                  32086  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 04:34:29.630997-0400 0x3291af   Default     0x0                  32086  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 04:53:31.971812-0400 0x329e99   Default     0x0                  32183  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 04:53:31.971905-0400 0x329e99   Default     0x0                  32183  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 05:12:07.637584-0400 0x32aaee   Default     0x0                  32264  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 05:12:07.637674-0400 0x32aaee   Default     0x0                  32264  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 05:20:43.028173-0400 0x32b918   Default     0x0                  32362  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 05:20:43.028351-0400 0x32b918   Default     0x0                  32362  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 05:38:56.592874-0400 0x32c377   Default     0x0                  32435  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 05:38:56.592999-0400 0x32c377   Default     0x0                  32435  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 05:57:10.653215-0400 0x32d078   Default     0x0                  32528  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 05:57:10.653354-0400 0x32d078   Default     0x0                  32528  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 06:14:55.736804-0400 0x32dc05   Default     0x0                  32603  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 06:14:55.737065-0400 0x32dc05   Default     0x0                  32603  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 06:33:26.427293-0400 0x32e898   Default     0x0                  32684  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 06:33:26.427533-0400 0x32e898   Default     0x0                  32684  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 06:40:58.827417-0400 0x32f3ec   Default     0x0                  32766  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 06:40:58.827530-0400 0x32f3ec   Default     0x0                  32766  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 06:59:12.397622-0400 0x32fee6   Default     0x0                  32838  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 06:59:12.397726-0400 0x32fee6   Default     0x0                  32838  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 07:16:55.847118-0400 0x330ac8   Default     0x0                  32911  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 07:16:55.847255-0400 0x330ac8   Default     0x0                  32911  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 07:35:53.046917-0400 0x3318db   Default     0x0                  33006  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 07:35:53.047003-0400 0x3318db   Default     0x0                  33006  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 07:41:59.066501-0400 0x33247f   Default     0x0                  33085  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 07:41:59.066633-0400 0x33247f   Default     0x0                  33085  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 07:45:52.181603-0400 0x33304d   Default     0x0                  33161  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 07:45:52.181790-0400 0x33304d   Default     0x0                  33161  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 07:46:11.663921-0400 0x3330bd   Default     0x0                  33161  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 07:46:11.663966-0400 0x3330bd   Default     0x0                  33161  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 07:46:11.664100-0400 0x3330bd   Default     0x0                  33161  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 07:46:11.664121-0400 0x3330bd   Default     0x0                  33161  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 07:46:11.664144-0400 0x3330bd   Default     0x0                  33161  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 07:46:11.664164-0400 0x3330bd   Default     0x0                  33161  0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 08:06:21.779191-0400 0x17e2     Default     0x0                  458    0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 137: Address already in use
2024-03-25 08:06:21.779354-0400 0x17e2     Default     0x0                  458    0    netbiosd: [com.apple.smb:default] unable to open broadcast UDP socket on <private>, port 138: Address already in use
2024-03-25 14:24:25.648353-0400 0x4c6b7    Activity    0x90750              11323  0    smbd: (libsystem_info.dylib) Retrieve User by ID
2024-03-25 14:24:25.653987-0400 0x4c6b7    Error       0x0                  11323  0    smbd: [com.apple.smb:default] smbd_detect_sg_mode: NOT enabling super guest mode, errno: 2
2024-03-25 14:24:25.658145-0400 0x4c6b7    Info        0x0                  11323  0    smbd: (libsandbox.1.dylib) sandbox compile: <private>
2024-03-25 14:24:25.671685-0400 0x4c6b7    Activity    0x90751              11323  0    smbd: (libsystem_info.dylib) Membership API: translate identifier
2024-03-25 14:24:25.672278-0400 0x4c6b7    Activity    0x90752              11323  0    smbd: (libsystem_info.dylib) Membership API: translate identifier
2024-03-25 14:24:25.672504-0400 0x4c6b7    Activity    0x90753              11323  0    smbd: (libsystem_info.dylib) Membership API: translate identifier
2024-03-25 14:24:25.672833-0400 0x4c6b7    Activity    0x90754              11323  0    smbd: (libsystem_info.dylib) Membership API: translate identifier
2024-03-25 14:24:25.676066-0400 0x4c6b7    Activity    0x90755              11323  0    smbd: (libsystem_info.dylib) Retrieve User by Name
2024-03-25 14:24:25.678378-0400 0x4c6b7    Activity    0x90756              11323  0    smbd: (libsystem_info.dylib) Membership API: translate identifier
2024-03-25 14:24:25.679612-0400 0x4c6b7    Activity    0x90757              11323  0    smbd: (libsystem_info.dylib) Membership API: translate identifier
2024-03-25 14:24:25.679862-0400 0x4c6b7    Activity    0x90758              11323  0    smbd: (libsystem_info.dylib) Membership API: translate identifier
2024-03-25 14:24:25.680171-0400 0x4c6b7    Activity    0x90759              11323  0    smbd: (libsystem_info.dylib) Membership API: translate identifier
2024-03-25 14:24:25.683089-0400 0x4c6b7    Error       0x0                  11323  0    smbd: [com.apple.smb:default] unable to impersonate the anonymous account
--------------------------------------------------------------------------------------------------------------------
Log      - Default:         74, Info:                1, Debug:             0, Error:          2, Fault:          0
Activity - Create:          10, Transition:          0, Actions:           0

 

[Kris Moore]

Without logs its hard to say exactly what might be the underlying issue. Have you checked any TrueNAS or Mac Logs to see if anything stands out?

 

[DenisInternet]

Without logs its hard to say exactly what might be the underlying issue. Have you checked any TrueNAS or Mac Logs to see if anything stands out?

Thank you Kris, happy to do so. What would the recommended commands to run? In shell should I run: /var/log/messages ?

*updating with logs at the top.

 

[Kris Moore]

Don't see anything quite obvious in those logs. Probably better to look at /var/log/samba4/log.smbd if, nothing jumps out, then we may need a bug ticket with debug file attached so we can look further.

 

[Stux]

On your Mac, launch a terminal window, and run a ping to another system on your network.

When you get a disconnect look at the ping window to see if the disconnects line up with ping failures.

If it does, this shows the issue is outside of your TrueNAS.

 

[DenisInternet]

@Kris Moore So I am not sure yet how to capture this info in logs.

But I did manage to isolate the steps required that lead to this behavior on MacOS.

If I have any folders saved under "Favorites" from my smb share, and I mount the smb share by clicking on this folder under favorites, that's when I start getting degraded performance and disconnects.

So I have to make sure I access the SMB share by using Finder, CMD+K and connect to the server via ip address "smb://192.168.1.251".
After doing this step, I can access/use the favorite shortcuts without issue.

Does this behavior makes sense? Seems like more of a MacOS issue than TrueNAS, but if anyone has any insights, I appreciate your time and input.

@Stux I can only get the ping failures when I repeat the steps above, so I guess it does point towards an issue outside of TrueNAS.

Thanks again!