Smbd exited in signal 6 since last update ( FreeNAS-9.10.2-U4)

Status
Not open for further replies.
M

Morpheus187

Explorer
Joined
Mar 11, 2016
Messages
61
Hi there

I've recently updated my FreeNAS to patch the security hole in samba and now I've got some strange samba crashes. I haven't noticed any abnormal behaviour, everything still seems to work.

dmesg:

Code:
pid 68779 (smbd), uid 0: exited on signal 6 (core dumped)
pid 27798 (smbd), uid 0: exited on signal 6 (core dumped)
pid 4798 (smbd), uid 0: exited on signal 6 (core dumped)
pid 97903 (smbd), uid 0: exited on signal 6 (core dumped)


/var/log/samba/log.smbd

Code:
[2017/05/29 04:17:52.019763,  1] ../source3/smbd/service.c:502(make_connection_snum)
  create_connection_session_info failed: NT_STATUS_ACCESS_DENIED
[2017/05/29 04:17:52.020704,  1] ../source3/smbd/service.c:502(make_connection_snum)
  create_connection_session_info failed: NT_STATUS_ACCESS_DENIED
[2017/05/29 04:17:52.021183,  1] ../source3/smbd/service.c:502(make_connection_snum)
  create_connection_session_info failed: NT_STATUS_ACCESS_DENIED
[2017/05/29 09:49:01.474835,  0] ../source3/smbd/oplock.c:178(update_num_read_oplocks)
  PANIC: assert failed at ../source3/smbd/oplock.c(178): lease_type_is_exclusive(e_lease_type)
[2017/05/29 09:49:01.477005,  0] ../source3/lib/util.c:791(smb_panic_s3)
  PANIC (pid 4798): assert failed: lease_type_is_exclusive(e_lease_type)
[2017/05/29 09:49:01.481892,  0] ../source3/lib/util.c:902(log_stack_trace)
  BACKTRACE: 25 stack frames:
  #0 0x80345c948 <smb_panic_s3+152> at /usr/local/lib/libsmbconf.so.0
  #1 0x800a854a5 <smb_panic+53> at /usr/local/lib/libsamba-util.so.0
  #2 0x80102efff <update_num_read_oplocks+303> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #3 0x800f891c0 <create_file_default+47408> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #4 0x800f84468 <create_file_default+27608> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #5 0x800f7ec66 <create_file_default+5078> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #6 0x800f7de65 <create_file_default+1493> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #7 0x8010eb1cc <vfs_default_init+5356> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #8 0x800f9be0c <smb_vfs_call_create_file+556> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #9 0x800ff2f6d <smbd_smb2_request_process_create+13645> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #10 0x800ff025e <smbd_smb2_request_process_create+2110> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #11 0x800fdc14b <smbd_smb2_request_dispatch+5499> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #12 0x800fe4469 <smbd_smb2_process_negprot+16761> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #13 0x800fe3a86 <smbd_smb2_process_negprot+14230> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #14 0x804e6a2d9 <tevent_req_set_cleanup_fn+5705> at /usr/local/lib/libtevent.so.0
  #15 0x804e66ff2 <_tevent_loop_once+114> at /usr/local/lib/libtevent.so.0
  #16 0x804e6a367 <tevent_req_set_cleanup_fn+5847> at /usr/local/lib/libtevent.so.0
  #17 0x800fbad90 <smbd_process+3872> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #18 0x40d80e <main+20910> at /usr/local/sbin/smbd
  #19 0x804e6a2d9 <tevent_req_set_cleanup_fn+5705> at /usr/local/lib/libtevent.so.0
  #20 0x804e66ff2 <_tevent_loop_once+114> at /usr/local/lib/libtevent.so.0
  #21 0x804e6a367 <tevent_req_set_cleanup_fn+5847> at /usr/local/lib/libtevent.so.0
  #22 0x40c21f <main+15295> at /usr/local/sbin/smbd
  #23 0x40a79f <main+8511> at /usr/local/sbin/smbd
  #24 0x406ecf <_start+367> at /usr/local/sbin/smbd
[2017/05/29 09:49:01.482034,  0] ../source3/lib/util.c:803(smb_panic_s3)
  smb_panic(): calling panic action [/usr/local/libexec/samba/samba-backtrace]
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
ptrace: Operation not permitted.
/mnt/VOL1/Storage/2173: No such file or directory.
/usr/local/etc/samba.gdb:3: Error in sourced command file:
set logging: Permission denied.
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
ptrace: Operation not permitted.
/mnt/VOL1/Storage/2176: No such file or directory.
/usr/local/etc/samba.gdb:3: Error in sourced command file:
set logging: Permission denied.
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
ptrace: Operation not permitted.
/mnt/VOL1/Storage/2181: No such file or directory.
/usr/local/etc/samba.gdb:3: Error in sourced command file:
set logging: Permission denied.
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied


I'm wondering if anyone is experience the same issues?
Build: FreeNAS-9.10.2-U4 (27ae72978)


Regards
Morpheus187
 
Last edited by a moderator:
anodos

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,554
Morpheus187 said:
Hi there
EDIT: Title should be: smbd exited on signal 6 ( not in signal 6 ) I don't know how to change it now :(

I've recently updated my freenas to patch the security hole in samba and now I've got some strange samba crashes. I haven't noticed any abnornmal behaviour, everything still seems to work.

dmesg:

Code:
pid 68779 (smbd), uid 0: exited on signal 6 (core dumped)
pid 27798 (smbd), uid 0: exited on signal 6 (core dumped)
pid 4798 (smbd), uid 0: exited on signal 6 (core dumped)
pid 97903 (smbd), uid 0: exited on signal 6 (core dumped)


/var/log/samba/log.smbd

Code:
[2017/05/29 04:17:52.019763,  1] ../source3/smbd/service.c:502(make_connection_snum)
  create_connection_session_info failed: NT_STATUS_ACCESS_DENIED
[2017/05/29 04:17:52.020704,  1] ../source3/smbd/service.c:502(make_connection_snum)
  create_connection_session_info failed: NT_STATUS_ACCESS_DENIED
[2017/05/29 04:17:52.021183,  1] ../source3/smbd/service.c:502(make_connection_snum)
  create_connection_session_info failed: NT_STATUS_ACCESS_DENIED
[2017/05/29 09:49:01.474835,  0] ../source3/smbd/oplock.c:178(update_num_read_oplocks)
  PANIC: assert failed at ../source3/smbd/oplock.c(178): lease_type_is_exclusive(e_lease_type)
[2017/05/29 09:49:01.477005,  0] ../source3/lib/util.c:791(smb_panic_s3)
  PANIC (pid 4798): assert failed: lease_type_is_exclusive(e_lease_type)
[2017/05/29 09:49:01.481892,  0] ../source3/lib/util.c:902(log_stack_trace)
  BACKTRACE: 25 stack frames:
  #0 0x80345c948 <smb_panic_s3+152> at /usr/local/lib/libsmbconf.so.0
  #1 0x800a854a5 <smb_panic+53> at /usr/local/lib/libsamba-util.so.0
  #2 0x80102efff <update_num_read_oplocks+303> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #3 0x800f891c0 <create_file_default+47408> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #4 0x800f84468 <create_file_default+27608> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #5 0x800f7ec66 <create_file_default+5078> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #6 0x800f7de65 <create_file_default+1493> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #7 0x8010eb1cc <vfs_default_init+5356> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #8 0x800f9be0c <smb_vfs_call_create_file+556> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #9 0x800ff2f6d <smbd_smb2_request_process_create+13645> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #10 0x800ff025e <smbd_smb2_request_process_create+2110> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #11 0x800fdc14b <smbd_smb2_request_dispatch+5499> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #12 0x800fe4469 <smbd_smb2_process_negprot+16761> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #13 0x800fe3a86 <smbd_smb2_process_negprot+14230> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #14 0x804e6a2d9 <tevent_req_set_cleanup_fn+5705> at /usr/local/lib/libtevent.so.0
  #15 0x804e66ff2 <_tevent_loop_once+114> at /usr/local/lib/libtevent.so.0
  #16 0x804e6a367 <tevent_req_set_cleanup_fn+5847> at /usr/local/lib/libtevent.so.0
  #17 0x800fbad90 <smbd_process+3872> at /usr/local/lib/samba/libsmbd-base-samba4.so
  #18 0x40d80e <main+20910> at /usr/local/sbin/smbd
  #19 0x804e6a2d9 <tevent_req_set_cleanup_fn+5705> at /usr/local/lib/libtevent.so.0
  #20 0x804e66ff2 <_tevent_loop_once+114> at /usr/local/lib/libtevent.so.0
  #21 0x804e6a367 <tevent_req_set_cleanup_fn+5847> at /usr/local/lib/libtevent.so.0
  #22 0x40c21f <main+15295> at /usr/local/sbin/smbd
  #23 0x40a79f <main+8511> at /usr/local/sbin/smbd
  #24 0x406ecf <_start+367> at /usr/local/sbin/smbd
[2017/05/29 09:49:01.482034,  0] ../source3/lib/util.c:803(smb_panic_s3)
  smb_panic(): calling panic action [/usr/local/libexec/samba/samba-backtrace]
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
ptrace: Operation not permitted.
/mnt/VOL1/Storage/2173: No such file or directory.
/usr/local/etc/samba.gdb:3: Error in sourced command file:
set logging: Permission denied.
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
ptrace: Operation not permitted.
/mnt/VOL1/Storage/2176: No such file or directory.
/usr/local/etc/samba.gdb:3: Error in sourced command file:
set logging: Permission denied.
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied
ptrace: Operation not permitted.
/mnt/VOL1/Storage/2181: No such file or directory.
/usr/local/etc/samba.gdb:3: Error in sourced command file:
set logging: Permission denied.
/usr/local/libexec/samba/samba-backtrace: cannot create /var/log/samba4/samba.backtraces: Permission denied


I'm wondering if anyone is experience the same issues?
Build: FreeNAS-9.10.2-U4 (27ae72978)


Regards
Morpheus187
Click to expand...

What client? Does testparm throw any errors?
 
M

Morpheus187

Explorer
Joined
Mar 11, 2016
Messages
61
testparm throws no errors and I have 2 clients connected, I cannot tell which one is causing the errors because I haven't noticed any interruptions

Connected Clients ( smbstatus )
Code:
Samba version 4.5.10
PID  Username  Group  Machine  Protocol Version  Encryption  Signing
----------------------------------------------------------------------------------------------------------------------------------------
99073  morpheus  morpheus  192.168.1.13 (ipv4:192.168.1.13:50382)  SMB3_11  -  partial(AES-128-CMAC)
90028  morpheus  morpheus  192.168.1.18 (ipv4:192.168.1.18:43252)  SMB2_10  -  -

Service  pid  Machine  Connected at  Encryption  Signing
---------------------------------------------------------------------------------------------
Backup  99073  192.168.1.13  Mon May 29 13:01:04 2017 CEST  -  -
Storage  90028  192.168.1.18  Mon May 29 09:49:03 2017 CEST  -  -
Storage  99073  192.168.1.13  Mon May 29 13:01:04 2017 CEST  -  -


I think the problem might come from the SMB2_10 connection, the "connected at" time is the same as the error time in the smbd logfile.
 
M

Morpheus187

Explorer
Joined
Mar 11, 2016
Messages
61
This is the content of smb4.conf
Code:
[global]
  server max protocol = SMB3_11
  interfaces = 127.0.0.1 192.168.1.5
  bind interfaces only = yes
  encrypt passwords = yes
  dns proxy = no
  strict locking = no
  oplocks = yes
  deadtime = 15
  max log size = 51200
  max open files = 1883936
  logging = file
  load printers = no
  printing = bsd
  printcap name = /dev/null
  disable spoolss = yes
  getwd cache = yes
  guest account = nobody
  map to guest = Bad User
  obey pam restrictions = yes
  directory name cache size = 0
  kernel change notify = no
  panic action = /usr/local/libexec/samba/samba-backtrace
  nsupdate command = /usr/local/bin/samba-nsupdate -g
  server string = FreeNAS Server
  ea support = yes
  store dos attributes = yes
  lm announce = yes
  time server = yes
  acl allow execute always = true
  dos filemode = yes
  multicast dns register = no
  domain logons = no
  local master = yes
  idmap config *: backend = tdb
  idmap config *: range = 90000001-100000000
  server role = standalone
  netbios name = FREENAS
  workgroup = WORKGROUP
  security = user
  pid directory = /var/run/samba
  create mask = 0666
  directory mask = 0777
  client ntlmv2 auth = yes
  dos charset = CP437
  unix charset = UTF-8
  log level = 1


[Backup]
  path = /mnt/VOL1/backup
  printable = no
  veto files = /.snapshot/.windows/.mac/.zfs/
  writeable = yes
  browseable = yes
  vfs objects = zfs_space zfsacl streams_xattr aio_pthread
  hide dot files = yes
  guest ok = no
  nfs4:mode = special
  nfs4:acedup = merge
  nfs4:chown = true
  zfsacl:acesort = dontcare


[Storage]
  path = /mnt/VOL1/Storage
  printable = no
  veto files = /.snapshot/.windows/.mac/.zfs/
  writeable = yes
  browseable = yes
  shadow:snapdir = .zfs/snapshot
  shadow:sort = desc
  shadow:localtime = yes
  shadow:format = auto-%Y%m%d.%H%M-1m
  shadow:snapdirseverywhere = yes
  vfs objects = shadow_copy2 zfs_space zfsacl streams_xattr aio_pthread
  hide dot files = yes
  guest ok = no
  nfs4:mode = special
  nfs4:acedup = merge
  nfs4:chown = true
  zfsacl:acesort = dontcare
 
M

Morpheus187

Explorer
Joined
Mar 11, 2016
Messages
61
That's a windows 7 vm I use to connect to from outside.
 
C

Cellobita

Contributor
Joined
Jul 15, 2011
Messages
107
I have been experiencing the exact same issue, on three boxes that never had this problem before the update (specs are very different among them, so it doesn't feel like a hardware issue).
 
C

Cellobita

Contributor
Joined
Jul 15, 2011
Messages
107
This is from the "daily security run output" mail from this morning:

Code:
servidor.xxxxx.lan kernel log messages:

pid 67671 (smbd), uid 0: exited on signal 6 (core dumped)
pid 67925 (smbd), uid 0: exited on signal 6 (core dumped)
pid 70026 (smbd), uid 0: exited on signal 6 (core dumped)
pid 71209 (smbd), uid 0: exited on signal 6 (core dumped)
pid 71220 (smbd), uid 0: exited on signal 6 (core dumped)
pid 71231 (smbd), uid 0: exited on signal 6 (core dumped)
pid 71242 (smbd), uid 0: exited on signal 6 (core dumped)
pid 76103 (smbd), uid 0: exited on signal 6 (core dumped)
pid 69238 (smbd), uid 0: exited on signal 6 (core dumped)

-- End of security output --


The others are even worse, probably because they are larger boxes, servicing more users; of the three, a couple use AD authentication and permissions, the other standard Linux ACL's.
 
Last edited:
anodos

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,554
Cellobita said:
This is from the "daily security run output" mail from this morning:

Code:
servidor.xxxxx.lan kernel log messages:

pid 67671 (smbd), uid 0: exited on signal 6 (core dumped)
pid 67925 (smbd), uid 0: exited on signal 6 (core dumped)
pid 70026 (smbd), uid 0: exited on signal 6 (core dumped)
pid 71209 (smbd), uid 0: exited on signal 6 (core dumped)
pid 71220 (smbd), uid 0: exited on signal 6 (core dumped)
pid 71231 (smbd), uid 0: exited on signal 6 (core dumped)
pid 71242 (smbd), uid 0: exited on signal 6 (core dumped)
pid 76103 (smbd), uid 0: exited on signal 6 (core dumped)
pid 69238 (smbd), uid 0: exited on signal 6 (core dumped)

-- End of security output --


The others are even worse, probably because they are larger boxes, servicing more users.
Click to expand...

Others? Is this occurring across multiple FreeNAS servers?
 
C

Cellobita

Contributor
Joined
Jul 15, 2011
Messages
107
Yes, multiple boxes at different sites - and all started having this issue after U4 and Samba 4.5.10
 
anodos

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,554
Cellobita said:
Yes, multiple boxes at different sites - and all started having this issue after U4 and Samba 4.5.10
Click to expand...

File a bug report at bugs.freenas.org and post a link to it here. Mention that it's affecting multiple boxes at different sites. You'll probably be prompted to upload a debug file. If you do this, you may want to flag the bug report as private.
 
Last edited:
anodos

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,554
By the way, if you've rolled back to U3 you should either set the auxiliary parameter nt pipe support = no or configure the datasets you're sharing to have the ZFS parameter "exec = off" through the command zfs set exec = off Tank/Share.
 
C

Cellobita

Contributor
Joined
Jul 15, 2011
Messages
107
Apart from the kernel errors logged, there's been no complaint from my users - I'll leave things as they are, for now.
 
anodos

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,554
Try setting the following parameter under "services" -> "SMB". smb2 leases = no. There is currently a bug in the oplock code in samba that can cause smbd processes to panic in certain situations involving smb2 leases. Let us know if turning off SMB2 leases fixes the problem.
 
C

Cellobita

Contributor
Joined
Jul 15, 2011
Messages
107
OK, thanks for following this up - I did what you suggested, let's see what happens.

I have also noticed that the "Server Maximum Protocol" is set to SMB2 - is this still the recommended setting as of Samba 4.5.10?
 
anodos

anodos

Sambassador
iXsystems
Joined
Mar 6, 2014
Messages
9,554
Cellobita said:
Thanks again!
Click to expand...

I misread the discussion in the samba bugtracker. It seems like the problem may only be fixed by applying the patch: https://attachments.samba.org/attachment.cgi?id=13241

If disabling SMB2 leases doesn't work (the samba devs say it won't), then you might try globally disabling oplocks (this may seriously hurt server performance).

@Kris Moore @JoshDW19 you might want put the above patch into FreeNAS 10 / FreeNAS 11. I'm not sure when the next samba release will come down the pipeline, but this is already affecting multiple users. I threw a private bug report in the FreeNAS bugtracker.

For reference, the samba bug report is here: https://bugzilla.samba.org/show_bug.cgi?id=12798#c13]]
 
C

Cellobita

Contributor
Joined
Jul 15, 2011
Messages
107
Disabling oplocks is not feasible for me; since nobody's complained this far, I'll just deal with the daily kernel errors on my mailbox.
 
Status
Not open for further replies.

Similar threads

LaserAllan
  • Locked
FreeNAS and Veeam 9.5
Replies
8
Views
5K
LaserAllan
LaserAllan
F
freenas11.2 u5 smb_panic problem
Replies
15
Views
3K
anodos
anodos
O
smbd (core dumped) again and again
Replies
7
Views
1K
ObiTobi
O
I
Win10 Open file dialog halt, smbd crash and cleanup
Replies
5
Views
1K
irep
I
Q
  • Locked
SOLVED smbd (core dumped)
Replies
2
Views
4K
Quinniedid
Q
Top