I have been trying to mount a shared dataset, with NFS v4 turned on, and sec set to krb5, krb5i or krb5p, without sys.
I have tried mounting from a RHEL6 client, and also from another FreeNAS 9.4 box.
From the FreeNas client box, I can mount using nfsv3, but can't do any directory listing (getting Permission denied). Using nfsv4, getting nfsv4 err=10016. mount_nfs: ... : Input/output error.
However, my main concern is mounting from RHEL6 because majority of my hosts are running RHEL 5 or RHEL6.
Does anything has a working example?
Using tcpdump and wireshark, I see the following:
(My configurations are explained after these 2 packet traces)
Frame 23: 718 bytes on wire (5744 bits), 718 bytes captured (5744 bits)
Ethernet II, Src: Qumranet_15:21:02 (00:1a:4a:15:21:02), Dst: All-HSRP-routers_3d (00:00:0c:07:ac:3d)
Internet Protocol Version 4, Src: 172.21.49.85 (172.21.49.85), Dst: 172.21.32.81 (172.21.32.81)
Transmission Control Protocol, Src Port: 45012 (45012), Dst Port: nfs (2049), Seq: 1, Ack: 1, Len: 652
Remote Procedure Call, Type:Call XID:0xda09dc45
Fragment header: Last fragment, 648 bytes
1... .... .... .... .... .... .... .... = Last Fragment: Yes
.000 0000 0000 0000 0000 0010 1000 1000 = Fragment Length: 648
XID: 0xda09dc45 (3658079301)
Message Type: Call (0)
RPC Version: 2
Program: NFS (100003)
Program Version: 4
Procedure: NULL (0)
[The reply to this request is in frame 25]
Credentials
Flavor: RPCSEC_GSS (6)
Length: 20
GSS Version: 1
GSS Procedure: RPCSEC_GSS_INIT (1)
GSS Sequence Number: 0
GSS Service: rpcsec_gss_svc_none (1)
GSS Context
GSS Context Length: 0
GSS Context: <MISSING>
Verifier
Flavor: AUTH_NULL (0)
Length: 0
Network File System
[Program Version: 4]
[V4 Procedure: NULL (0)]
GSS Token: 000002456082024106092a864886f71201020201006e8202...
GSS Token Length: 581
GSS-API Generic Security Service Application Program Interface
OID: 1.2.840.113554.1.2.2 (KRB5 - Kerberos 5)
krb5_blob: 01006e8202303082022ca003020105a10302010ea2070305...
krb5_tok_id: KRB5_AP_REQ (0x0001)
Kerberos AP-REQ
Pvno: 5
MSG Type: AP-REQ (14)
Padding: 0
APOptions: 20000000 (Mutual required)
0... .... .... .... .... .... .... .... = reserved: RESERVED bit off
.0.. .... .... .... .... .... .... .... = Use Session Key: Do NOT use the session key to encrypt the ticket
..1. .... .... .... .... .... .... .... = Mutual required: MUTUAL authentication is REQUIRED
Ticket
Tkt-vno: 5
Realm: PCDSN
Server Name (Service and Host): nfs/psnfs1.pcdsn
Name-type: Service and Host (3)
Name: nfs
Name: psnfs1.pcdsn
enc-part des-cbc-crc
Encryption type: des-cbc-crc (1)
Kvno: 1
enc-part: 04083acff1a30163a376ad8a3ea190543e13274036ccfc6d...
Authenticator des-cbc-crc
Encryption type: des-cbc-crc (1)
Authenticator data: 3c0adb6fabe80574ed3a5db4601be9309c05dc05e5d0a50e...
Frame 25: 90 bytes on wire (720 bits), 90 bytes captured (720 bits)
Ethernet II, Src: Cisco_9b:27:00 (00:1d:71:9b:27:00), Dst: Qumranet_15:21:02 (00:1a:4a:15:21:02)
Internet Protocol Version 4, Src: 172.21.32.81 (172.21.32.81), Dst: 172.21.49.85 (172.21.49.85)
Transmission Control Protocol, Src Port: nfs (2049), Dst Port: 45012 (45012), Seq: 1, Ack: 653, Len: 24
Remote Procedure Call, Type:Reply XID:0xda09dc45
Fragment header: Last fragment, 20 bytes
1... .... .... .... .... .... .... .... = Last Fragment: Yes
.000 0000 0000 0000 0000 0000 0001 0100 = Fragment Length: 20
XID: 0xda09dc45 (3658079301)
Message Type: Reply (1)
[Program: NFS (100003)]
[Program Version: 4]
[Procedure: NULL (0)]
Reply State: denied (1)
[This is a reply to a request in frame 23]
[Time from request: 0.000287000 seconds]
Reject State: AUTH_ERROR (1)
Auth State: client must begin new session (2)
My /etc/exports file:
[root@psnfs1] /etc/rc.d# cat /etc/exports
V4: /
/mnt/datapool/ling -alldirs -sec=krb5 -network 172.21.0.0/16
/mnt/datapool/ling -alldirs -sec=krb5 127.0.0.1
Server:
[root@psnfs1] /etc/rc.d# host psnfs1
psnfs1.pcdsn has address 172.21.32.81
Client:
[root@psnfs1] /etc/rc.d# host psana107
psana107.pcdsn has address 172.21.49.85
My domain (private) is pcdsn.
Kerberos REALM is PCDSN
I have set up host and ftp kerberos principal in the keytab files for both machines, using des-cbc-crc only. I have tried other encryption type but made no difference.
I have rpc.gssd, rpc.idmapd running on the RHEL6 client.
I can get a lot of logs on my RHEL6 client, but unfortunately I am not sure how to get more logs on the FeeeNAS box. I have set syslog-ng to send everything the /var/log/all.log but there is just nothing much to see. I also tried running gssd on the FreeNAS box with -d -d -d and also nothing was shown. If I try to run truss on nfsd: server process , it dies right away. nfsd: master doesn't show anything.
Thanks,
I have tried mounting from a RHEL6 client, and also from another FreeNAS 9.4 box.
From the FreeNas client box, I can mount using nfsv3, but can't do any directory listing (getting Permission denied). Using nfsv4, getting nfsv4 err=10016. mount_nfs: ... : Input/output error.
However, my main concern is mounting from RHEL6 because majority of my hosts are running RHEL 5 or RHEL6.
Does anything has a working example?
Using tcpdump and wireshark, I see the following:
(My configurations are explained after these 2 packet traces)
Frame 23: 718 bytes on wire (5744 bits), 718 bytes captured (5744 bits)
Ethernet II, Src: Qumranet_15:21:02 (00:1a:4a:15:21:02), Dst: All-HSRP-routers_3d (00:00:0c:07:ac:3d)
Internet Protocol Version 4, Src: 172.21.49.85 (172.21.49.85), Dst: 172.21.32.81 (172.21.32.81)
Transmission Control Protocol, Src Port: 45012 (45012), Dst Port: nfs (2049), Seq: 1, Ack: 1, Len: 652
Remote Procedure Call, Type:Call XID:0xda09dc45
Fragment header: Last fragment, 648 bytes
1... .... .... .... .... .... .... .... = Last Fragment: Yes
.000 0000 0000 0000 0000 0010 1000 1000 = Fragment Length: 648
XID: 0xda09dc45 (3658079301)
Message Type: Call (0)
RPC Version: 2
Program: NFS (100003)
Program Version: 4
Procedure: NULL (0)
[The reply to this request is in frame 25]
Credentials
Flavor: RPCSEC_GSS (6)
Length: 20
GSS Version: 1
GSS Procedure: RPCSEC_GSS_INIT (1)
GSS Sequence Number: 0
GSS Service: rpcsec_gss_svc_none (1)
GSS Context
GSS Context Length: 0
GSS Context: <MISSING>
Verifier
Flavor: AUTH_NULL (0)
Length: 0
Network File System
[Program Version: 4]
[V4 Procedure: NULL (0)]
GSS Token: 000002456082024106092a864886f71201020201006e8202...
GSS Token Length: 581
GSS-API Generic Security Service Application Program Interface
OID: 1.2.840.113554.1.2.2 (KRB5 - Kerberos 5)
krb5_blob: 01006e8202303082022ca003020105a10302010ea2070305...
krb5_tok_id: KRB5_AP_REQ (0x0001)
Kerberos AP-REQ
Pvno: 5
MSG Type: AP-REQ (14)
Padding: 0
APOptions: 20000000 (Mutual required)
0... .... .... .... .... .... .... .... = reserved: RESERVED bit off
.0.. .... .... .... .... .... .... .... = Use Session Key: Do NOT use the session key to encrypt the ticket
..1. .... .... .... .... .... .... .... = Mutual required: MUTUAL authentication is REQUIRED
Ticket
Tkt-vno: 5
Realm: PCDSN
Server Name (Service and Host): nfs/psnfs1.pcdsn
Name-type: Service and Host (3)
Name: nfs
Name: psnfs1.pcdsn
enc-part des-cbc-crc
Encryption type: des-cbc-crc (1)
Kvno: 1
enc-part: 04083acff1a30163a376ad8a3ea190543e13274036ccfc6d...
Authenticator des-cbc-crc
Encryption type: des-cbc-crc (1)
Authenticator data: 3c0adb6fabe80574ed3a5db4601be9309c05dc05e5d0a50e...
Frame 25: 90 bytes on wire (720 bits), 90 bytes captured (720 bits)
Ethernet II, Src: Cisco_9b:27:00 (00:1d:71:9b:27:00), Dst: Qumranet_15:21:02 (00:1a:4a:15:21:02)
Internet Protocol Version 4, Src: 172.21.32.81 (172.21.32.81), Dst: 172.21.49.85 (172.21.49.85)
Transmission Control Protocol, Src Port: nfs (2049), Dst Port: 45012 (45012), Seq: 1, Ack: 653, Len: 24
Remote Procedure Call, Type:Reply XID:0xda09dc45
Fragment header: Last fragment, 20 bytes
1... .... .... .... .... .... .... .... = Last Fragment: Yes
.000 0000 0000 0000 0000 0000 0001 0100 = Fragment Length: 20
XID: 0xda09dc45 (3658079301)
Message Type: Reply (1)
[Program: NFS (100003)]
[Program Version: 4]
[Procedure: NULL (0)]
Reply State: denied (1)
[This is a reply to a request in frame 23]
[Time from request: 0.000287000 seconds]
Reject State: AUTH_ERROR (1)
Auth State: client must begin new session (2)
My /etc/exports file:
[root@psnfs1] /etc/rc.d# cat /etc/exports
V4: /
/mnt/datapool/ling -alldirs -sec=krb5 -network 172.21.0.0/16
/mnt/datapool/ling -alldirs -sec=krb5 127.0.0.1
Server:
[root@psnfs1] /etc/rc.d# host psnfs1
psnfs1.pcdsn has address 172.21.32.81
Client:
[root@psnfs1] /etc/rc.d# host psana107
psana107.pcdsn has address 172.21.49.85
My domain (private) is pcdsn.
Kerberos REALM is PCDSN
I have set up host and ftp kerberos principal in the keytab files for both machines, using des-cbc-crc only. I have tried other encryption type but made no difference.
I have rpc.gssd, rpc.idmapd running on the RHEL6 client.
I can get a lot of logs on my RHEL6 client, but unfortunately I am not sure how to get more logs on the FeeeNAS box. I have set syslog-ng to send everything the /var/log/all.log but there is just nothing much to see. I also tried running gssd on the FreeNAS box with -d -d -d and also nothing was shown. If I try to run truss on nfsd: server process , it dies right away. nfsd: master doesn't show anything.
Thanks,