Upgrading to 9.2 B2 breaks AD

[LostInIgnorance]

I am currently running a 2008 r2 server with freenas 9.1.1 x64. I took the liberty to backup my settings on both machines before trying this ;)
The FreeNAS box is nothing special. an old server chassis I received from a friend who sells backup equipment. It is a Supermicro board and 12 bay (8bays populated) chassis with 8GB ram, a Intel Pentium Dual CPU E2180 @ 2.00GHz, and a sata RAID card (RAID 5 on 2TB drives). Nothing really special. It is joined to the DC via the AD integration. DC is running virtualized on a different server physical server.
I am stating here, everything in the above configuration is running fine. No issues to speak of.

I decided to try out the new 9.2 on a gui upgrade. Update went well, no errors. Took a bit longer to boot, but that was expected. It first asked me to set up the root password. Set that up. Then tried accessing the server via network discover. No server exists. Try the manual server name. Can't find it.
At this point I am thinking its just something simple, the AD integration service didn't start. Sure enough, it wasn't. Tried starting it. "Cannot start service." Looked at the time and resynced just to be safe. Pinged the dns entry of my server. Everything I am doing is showing no errors. Tried flushing from the DC with netdom and a kdestroy on the NAS thinking something was out of sync. Still can't get the service to start after multiple reboots. I was pretty much getting the same errors as described in this thread. It was pretty much giving me errors like I never was joined. Messed around with the tests described in the documentation. It was showing that no users were on the system. Tried resyncing from the web gui (settings > advanced > rebuild ldap\ad cache). Could not get it to sync what-so-ever.
Decided to tackle it the next day. Didn't change anything on the DC. Downloaded an ISO of 9.1.1 x64. Reinstalled it on the NAS box. Recovered from the backup I took of the config. BAM, back up and running.
What changed or has changed in the handling of AD on 9.2?

 

[cyberjock]

I'm not sure of any specific changes to AD that would cause your problem.

But holy smokes, you are much too low on RAM and shouldn't be using RAID with ZFS. You're supposed to have 8GB of RAM minimum plus 2GB of RAM for AD machines. You don't have that. Additionally, there is the thumbrule that you should have 1GB of RAM per TB of disk storage(which is 10GB) and you don't have that either. I'd upgrade to at least 16GB before looking into this further. When you don't have enough RAM services often fail to start, randomly crash, or cause other odd behavior that makes you scratch your head. Not to mention you aren't using ECC RAM which is a big no-no.

Also, using RAID5 with ZFS is very dangerous. That should be fixed pronto!

To be honest, all of those mistakes are noobie mistakes. It does make me question what other serious mistakes you've made that may be affecting your situation.

 

[LostInIgnorance]

I'd upgrade to at least 16GB before looking into this further.

Unfortunately the hardware, as I am aware, will not allow more than 8GB (4 x 2). I am using ECC memory in this system as it is an old server board. I am also using this in a home environment for mass storage so am not that worried about performance as it is used as a dumping ground for non active files.

Also, using RAID5 with ZFS is very dangerous. That should be fixed pronto!

The RAID is managed by the card that is installed on the machine. I'm not worried about it being managed by freenas as it is managed independently.

I would like to figure out the issues as they seem to be attributed to the update more than anything else. I have not had any issues through the 8 updates and then going to the 9 updates until now. I was going to try the update from the ISO option next to see if I have the same issues or if it is just the GUI update that caused the issue.

 

[cyberjock]

Unfortunately the hardware, as I am aware, will not allow more than 8GB (4 x 2). I am using ECC memory in this system as it is an old server board. I am also using this in a home environment for mass storage so am not that worried about performance as it is used as a dumping ground for non active files.

Some people seem to think that the CPU doesn't use ECC RAM(Intels datasheet doesn't mention it) so there's a good chance you may be using ECC RAM, but the ECC feature isn't being used. What is your motherboard model?

The RAID is managed by the card that is installed on the machine. I'm not worried about it being managed by freenas as it is managed independently.

And that's where you do not understand. FreeNAS DOES expect to manage its drives. It is engineered into ZFS and cannot be "disabled". What happens when 2 managers try to tell you what to do? You get nothing done right. We've gone over mixing ZFS with hardware RAID and it always ends with comments like "what a moron" so I won't go any further. Do your own home if you wish and keep it how you wish. You'll get no sympathy when you do what you did and come back later asking for help with data recovery. There's a link somewhere of someone that bought a rather expensive server and then later started losing data. It was later found that he bought a machine from a company that didn't know how ZFS worked and put a rather expensive RAID controller in a system that not only cost them money they didn't need to spend, but also was the single cause for a loss of data for the customer. So feel free to keep doing what you are doing. Just because its working doesn't mean its right. It's nothing more than a ticking timebomb in your server. And if it goes *boom* you're going to get some chuckles for having your server in that configuration to begin with. The FreeNAS manual covers this clearly, so feel free to read it.

I would like to figure out the issues as they seem to be attributed to the update more than anything else. I have not had any issues through the 8 updates and then going to the 9 updates until now. I was going to try the update from the ISO option next to see if I have the same issues or if it is just the GUI update that caused the issue.

Unfortunately, as stuff gets updated and more features are added more RAM will be needed. It could be a RAM problem or it might be a bug with FreeNAS. Unfortunately, if your system requirements are too low for what the expected norm is, most people are going to ignore you because its not worth spending developer resources trying to fix a problem that might not be a problem at all. Just look at how many people show up with less than the minimum required RAM and complain about performance. We tell them to buzz off or upgrade.

I wouldn't expect the ISO option to have an impact on the outcome. The ISO does nothing more than install the image that you install with a GUI upgrade.

 

[Tud]

Did you find a solution to this? I have the same issue (different hardware) I went from 9.1.1 to 9.2 and now Directory Services wont start, It doesnt really show anything obvious in the logs, if anyone here can help let me know what ifo you need me to send
Cheers

 

[John Hixson]

For anyone of you having trouble getting Active Directory issues, I would try the following things to troubleshoot it:

sqlite3 /data/freenas-v1.db "update services_services set srv_enable = 1 where srv_service = 'directoryservice'"
service ix-kerberos start
service ix-kinit start
service ix-kinit status
echo $? # this should be 0
klist # this should show kerberos tickets

service ix-pam start
service ix-nsswitch start

# For versions of FreeNAS 9.2.1.8 and earlier
service ix-samba start

# For 9.2.1.9
service ix-pre-samba start

service ix-activedirectory start
service ix-activedirectory status
echo $? # this should be 0
service samba_server restart

# For 9.2.1.9
service ix-post-samba start

At this point, you should have all AD users and groups when doing a getent passwd and getent group.

wbinfo -t
wbinfo -u
wbinfo -g

If at any point in the above series of commands a failure occurs, then you've isolated the problem and please report back to this forum with what was found ;-)

Hope this helps.

- John

 

[Tud]

For anyone of you having trouble getting Active Directory issues, I would try the following things to troubleshoot it:

sqlite /data/freenas-v1.db "update services_services set srv_enable = 1 where srv_service = 'directoryservice'"
service ix-kerberos start
service ix-kinit start
service ix-kinit status
echo $? # this should be 0
klist # this should show kerberos tickets

service ix-pam start
service ix-nsswitch start

service ix-samba start
service ix-activedirectory start
service ix-activedirectory status
echo $? # this should be 0
service samba restart

At this point, you should have all AD users and groups when doing a getent passwd and getent group.

wbinfo -t
wbinfo -u
wbinfo -g

Thanks John, that is probably the most useful piece of troubleshooting info I have found form all the threads I have looked in, this got me what I needed.
Turned out to be damn clock skew, I was so busy making sure the time was the same, I wasn't looking to see that I was out by a year!

Thanks for your help I will definitely keep this on file

Cheers

 

[mmerlone]

Hi,

Those service ix-pam start and service ix-nsswitch start fails to me. After start'ing those both, I went to try their status and got no good news:
[root@freenas ~]# service ix-pam status
ix-pam is not running.
[root@freenas ~]# service ix-nsswitch status
ix-nsswitch is not running.
[root@freenas ~]#

Machine was able to join AD, wbinfo -u return successful, all seems fine, but getent passwd does not return AD users, obviously it lacks pam and nsswitch. Anything else I should check?

Regards.

 

[StevenA]

Me too! Me too!

Oh wait, I'm on the 9.2.1 RELEASE version, but my AD is still messed up.

I originally had a spat with 9.2.0 (or was it 9.1?) to get AD working, I think the end fix was adding the CIF parameters in there.

But now I upgraded to 9.2.1 and AD won't start again and I'm getting a completely different error, which for some odd reason, nobody else has it!

After "service ix-activedirectory start" I get, "Failed to join domain: failed to lookup DC info for domain 'ABC123.COM' over rpc: NT_STATUS_CONNECTION_RESET"

Checked all the boxes, white spaces, etc etc, but still the same error, is there something new again in 9.2.1 that I'm missing?

Thanks,

Steven

 

[Angus M]

Hi John

I am having trouble starting active directory on a new install of FreeNas 9.2.1.8. I have used your troubleshooting instructions above but when I enter

service ix-activedirectory start

I get the following

Failed to join domain: failed to lookup DC info for domain 'am.local' over rpc:
Logon failure
Failed to leave domain: Unable to fetch domain sid: are we joined?
winbindd not running? (check /var/run/samba/winbindd.pid).
smbd not running? (check /var/run/samba/smbd.pid).
nmbd not running? (check /var/run/samba/nmbd.pid).
[root@freenas2 ~]#

Anyone know how to correct this problem?

Many thanks

Angus

 

[praetorian46]

Hello John,
I am having a similar issue with Active Directory and FreeNAS 9.3-BETA

My problem is that the users and groups don't show up in the GUI when I try to set the permissions of a dataset
I have followed all of your steps above and here is what I found:
ix-samba doesn't seem to exist in 9.3-BETA:

Code:

# service ix-samba start
ix-samba does not exist in /etc/rc.d or the local startup
directories (/etc/ix.rc.d /usr/local/etc/rc.d)

And I get a strange error when I start or restart:

Code:

# service ix-activedirectory start
Using short domain name -- HILL
Joined 'HILL-SAN02' to dns domain 'Hill.Local'
Error: no such column: cifs_srv_homedir

I've googled for "Error: no such column: cifs_srv_homedir" and I get zero results...
Would this have something to do with "UNIX extensions" being enabled or disabled for the Active Directory Service?

getent passwd and getent group don't return any domain users or groups.

But all three wbinfo commands return the correct domain users and groups.

I'm not sure what else to do.

Any clues would be greatly appreciated.
Thank you,
Cody Hill

 

[meelos]

I am experiencing two of the same exact things as praetorian46, the error about ix-samba not existing, and the wbinfo commands working, but not the getent ones. (but I'm running a rev back @ FreeNAS-9.2.1.9) :/

 

[Thomymaster]

Hi

After upgrading from 9.2.1.6 i have the same problem as Angus M. The service ix-activedirectory fails to start und thus samba also. I reverted back to 9.2.1.6 where it works and setup a test system (where i could confirm the issue).
I opened a Bug https://bugs.freenas.org/issues/6907

 

[macxs]

I was on 9.2.1.8. After changing the password of the domain user that joined Freenas initially, I couldn't re-join. I upgraded to 9.3 STABLE but that didn't help. I followed the instructions here. Most likely there is something wrong with the kerberos service. I can always get tickets manually but the service does not start.

service ix-kerberos start
service ix-kinit start
service ix-kinit status
echo $?
1​

winbindd is terminating immedeately after start. ( "../source3/nmbd/nmbd.c:57(terminate)" )

Where can I view the kerberos logs?


Thanks!
Marco

 

[Thomymaster]

By the way, the followup ticket is

https://bugs.freenas.org/issues/7034