Certificates unusable

[mururoa]

Okay the Certificate part of Truenas is broken.
And it's broken since several releases. At first I thought it was a problem with navigators not accepting private CA anymore but it's not.
I used several certificates from the same Truenas internal CA and it used to work until some months agos. That CA was created in the Freenas times and is valid until 2026 or like.
At least the default setup is broken and maybe it is now needed to specify some options that are not defaults.
I tried hard today issuing a certificate for an internal lab server but no way to get it working. Certificate is allways stated as invalid for browsers (firefox and chrome on linux, windows and android).
So I gave a try on pfsense certificate authority and it worked like a charm. Same server, same CN, same SAN, same ... Of course I had to set lifetime of 398 days since Apple forced the rest of the world to use that limit but I had it set on both pfsense and Truenas so that's not the point.
Very easy to reproduce : create a CA, create certificate, export CA, certificate and key. Import CA on browser/OS. Configure server with issued certificate and try to access it. Truenas certificate : invalid, Pfsense : ok.
So the Truenas certificates are somehow broken. Maybe it's only a matter of options that have now to be set and are not set or something else.
So now I have a Pfsense server certificate the Truenas server. What else ?

 

[Patrick M. Hausen]

What steps did you take to make the certificates acceptable in your browser? With every local CA you need to import the CA certificate into your browser (or system wide certificate store) and mark it as trustworthy.

1. Create a new CA on TrueNAS CORE 12.0-U5

I misspelled "hausen.com" in the email address, but who cares? Now we have a CA.

2. Issue a certificate with an "Apple compliant" lifetime of 365 days


3. Set that as the UI certificate in System --> General

4. Export the CA cert, import it into Mac OS keychain, mark it as trustworthy

"Immer Vertrauen" == "always trust"

5. Navigate to the TrueNAS system in the browser and check the cert


So: works as designed in my opinion. What precisely did you do and what precise error messages do you get?

I run my private CA on OPNsense, issuing certs for all TrueNAS systems, ESXi, my UPS, ... but I would not say that TrueNAS' implementation was broken.

 

[invar]

Okay the Certificate part of Truenas is broken.
And it's broken since several releases. At first I thought it was a problem with navigators not accepting private CA anymore but it's not.
I used several certificates from the same Truenas internal CA and it used to work until some months agos. That CA was created in the Freenas times and is valid until 2026 or like.
At least the default setup is broken and maybe it is now needed to specify some options that are not defaults.
I tried hard today issuing a certificate for an internal lab server but no way to get it working. Certificate is allways stated as invalid for browsers (firefox and chrome on linux, windows and android).
So I gave a try on pfsense certificate authority and it worked like a charm. Same server, same CN, same SAN, same ... Of course I had to set lifetime of 398 days since Apple forced the rest of the world to use that limit but I had it set on both pfsense and Truenas so that's not the point.
Very easy to reproduce : create a CA, create certificate, export CA, certificate and key. Import CA on browser/OS. Configure server with issued certificate and try to access it. Truenas certificate : invalid, Pfsense : ok.
So the Truenas certificates are somehow broken. Maybe it's only a matter of options that have now to be set and are not set or something else.
So now I have a Pfsense server certificate the Truenas server. What else ?

Yeah, I noticed something like this as well and figured it was an issue with the wrong settings for the CA creation but after messing with it many times over, I just gave up. None of my browsers would recognize the newly created CA as valid for any of my jails despite being manually imported as a root CA. I gave up and just setup things through Lets Encrypt.

I had an older CA and cert that I had created from FreeNAS 11 that worked but because I changed domains I couldn't use it any longer.

 

[Patrick M. Hausen]

None of my browsers would recognize the newly created CA as valid for any of my jails despite being manually imported as a root CA.

As I demonstrated above this does work as designed. Nonetheless Letsencrypt is preferrable, because you do not need to explicitly import the CA, and if you set up things correctly, you get automatic renewal, too.

 

[invar]

As I demonstrated above this does work as designed. Nonetheless Letsencrypt is preferrable, because you do not need to explicitly import the CA, and if you set up things correctly, you get automatic renewal, too.

Patrick, I'm familiar with how to generate a CA and import it as I had it working perfectly fine on FreeNAS 11. I did it multiple times in fact and successfully utilized the same internal CA and certs in my jails for Emby, Plex and Nextcloud.

I even went so far as to remove the new CA from all my devices, put the old one back in as well as put the old cert back into WebGUI and jails and tested it and confirmed it was working again.

Unfortunately, I couldn't be bothered to troubleshoot much more than I had already because it would've amounted to putting in a bunch of work for a 'hack', whereas instead of doing that, I put in the work to figure out how to get LE to work and to have it auto-renew and redeploy to the WebGUI and all of my jails.

Respectfully, I wouldn't be so quick to dismiss our troubles simply as user error.

 

[Patrick M. Hausen]

What did you do differently compared to my steps above? Because different outcome implies different prerequisites, doesn't it?

I am not "simply" dismissing your troubles, I went through all the steps creating a CA and certificate, changing the certificate of my TrueNAS, etc. in order to help. The steps in my first post work perfectly well in my environment.

Now document your steps and your result so we can identify that bug. "Doesn't work" is not a problem report.

 

[invar]

What did you do differently compared to my steps above? Because different outcome implies different prerequisites, doesn't it?

I am not "simply" dismissing your troubles, I went through all the steps creating a CA and certificate, changing the certificate of my TrueNAS, etc. in order to help. The steps in my first post work perfectly well in my environment.

Now document your steps and your result so we can identify that bug. "Doesn't work" is not a problem report.

Respectfully (again), I appreciate your input but I'm not the OP. I chimed in merely to affirm the OP's experience. I'll defer to you and the OP as far as troubleshooting steps.

I have a working setup but when I was going through this, I generated many, many CAs and certs trying all sorta of different profiles, no profiles, constraints, no constraints, CNs, SANs, key usages, etc. And eventually that's when I gave up. I'm almost certain that one if my variations matches yours.

I'll bow out now but if I find some time I will try redoing it matching your settings and take screenshots. Assuming the OP hasn't yet.

 

[mururoa]

Okay, I know how certificates and CA work, thanks. Obviously otherwise the Pfsense CA would not work better than Truenas CA.
Let's encrypt is not an option since I use the certificates for lan server with local IPs.
I'll try to see if the certificates are considered valid using openssl commands. Maybe it may point to what is invalid.

Edit : ok check done. openssl is coherent with browsers : invalid certificate.
I replaced some informations with xxx on the following commands and results :

Code:

# first check certificate on pfsense CA
openssl verify -CAfile pfsense+JGA+vpn.crt wiki\ \(1\).crt 
wiki (1).crt: OK
# now for same server, same FQDN, same CN, same SAN, same ... with TrueNas CA
openssl verify -CAfile xxxx\ \(1\).crt wiki.crt 
CN = wiki.maison.local, C = xxx, ST = xxx, L = xxx, O = xxx
error 20 at 0 depth lookup: unable to get local issuer certificate
error wiki.crt: verification failed

 

[Patrick M. Hausen]

Code:

$ openssl verify -CAfile My_CA.crt Test_Cert.crt
Test_Cert.crt: OK

$ openssl x509 -noout -text -in My_CA.crt 
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 9204630 (0x8c7396)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = My_CA, C = DE, ST = Baden-Wuerttemberg, L = Ettlingen, O = Just me, emailAddress = hostmaster@hausen.com
        Validity
            Not Before: Aug 16 17:26:27 2021 GMT
            Not After : Aug 14 17:26:27 2031 GMT
        Subject: CN = My_CA, C = DE, ST = Baden-Wuerttemberg, L = Ettlingen, O = Just me, emailAddress = hostmaster@hausen.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
[...]
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Alternative Name: 
                DNS:My_CA
            X509v3 Subject Key Identifier: 
                B3:ED:2F:C4:1D:71:DC:DB:40:6E:7B:93:55:71:30:C8:BA:51:3A:68
            X509v3 Basic Constraints: critical
                CA:TRUE
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication
            X509v3 Key Usage: critical
                Certificate Sign, CRL Sign
    Signature Algorithm: sha256WithRSAEncryption
[...]
 
$ openssl x509 -noout -text -in Test_Cert.crt 
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 9204631 (0x8c7397)
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = My_CA, C = DE, ST = Baden-Wuerttemberg, L = Ettlingen, O = Just me, emailAddress = hostmaster@hausen.com
        Validity
            Not Before: Aug 16 17:27:36 2021 GMT
            Not After : Aug 16 17:27:36 2022 GMT
        Subject: CN = freenas.ettlingen.hausen.com, C = DE, ST = Baden-Wuerttemberg, L = Ettlingen, O = Just me, emailAddress = hostmaster@hausen.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
[...]
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Alternative Name: 
                DNS:freenas.ettlingen.hausen.com
            X509v3 Subject Key Identifier: 
                10:7D:48:58:57:F6:50:DB:EC:A5:30:A6:EF:49:DD:1F:52:AC:39:89
    Signature Algorithm: sha256WithRSAEncryption
[...]

 

[mururoa]

Okay, my fault it seems.
The TrueNas CA was an old one created on Freenas years ago.
I tried with a brand new CA created on TrueNas and it works.
Sooooo, I think the old CA was ok years ago but now is considered invalid by recent openssl and browsers. Maybe missing params or extensions.

 

[JollyWaffl]

Is it possible the issue is related to a change in the ordering of attributes in the Issuer line?

I have a root CA and certificates generated in a previous version of FreeNAS (possibly 11.3), all of which currently work: the root is imported into Firefox, and it happily authenticates the existing certificates from my various local servers. I'm now on TrueNAS Core 12, trying to create a new certificate signed by the old CA, and I can't get a certificate that the browser will accept. Looking at the existing, working certificates and CA, I see that the issuer and subject attribute order is C, ST, L, O, CN, emailAddress, and it's also reported in that order in the GUI. The new certificate has an issuer attribute order CN, C, ST, L, O, emailAddress, and openssl verify complains that it can't get the corresponding issuer certificate, even though I just pulled it out of the GUI.

Quick google suggests that the order theoretically shouldn't matter but that not all software allows for a different ordering.

 

[JollyWaffl]

I exported my CA private key from TrueNAS, and created a certificate for the new server from the command line using openssl. This new cert has the issuer attributes in the original order and is accepted by openssl verify and by Firefox without complaint.

Sooooo, I think the old CA was ok years ago but now is considered invalid by recent openssl and browsers. Maybe missing params or extensions.

The old CA is still valid, and it is useable with openssl to sign a new key with the appropriate extensions such that modern devices accept it (tested on Android 11), it's just TrueNAS that doesn't like it. Some kind of error in the GUI to that effect would be nice...