How to troubleshoot Forms authentication crawling rule creation failures caused by SSL certificates

Introduction

This document provides troubleshooting steps to diagnose and fix common Forms authentication crawling rule creation failures caused by SSL certificates.

This document is only applicable if your content server AND/OR the SSO system is using HTTPS. This document discusses several scenarios of Forms authentication crawling rule creation failures due to SSL handshake issues. When you are creating a Forms Authentication rule for crawling, the admin console may display an error "Forms Authentication Login Failed." followed by a Java exception message. This typically occurs due to a SSL handshake failure. This document provides verification steps and workarounds to these failures.

Terminology

  • CA - Certificate Authority. CA is an entity that issues digital certificates for use by other parties. It is an example of a trusted third party. CAs are characteristic of many public key infrastructure (PKI) schemes.
  • CSR - Certificate Signing Request. CSR is a message sent from an applicant to a certificate authority in order to apply for a digital identity certificate.
  • SSO - Single Sign-On. SSO is a property of access control of multiple, related, but independent software systems.
  • SSL Server - In this document the term "SSL server" is used as a generic term which applies to both the Content server and the SSO server.
  • GSA - Google Search Appliance

Troubleshooting Checklist


  1. Scenario #1

    You are trying to create a Forms authentication rule on the GSA and the following error message is displayed from the admin console.

    Forms Authentication Login Failed."sun.security.validator.ValidatorException: PKIX path building failed: 
    sun.security.provider.certpath.SunCertPathBuilderException:unable to find valid certification path to requested target"
    

    This scenario typically gets triggered if the SSL Server presents the end entity certificate but not the whole certificate chain (end entity server certificate, Intermediate CA certificate and Root CA certificate)

    Steps to verify : Use the Openssl tool to read the certificate chain served by the SSL server.
    For example : openssl s_client -connect harvest.acme.com:443 -showcerts

    Here is the sample output when the SSL server presents the end entity certificate but NOT the whole certificate chain :

    $ openssl s_client -connect harvest.acme.com:443 -showcerts
    CONNECTED(00000003)
    depth=0 /C=US/ST=Missouri/L=St Louis/O=Acme LLC/OU=IT Tech Ops/OU=Terms of use at www.verisign.com/rpa (c)05/CN=harvest.acme.com
    verify error:num=20:unable to get local issuer certificate
    verify return:1
    depth=0 /C=US/ST=Missouri/L=St Louis/O=Acme LLC/OU=IT Tech Ops/OU=Terms of use at www.verisign.com/rpa (c)05/CN=harvest.acme.com
    verify error:num=27:certificate not trusted
    verify return:1
    depth=0 /C=US/ST=Missouri/L=St Louis/O=Acme LLC/OU=IT Tech Ops/OU=Terms of use at www.verisign.com/rpa (c)05/CN=harvest.acme.com
    verify error:num=21:unable to verify the first certificate
    verify return:1
    ---
    Certificate chain
     0 s:/C=US/ST=Missouri/L=St Louis/O=Acme LLC/OU=IT Tech Ops/OU=Terms of use at www.verisign.com/rpa (c)05/CN=harvest.acme.com
       i:/O=VeriSign Trust Network/OU=VeriSign, Inc./OU=VeriSign International Server CA - Class 3/OU=www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign
    
    -----BEGIN CERTIFICATE-----
    MIIEpTCCBA6gAwIBAgIQbEwMI5SmnyqGZVdYf7pVGDANBgkqhkiG9w0BAQUFADCB
    dmVyIENBIC0gQ2xhc3MgMzFJMEcGA1UECxNAd3d3LnZlcmlzaWduLmNvbS9DUFMg
    SW5jb3JwLmJ5IFJlZi4gTElBQklMSVRZIExURC4oYyk5NyBWZXJpU2lnbjAeFw0w
    OTAzMjcwMDAwMDBaFw0xMTAzMjgyMzU5NTlaMIG1MQswCQYDVQQGEwJVUzERMA8G
    A1UECBMITWlzc291cmkxETAPBgNVBAcUCFN0IExvdWlzMRMwEQYDVQQKFApQYW5l
    cmEgTExDMRQwEgYDVQQLFAtJVCBUZWNoIE9wczEzMDEGA1UECxQqVGVybXMgb2Yg
    dXNlIGF0IHd3dy52ZXJpc2lnbi5jb20vcnBhIChjKTA1MSAwHgYDVQQDFBdoYXJ2
    ZXN0LnBhbmVyYWJyZWFkLmNvbTCBnzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEA
    stypH/BrP7aSnJ5MVao+3nalWf/wlqRjFI+Eb1CAYGzH6uEZmYCSiqXeIM4VkwsH
    zhdzYjx7MIDiY2wGQWCCseM1tS7yVQVNy/8gTn/+iFV9ncjhFrt1cz57iTy+pNYP
    bct1D2MmsmF68iNszCUQKlzvqgXfSwiFlacwP31dev0CAwEAAaOCAa0wggGpMAkG
    A1UdEwQCMAAwCwYDVR0PBAQDAgWgMEQGA1UdIAQ9MDswOQYLYIZIAYb4RQEHFwMw
    KjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL3JwYTA8BgNV
    HR8ENTAzMDGgL6AthitodHRwOi8vU1ZSSW50bC1jcmwudmVyaXNpZ24uY29tL1NW
    UkludGwuY3JsMCgGA1UdJQQhMB8GCCsGAQUFBwMBBggrBgEFBQcDAgYJYIZIAYb4
    QgQBMHEGCCsGAQUFBwEBBGUwYzAkBggrBgEFBQcwAYYYaHR0cDovL29jc3AudmVy
    aXNpZ24uY29tMDsGCCsGAQUFBzAChi9odHRwOi8vU1ZSSW50bC1haWEudmVyaXNp
    Z24uY29tL1NWUkludGwtYWlhLmNlcjBuBggrBgEFBQcBDARiMGChXqBcMFowWDBW
    FglpbWFnZS9naWYwITAfMAcGBSsOAwIaBBRLa7kolgYMu9BSOJsprEsHiyEFGDAm
    FiRodHRwOi8vbG9nby52ZXJpc2lnbi5jb20vdnNsb2dvMS5naWYwDQYJKoZIhvcN
    nQBTD4Vaqabomp4d5UxrPqQlLalQFdUJEnj6s5b1EBnAUz6ckkvClC8=
    -----END CERTIFICATE-----
    
    ---
    Server certificate
    subject=/C=US/ST=Missouri/L=St Louis/O=Acme LLC/OU=IT Tech Ops/OU=Terms of use at www.verisign.com/rpa (c)05/CN=harvest.acme.com
    issuer=/O=VeriSign Trust Network/OU=VeriSign, Inc./OU=VeriSign International Server CA - Class 3/OU=www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign
    ---
    No client certificate CA names sent
    ---
    SSL handshake has read 1339 bytes and written 300 bytes
    ---
    New, TLSv1/SSLv3, Cipher is RC4-MD5
    Server public key is 1024 bit
    Compression: NONE
    Expansion: NONE
    SSL-Session:
        Protocol  : TLSv1
        Cipher    : RC4-MD5
        Session-ID: DC15EA7108AC3079E0EB3C62DA92EBBDCC3532959A13972901FB7C1AED766771
        Session-ID-ctx:
        Master-Key: C2C99E81AB25C2FD53A6A407F14B574CFF82386693DC5FD88158567D2B3D596526EAC5EE7814DC3638E942386AB8C17E
        Key-Arg   : None
        Start Time: 1248910118
        Timeout   : 300 (sec)
        Verify return code: 21 (unable to verify the first certificate)
    

     

    Here is the sample output when the SSL sever properly provides the whole certificate chain:

    ~$ openssl s_client -connect harvsso.acme.com:4443 -showcerts
    CONNECTED(00000003)
    depth=2 /C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
    verify error:num=19:self signed certificate in certificate chain
    verify return:0
    ---
    Certificate chain
     0 s:/C=US/ST=Missouri/L=St.Louis/O=Acme LLC/OU=INFORMATION SYSTEMS/OU=Terms of use at www.verisign.com/rpa (c)05/CN=harvsso.acme.com
       i:/O=VeriSign Trust Network/OU=VeriSign, Inc./OU=VeriSign International Server CA - Class 3/OU=www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign
    
    -----BEGIN CERTIFICATE-----
    MIIErTCCBBagAwIBAgIQV9wvXCgX7o1J2m2eAPUBWTANBgkqhkiG9w0BAQUFADCB
    dmVyIENBIC0gQ2xhc3MgMzFJMEcGA1UECxNAd3d3LnZlcmlzaWduLmNvbS9DUFMg
    SW5jb3JwLmJ5IFJlZi4gTElBQklMSVRZIExURC4oYyk5NyBWZXJpU2lnbjAeFw0w
    OTAzMjYwMDAwMDBaFw0xMTAzMjYyMzU5NTlaMIG9MQswCQYDVQQGEwJVUzERMA8G
    A1UECBMITWlzc291cmkxETAPBgNVBAcUCFN0LkxvdWlzMRMwEQYDVQQKFApQYW5l
    cmEgTExDMRwwGgYDVQQLFBNJTkZPUk1BVElPTiBTWVNURU1TMTMwMQYDVQQLFCpU
    ZXJtcyBvZiB1c2UgYXQgd3d3LnZlcmlzaWduLmNvbS9ycGEgKGMpMDUxIDAeBgNV
    BAMUF2hhcnZzc28ucGFuZXJhYnJlYWQuY29tMIGfMA0GCSqGSIb3DQEBAQUAA4GN
    ADCBiQKBgQCb/kGdU9iEEK9fGgs/FnvCaPJo09G2FH8kaGIw/SuKlj7n7rJ5YFrh
    bQqZ5L7nQ4vepS53ht65BElAdQqC9B0aiBjS2s1BRLEjqL9gsYxpGniDROroST2c
    qtJRh4tu0qd1wslCMmufVAlm53LVUP23ToeOzEVTQRButkiDEgoFFQIDAQABo4IB
    rTCCAakwCQYDVR0TBAIwADALBgNVHQ8EBAMCBaAwRAYDVR0gBD0wOzA5BgtghkgB
    hvhFAQcXAzAqMCgGCCsGAQUFBwIBFhxodHRwczovL3d3dy52ZXJpc2lnbi5jb20v
    cnBhMDwGA1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9TVlJJbnRsLWNybC52ZXJpc2ln
    bi5jb20vU1ZSSW50bC5jcmwwKAYDVR0lBCEwHwYIKwYBBQUHAwEGCCsGAQUFBwMC
    BglghkgBhvhCBAEwcQYIKwYBBQUHAQEEZTBjMCQGCCsGAQUFBzABhhhodHRwOi8v
    b2NzcC52ZXJpc2lnbi5jb20wOwYIKwYBBQUHMAKGL2h0dHA6Ly9TVlJJbnRsLWFp
    YS52ZXJpc2lnbi5jb20vU1ZSSW50bC1haWEuY2VyMG4GCCsGAQUFBwEMBGIwYKFe
    oFwwWjBYMFYWCWltYWdlL2dpZjAhMB8wBwYFKw4DAhoEFEtruSiWBgy70FI4myms
    SweLIQUYMCYWJGh0dHA6Ly9sb2dvLnZlcmlzaWduLmNvbS92c2xvZ28xLmdpZjAN
    Xos6EvLtqiJDFl348TpPe83dpqBbwDFJM7YV015wJyhdYiGy+hTy3Gwd6v7V4gKo
    mw==
    -----END CERTIFICATE-----
    
     1 s:/O=VeriSign Trust Network/OU=VeriSign, Inc./OU=VeriSign International Server CA - Class 3/OU=www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign
       i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
    
    -----BEGIN CERTIFICATE-----
    MIIDgzCCAuygAwIBAgIQRvzrurTQLw+SYJgjP5MHjzANBgkqhkiG9w0BAQUFADBf
    HhcNOTcwNDE3MDAwMDAwWhcNMTYxMDI0MjM1OTU5WjCBujEfMB0GA1UEChMWVmVy
    aVNpZ24gVHJ1c3QgTmV0d29yazEXMBUGA1UECxMOVmVyaVNpZ24sIEluYy4xMzAx
    BgNVBAsTKlZlcmlTaWduIEludGVybmF0aW9uYWwgU2VydmVyIENBIC0gQ2xhc3Mg
    MzFJMEcGA1UECxNAd3d3LnZlcmlzaWduLmNvbS9DUFMgSW5jb3JwLmJ5IFJlZi4g
    TElBQklMSVRZIExURC4oYyk5NyBWZXJpU2lnbjCBnzANBgkqhkiG9w0BAQEFAAOB
    jQAwgYkCgYEA2IKA6NYZAn0fhRg5JaJlK+G/1AXTvOY2O6rwTGxbtueqPHNFVbLx
    veqXQu2aNAoV1Klc9UAl3dkHwTKydWzEyruj/lYncUOqY/UwPpMo5frxCTvzt01O
    OfdcSVq4wR3Tsor+cDCVQsv+K1GLWjw6+SJPkLICp1OcTzTnqwSye28CAwEAAaOB
    4zCB4DAPBgNVHRMECDAGAQH/AgEAMEQGA1UdIAQ9MDswOQYLYIZIAYb4RQEHAQEw
    KjAoBggrBgEFBQcCARYcaHR0cHM6Ly93d3cudmVyaXNpZ24uY29tL0NQUzA0BgNV
    HSUELTArBggrBgEFBQcDAQYIKwYBBQUHAwIGCWCGSAGG+EIEAQYKYIZIAYb4RQEI
    ATALBgNVHQ8EBAMCAQYwEQYJYIZIAYb4QgEBBAQDAgEGMDEGA1UdHwQqMCgwJqAk
    oCKGIGh0dHA6Ly9jcmwudmVyaXNpZ24uY29tL3BjYTMuY3JsMA0GCSqGSIb3DQEB
    6UlIE0uDihtIeyT3ON5vQVS4q1drBt/HotSp9vE2YoCI8ot11oBx
    -----END CERTIFICATE-----
    
     2 s:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
       i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification Authority
    
    -----BEGIN CERTIFICATE-----
    MIICPDCCAaUCEHC65B0Q2Sk0tjjKewPMur8wDQYJKoZIhvcNAQECBQAwXzELMAkG
    MDEyOTAwMDAwMFoXDTI4MDgwMTIzNTk1OVowXzELMAkGA1UEBhMCVVMxFzAVBgNV
    BAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFzcyAzIFB1YmxpYyBQcmlt
    YXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIGfMA0GCSqGSIb3DQEBAQUAA4GN
    ADCBiQKBgQDJXFme8huKARS0EN8EQNvjV69qRUCPhAwL0TPZ2RHP7gJYHyX3KqhE
    BarsAx94f56TuZoAqiN91qyFomNFx3InzPRMxnVx0jnvT0Lwdd8KkMaOIG+YD/is
    I19wKTakyYbnsZogy1Olhec9vn2a/iRFM9x2Fe0PonFkTGUugWhFpwIDAQABMA0G
    AA9WjQKZ7aKQRUzkuxCkPfAyAw7xzvjoyVGM5mKf5p/AfbdynMk2OmufTqj/ZA1k
    -----END CERTIFICATE-----
    
    ---
    Server certificate
    subject=/C=US/ST=Missouri/L=St.Louis/O=Acme LLC/OU=INFORMATION SYSTEMS/OU=Terms of use at www.verisign.com/rpa (c)05/CN=harvsso.acme.com
    issuer=/O=VeriSign Trust Network/OU=VeriSign, Inc./OU=VeriSign International Server CA - Class 3/OU=www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign
    ---
    No client certificate CA names sent
    ---
    SSL handshake has read 2832 bytes and written 316 bytes
    ---
    New, TLSv1/SSLv3, Cipher is AES256-SHA
    Server public key is 1024 bit
    Compression: NONE
    Expansion: NONE
    SSL-Session:
        Protocol  : TLSv1
        Cipher    : AES256-SHA
        Session-ID: DBBABAF9B1D7C34AA867F2B17F0BAB2D
        Session-ID-ctx:
        Master-Key: 8D329755467C08D56626EDCEDE6051336F142F8907B86CA389C60F75F3E5505B2E67A3202835567297DD96CA5D058813
        Key-Arg   : None
        Start Time: 1248994847
        Timeout   : 300 (sec)
        Verify return code: 19 (self signed certificate in certificate chain)
    

    Solution : Modify the SSL server to present the whole certificate chain.

    Workaround : Import the Intermediate CA certificate and the Root CA certificate into the appliance's certificate authorities store,
    located under : Administration >Certificate Authorities

    Note: Make sure that the certificates are in PEM format. In most cases, you can download the intermediate CA and Root CA certificates from the CA's website. If it is unclear or the certificates are unavailable online then you should contact your CA who signed the server certificate. Also refer to Scenario #2.

    Scenario #2

    You are trying to create a Forms authentication rule on the GSA and the following error message is displayed from the admin console.

    Forms Authentication Login Failed."java.security.cert.CertificateException: Certificate chain verification failed. 
    For CAs with CRLs: there were no CAs with CRLs.; 
    For CAs without CRLs: java.security.cert.CertPathValidatorException: basic constraints check failed: this is not a CA certificate"
    

     

    This scenario gets triggered if the following conditions are true:

    1. The SSL server presents just the end entity certificate but not the whole chain. The GSA administrator imported the Root CA and Intermediate CA certificates into the appliance to workaround the issue. (AND)
    2. There is a version mismatch between the Root CA certificate and the Intermediate CA certificate.

    To establish security in a way that end entity certificate can not issue/sign someone else's SSL server key, end entity certificates in X.509 v3 are issued with Basic Constraints: Critical, CA: False. This basically indicates that your end entity certificate is not a CA certificate, in other words, it is not allowed to issue any certificates. Basic Constraints was not yet defined in X.509 v1. When the GSA code tries to build the Certificate path, the missing basic constraint tag in version1 certificate triggers the exception. The appliance strictly validates the certificates when building the certificate chain per RFC 3280. This results in a SSL handshake failure.

    Steps to verify : Download and save the copies of Root CA and intermediate CA certificates in a standard PEM format. Use the OpenSSL tool to read the files.
    For example : $ openssl x509 -in intermediateca.cert -text

    Here is the output from the Intermediate CA certificate:

    $ openssl x509 -in intermediateca.cert -text
    
    Certificate:
        Data:
            Version: 3 (0x2) 
            Serial Number:
                46:fc:eb:ba:b4:d0:2f:0f:92:60:98:23:3f:93:07:8f
            Signature Algorithm: sha1WithRSAEncryption
            Issuer: C=US, O=VeriSign, Inc., OU=Class 3 Public Primary Certification Authority
            Validity
                Not Before: Apr 17 00:00:00 1997 GMT
                Not After : Oct 24 23:59:59 2016 GMT
            Subject: O=VeriSign Trust Network, OU=VeriSign, Inc., OU=VeriSign International Server CA - Class 3, OU=www.verisign.com/CPS Incorp.by Ref. LIABILITY LTD.(c)97 VeriSign
            Subject Public Key Info:
                Public Key Algorithm: rsaEncryption
                RSA Public Key: (1024 bit)
                    Modulus (1024 bit):
                        00:d8:82:80:e8:d6:19:02:7d:1f:85:18:39:25:a2:
                        65:2b:e1:bf:d4:05:d3:bc:e6:36:3b:aa:f0:4c:6c:
                        5b:b6:e7:aa:3c:73:45:55:b2:f1:bd:ea:97:42:ed:
                        9a:34:0a:15:d4:a9:5c:f5:40:25:dd:d9:07:c1:32:
                        b2:75:6c:c4:ca:bb:a3:fe:56:27:71:43:aa:63:f5:
                        30:3e:93:28:e5:fa:f1:09:3b:f3:b7:4d:4e:39:f7:
                        5c:49:5a:b8:c1:1d:d3:b2:8a:fe:70:30:95:42:cb:
                        fe:2b:51:8b:5a:3c:3a:f9:22:4f:90:b2:02:a7:53:
                        9c:4f:34:e7:ab:04:b2:7b:6f
                    Exponent: 65537 (0x10001)
            X509v3 extensions:
                X509v3 Basic Constraints:
                    CA:TRUE, pathlen:0 
                X509v3 Certificate Policies:
                    Policy: 2.16.840.1.113733.1.7.1.1
                      CPS: https://www.verisign.com/CPS
    
                X509v3 Extended Key Usage:
                    TLS Web Server Authentication, TLS Web Client Authentication, Netscape Server Gated Crypto, 2.16.840.1.113733.1.8.1
                X509v3 Key Usage:
                    Certificate Sign, CRL Sign
                Netscape Cert Type:
                    SSL CA, S/MIME CA
                X509v3 CRL Distribution Points:
                    URI:http://crl.verisign.com/pca3.crl
    

    Output from Root CA certificate:

    Certificate:
        Data:
            Version: 1 (0x0) 
            Serial Number:
                70:ba:e4:1d:10:d9:29:34:b6:38:ca:7b:03:cc:ba:bf
            Signature Algorithm: md2WithRSAEncryption
            Issuer: C=US, O=VeriSign, Inc., OU=Class 3 Public Primary Certification Authority
            Validity
                Not Before: Jan 29 00:00:00 1996 GMT
                Not After : Aug  1 23:59:59 2028 GMT
            Subject: C=US, O=VeriSign, Inc., OU=Class 3 Public Primary Certification Authority
            Subject Public Key Info:
                Public Key Algorithm: rsaEncryption
                RSA Public Key: (1024 bit)
                    Modulus (1024 bit):
                        00:c9:5c:59:9e:f2:1b:8a:01:14:b4:10:df:04:40:
                        db:e3:57:af:6a:45:40:8f:84:0c:0b:d1:33:d9:d9:
                        11:cf:ee:02:58:1f:25:f7:2a:a8:44:05:aa:ec:03:
                        1f:78:7f:9e:93:b9:9a:00:aa:23:7d:d6:ac:85:a2:
                        63:45:c7:72:27:cc:f4:4c:c6:75:71:d2:39:ef:4f:
                        42:f0:75:df:0a:90:c6:8e:20:6f:98:0f:f8:ac:23:
                        5f:70:29:36:a4:c9:86:e7:b1:9a:20:cb:53:a5:85:
                        e7:3d:be:7d:9a:fe:24:45:33:dc:76:15:ed:0f:a2:
                        71:64:4c:65:2e:81:68:45:a7
                    Exponent: 65537 (0x10001)
    

    Solution : Modify the SSL server to serve up the whole certificate chain. Also remove the Root CA and Intermediate CA certificates that are imported under Administration >Certificate Authorities

    Troubleshooting tools

    1. Openssl - Most of the Linux distribution comes with this tool by default. Alternatively, you can download it from http://www.openssl.org/source/
    2.  How to convert various certificate formats to the PEM format ?
    3. The GSA only accepts certificates in PEM format. If your CA provides you with certificates other than PEM format, make sure to convert them to PEM format before importing into the GSA. The following articles cover most of the use cases.
Was this helpful?
How can we improve it?