Troubleshoot common GCDS issues

If you're having trouble getting a synchronization to run properly, confirm that the configuration information is correct in Configuration Manager and note which tests fail:

1. In Configuration Manager, open the XML file you're using to configure the synchronization.
2. On the LDAP Connections page, click Test Connections to confirm you can connect to your LDAP server.
3. On the Notifications page, click Test Notification to confirm you can send a test notification.
4. On the Sync page, click Simulate Sync to confirm you have completed all the required fields and to confirm that the synchronization runs.

In rare cases, support might ask you to turn on full HTTP logging in addition to turning on trace-level logging in GCDS. Full HTTP logging is used to see the exact API request made by GCDS and the response provided by the Google APIs.

Important: Full HTTP logs can contain highly sensitive information. Remove any sensitive information (such as current refresh_token or access_token fields) before sending the logs to support.

To turn on full HTTP logging:

1. Make sure GCDS isn't running with either sync-cmd or Configuration Manager.
2. Navigate to the GCDS installation folder.

3. Edit the file jre/lib/logging.properties.

4. Add the following lines to the end of the file:

   java.util.logging.FileHandler.pattern = %h/gcdshttp%u.%g.log
   java.util.logging.FileHandler.limit = 5000000
   java.util.logging.FileHandler.count = 100
   java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
   handlers = java.util.logging.FileHandler
   com.google.api.client.http.level = CONFIG
   
   com.google.gdata.client.http.HttpGDataRequest.level = ALL
   sun.net.www.protocol.http.HttpURLConnection.level = ALL

5. Save the file.
6. Run another GCDS sync (with logging set to Trace).

   Log files named gcdshttp*.log are created in homedir (Linux) or profile folder (Microsoft Windows). Archive these files together, because they can be quite big.

7. Delete the lines added in step 4 to prevent large log files being created in the future.
8. Provide the following files to support:
   • XML file
   • Trace-level logs and the gcdshttp*.log files from the latest sync

TIP: If you want to enable logging for a new class, append a line in the form of class.fqdn.level = ALL, no need to duplicate the whole setup block.

Logged request and response bodies are each limited to a size of 16 KB. If you see a log entry that's truncated because it's exceeding that limit, use a debugging proxy, such as Fiddler.

To enable Fiddler, follow these steps:

1. Go to the path where GCDS is installed, for example, C:\Program Files\Google Cloud Directory Sync.
2. Add the following flags to the .vmoptions files (for example, config-manager.vmoptions or sync-cmd.vmoptions) to turn off the CRL checks:

   -Dcom.sun.security.enableCRLDP=false

   -Dcom.sun.net.ssl.checkRevocation=false

Restart GCDS for the changes to take effect.

1. Configure Fiddler as a proxy in your Google domain configuration's proxy settings. In the hostname field, add the local IP address 127.0.0.1. The default port is 8888, but you can confirm this by opening Fiddler, going to Options > Connections, and checking the value in the "Fiddler Classiclistens on port" field.

If you're using GCDS on Linux, you can't use the Windows trusted certificate store, so you need to import the Fiddler root certificate into GCDS's Java trust store. For details on these steps, go to Troubleshoot certificate-related problems.

If you're having issues setting up the SMTP relay host for your notifications, try the following troubleshooting steps.

Connection failed & unknown SMTP host message

1. Open a command prompt.
2. To check if the configured host name of the SMTP server resolves to an IP address, enter the following command:

   nslookup smtp-host-name.com

Connection failed & could not connect to SMTP host message

Check if the server running GCDS can connect to the SMTP host.

1. To check the connection, from the Windows command line or terminal, enter the following command:

   telnet smtp.gmail.com 587

2. If the host can't connect, check the ingress firewall rules of the SMTP server and egress firewall rules of the GCDS server.
3. Make sure that you have allowed traffic on the SMTP port.

Could not convert socket to TLS error in logs

Turn off certificate revocation list (CRL) checks. For details, go to How GCDS checks certificate revocation lists.

See Work with configuration files for instructions on how to open an XML file that was saved on a different machine, or as a different user on the same machine.

If the LDAP data in the GCDS trace-level logs doesn't match what you expect to read on the LDAP server (for example, a user isn't found or an attribute doesn’t have the correct value), export the data from the LDAP directory in LDIF format. Support can compare the data with the LDAP data from the GCDS logs.

When you export the data, use an LDAP query tool such as ldapsearch (Linux) or ldifde (Windows) and simulate the same conditions that GCDS is running with:

• Use the same connection settings as the ones that GCDS is configured to use.
• Run the query tool from the same machine that GCDS is running on.
• Use the same username for GCDS LDAP authentication.

Example

Your GCDS logs don't show the mail attribute of your users, and your GCDS search rule settings are:

• Base DN: ou=Ireland,dc=altostrat,dc=com
• Scope: Subtree
• Search filter: (&(objectCategory=person)(objectClass=user))
• Server: dc01.altostrat.com
• Port: 636
• Protocol: LDAP+SSL
• Authentication user DN: cn=GCDS,ou=Users,dc=altostrat,dc=com

Use these commands:

• Linux: ldapsearch -v -b "ou=Ireland,dc=altostrat,dc=com" -s sub -h dc01.altostrat.com -p 636 -x -Z -D "cn=GCDS,ou=Users,dc=altostrat,dc=com" "(&(objectCategory=person)(objectClass=user))" mail givenname uniqueidentifier sn > out.ldif (You might need to modify the command depending on your system.)
• Windows: ldifde -f out.ldif -s dc01.altostrat.com -v -t 636 -d "ou=Ireland,dc=altostrat,dc=com" -r "(&(objectCategory=person)(objectClass=user))" -p SubTree -l mail,givenname,uniqueidentifier,sn -a "cn=GCDS,ou=Users,dc=altostrat,dc=com" PASSWORD (Replace PASSWORD with the password of the LDAP user set in GCDS.)

If the output (out.ldif) does not contain the mail attribute for an affected user, there’s a problem with the LDAP infrastructure. It might be related to the permissions of the user that you’re using to access the LDAP (for example, both OpenLDAP and Active Directory allow setting attribute-level permissions). Or the attribute might not be replicated to the Global Catalog, if you’re using a Global Catalog port such as 3268 or 3269.

If the output does contain the mail attribute for an affected user, provide the following details to Google Workspace support:

• The out.ldif file
• A screenshot of the command prompt or terminal window where you ran the command 
  (Make sure to remove the password first.)
• The GCDS trace-level log

To run a simulated synchronization, you need a server capable of sending mail. If you're running GCDS on a mail server machine, you can use the IP address 127.0.0.1 for your mail server. Otherwise, contact your mail administrator for the correct mail information.

If you're using the command line to run a sync and the sync doesn't start, check whether you used the -o or --oneinstance argument in the command line.

If you use one of those arguments, GCDS creates a LOCK (.lock) file associated with the XML configuration file. And, if another LOCK file is found on the same server, GCDS won't run the sync to prevent multiple instances of GCDS running simultaneously.

If there's no other instance of GCDS running, check for another LOCK file on the server. Delete the file manually and try running the sync again.

If your sync was incomplete, for example, the full membership of a group didn't sync, the Directory API could have an issue. To verify whether the issue relates to an API rather than the GCDS product, manually call the Directory API and review the results. To manually call the API, choose one of 2 options.

Option 1: Use the API reference page

1. Go to Admin SDK API reference overview.
2. On the left, click Directory API, then, for REST Resources, go to the REST Resource you want to query.
3. On the right, click the method you want to try and click Try it.

   If the API reference page doesn't show Try it, go to Option 2: Use the OAuth 2.0 Playground.

4. Enter the admin credentials that you used to authorize GCDS.

   For details, go to Define your Google domain settings.

5. Review the information to ensure that the API responded as expected.

Option 2: Use the OAuth 2.0 Playground

1. Open the OAuth 2.0 Playground.
2. Choose an option:
   • Select a scope from the list.
   • Copy a scope from the Authorization scopes list on the API reference page. Then, paste the scope in the Input your own scopes field.
3. Click Authorize APIs.
4. Enter the admin credentials that you used to authorize GCDS.

   For details, go to Define your Google domain settings.

5. Click Exchange authorization code for tokens.

   If the process is successful, you're redirected to Step 3: Configure request to API.

6. Complete the requested information.

   Tip: You can find most of the information on the API method reference webpage.

7. Click Send the request.
8. Review the information to ensure that the API responded as expected.

In your XML configuration file, set the useDynamicMaxCacheLifetime option. This option configures GCDS to cache data for a maximum of 8 days, and clear the cache more frequently in small-to-medium data sets to reduce the chance of cached data growing stale or conflicting with new data. The useDynamicMaxCacheLifetime option is automatic in configurations created with GCDS 3.2.1 and above.

Note: These errors usually occur when modifications are done directly in your Google domain. When using GCDS to sync, you should avoid making changes to your Google domain directly. Instead, make changes to users, groups, and other entities in your LDAP directory. Then, use GCDS to sync these changes to your Google domain.

If you're seeing memory-related errors, you need to increase the heap size for Java Virtual Machine. Increase the heap size by editing the sync-cmd.vmoptions and config-manager.vmoptions files in the installation directory of GCDS. The relevant entries look like this:

• -Xmx1000m (the maximum amount of memory allocated for the heap size)
• -Xms64m (the minimum amount of memory allocated for the heap size)

Edit both the sync-cmd.vmoptions and config-manager.vmoptions files so that the change applies to both sync-cmd and Configuration Manager versions.

Edit the -Xmx number to increase the amount of memory. The "m" following the number indicates that the memory is measured in megabytes (MB). The correct amount of memory depends on how much the GCDS server has, and how much it needs for a synchronization. You might need to revise the number several times to set the correct size. For more information on the amount of free RAM required to run GCDS, go to System requirements for GCDS.

The issue might be caused by a configuration issue, such as an exclusion rule misconfiguration. This type of misconfiguration can be hidden by GCDS caching. 

GCDS keeps a cache of data for your Google service (such as Google Workspace or Cloud Identity) for a maximum of 8 days. GCDS might clear the cache more frequently, depending on the cached data size. However, if the cache isn't cleared, you might not see your updates for up to 8 days. 

For example, you sync your LDAP data and create a new group for your Google service (such as Google Workspace or Cloud Identity). You then create an exclusion rule to exclude that group from subsequent syncs. The exclusion rule is misconfigured and will fail. However, the subsequent sync calls on cached data and the group remains in your Google service. When you sync again with clear cache, the misconfiguration causes the group to be removed from your Google service.

To manually clear the cache:

• Run a sync from Configuration Manger and select the option to clear the cache when performing a sync.
• Run a sync from the command and use the argument -f to force a flush of the cache.
• Modify the XML config file to set the maxCacheLifetime value to 0.

Important: Forcing a flush of cache can dramatically increase synchronization time.

If you get the error 409: Entity already exists, GCDS is trying to create Google users that already exist. If you don't see the error in subsequent syncs, it's likely that the GCDS cache was out-of-date and it's safe to ignore the error.

If the issue occurs every sync or every several days, the most likely reasons are:

• A Google user exclusion rule is too broad—the rule matches some Google users that also exist in the LDAP directory.
• A query is too narrow—the query doesn't match some Google users that also exist in the LDAP directory.

Both scenarios can cause GCDS to ignore Google users that already exist. If those users exist in the LDAP user search rule results, GCDS tries to create them in your Google Account.

To resolve the issue, adjust the exclusion rule or search query. Or, if you want GCDS to completely ignore the users in your LDAP directory, adjust your LDAP user search rule or create an LDAP user exclusion rule. For details, go to Omit data with exclusion rules & queries.

To synchronize group members separately from the results of any user search rules, GCDS turns on INDEPENDENT_GROUP_SYNC by default. If you're using member-reference attributes for group synchronizations, GCDS tries to resolve the email address of every user in the LDAP directory, regardless of any user search rules.

To synchronize group members based on only the results of user search rules, remove INDEPENDENT_GROUP_SYNC from your configuration XML file. GCDS:

• Uses the results of the user search rules to resolve group membership
• Only syncs as group members those users included in your user sync
• Executes user search rules, even if you turn off user account sync in General Settings

  (However, the results don’t sync to Google as users but as group members, if the eligible users also qualify as group members.)

Typically, this isn’t how you want your sync to run, particularly if you're syncing shared contacts and have group members that are contacts. In this case, the contacts won't synchronize as group members.

This issue happens when the LDAP attribute configured as the Group Name Attribute doesn't contain a full email address. To resolve this issue, check your Group Search rules and make sure that GCDS uses a full email address for the group names. Use one of the following methods:

• Set the Group Name Attribute to a different LDAP attribute that specifies a full email address for each group, such as mail.
• Turn on Replace domain names in LDAP email addresses in the Google Domain Settings, so that your Group Name Attribute matches the Google-side group names.
• Add the domain name to the group name by specifying a Group Name Suffix in your Group Search Rule.

Make sure you have selected MS Active Directory in the server type field of the LDAP Configuration section.

This option (shown as SUPPRESS_DOMAIN in the XML file) is used if the email addresses in the LDAP directory are in a different domain than your Google domain. When you turn it on, GCDS strips the domain part off all of the email addresses it reads.

Any processing is done without the domain name. If you use exclusion rules based on email addresses, you need to consider only the local part of the email address for the exclusion rule.

For example, if you have Replace domain names in LDAP email addresses turned off, and you're creating an Exact Match exclusion rule, enter luka@example.com as the user's email address to match. If you've turned on Replace domain names in LDAP email addresses, use luka. Trying to match luka@example.com won't work, because @example.com is stripped prior to the comparison.

When provisioning groups using GCDS, you can't nest dynamic groups underneath static groups (or static groups underneath dynamic groups). GCDS requires static groups to be queried separately from dynamic groups; however all nested groups have to be a part of the same query.

Find a way to implement dynamic groups as static groups, possibly by automating a task that periodically queries every dynamic group to populate static groups in the directory. GCDS can then use the static groups (as created from the dynamic groups) for provisioning and not provision the dynamic group.

Results for LDAP queries depend on Configuration Manager settings and the LDAP server. Use these troubleshooting tips if your LDAP search rule returns an unexpected result. Make sure:

• The LDAP query is set up correctly in Configuration Manager—When you set up a search rule, click Test LDAP Query to verify. For details, go to Use LDAP search rules to synchronize data.
• Multiple queries don't contradict each other—Check that you haven’t set up a search or exclusion rule that changes the result of a query.
• The authorized user for the LDAP server has sufficient permissions—Make sure that the administrator used to authenticate the LDAP server can use the command line on the same server. Try the query on the LDAP server and verify the results.

You might encounter the error message Group ... could not be created. Message: Not Authorized to access this resource/api in the GCDS logs. 

To troubleshoot, check that the Active Directory (AD) attribute that contains the domain for user and group email addresses matches the domain that your super admin account uses.

This issue usually happens if you're syncing shared contacts and the search rules are built incorrectly.

There are 2 types of relevant objects that you can sync with GCDS:

• User profiles—Users in your Google domain with additional data such as a phone number or address. You can sync a profile only for a user that exists in your domain.
• Shared contacts—Contacts of external parties that users in your domain need to contact.

To resolve this issue, correct your shared contact search rules to exclude users in your own domain. On the next sync, GCDS attempts to delete the redundant contacts. You might need to adjust the shared contact deletion limit for that first sync.

In some circumstances, users don't see their main work location in Google Calendar when scheduling or arranging meetings.

If you see this issue, make sure that the location type and area attributes are set to "desk."

If you're having problems with search results, check: 

• The scope of the rule. You might need to set the scope to Sub-tree.
• The search rule that you're using is correct.
• The attributes that are used exist and are visible.

Your LDAP query. Verify that the query on your LDAP server is using the same admin username that's configured in GCDS.

For more details, go to Use LDAP search rules to synchronize data.

You might be using a font that is too large for the screen. The dialog box does not work with large or extra large fonts. Change your font size or edit your XML file directly.