We have identified the root cause as an issue with the Google Workspace API. Google has identified and rolled back the change that caused this issue. Directory Sync for Google Workspaces is now working as intended.
Beginning 19:29 UTC on August 9 2023, some calls to retrieve group and group membership information from the Google APIs began returning incomplete data, which made it appear that some groups and associated memberships had been deleted. These deletions are critical directory events that WorkOS would sync to our customers’ systems. We chose to pause directory syncing for Google Workspaces rather than send these erroneous events.
We have validated that Directory Sync events are now accurate and resumed syncing. Google Workspace directory webhooks are being delivered with the correct updates.
Posted Aug 10, 2023 - 19:45 EDT
Investigating
We are encountering issues with Google Workspace Directory Syncing such that we are unable to guarantee the accuracy of group and group membership events. We are actively investigating and working towards identifying the root cause, and have escalated to Google and are working directly with them to move towards a resolution.
Until this is resolved, we have paused syncing for Google Workspace directories and are not sending webhooks.
Posted Aug 09, 2023 - 15:29 EDT
This incident affected: Core Services (Directory Sync).