To prevent future email delivery issues, Zendesk implemented several remediation measures. These include adding an internal rate limit to Google domains to better control traffic, creating a feedback loop to intelligently analyze and retry customer traffic marked as spam, refining the release process to mitigate spam impact, creating new alerts sensitive to error codes, and improving spam detection capabilities. These steps aim to enhance the reliability of email delivery and prevent similar incidents.
The email delays to Google-managed domains in November 2023 were caused by a rate limit applied by Google. A subset of Zendesk's IP addresses was briefly listed on a blocklist, leading Google to apply a rate limit to all email traffic from…
Zendesk resolved the email delivery issue with Google-managed domains by collaborating with Google to apply a fix. This fix alleviated the strain from the rate-limiting that had been previously applied due to a blocklist issue. Additionally,…
The email delivery issue to Google-managed domains was resolved on November 21, 2023. After monitoring improvements over the weekend, Zendesk confirmed that the fix released by Google restored expected email delivery behavior. By November 21, the…
The email delays impacted Zendesk customers by causing delays in outbound email reception for end users with Gmail addresses or other Google-managed domains. While the majority of emails were delivered successfully, the small proportion of delayed…
During the email delivery incident, Zendesk communicated updates through a timeline of status updates. These updates provided information on the progress of the investigation, measures being taken, and improvements observed. Zendesk kept customers…