How Zendesk Resolved the July 23, 2024 Service Incident
How did Zendesk resolve the service incident on July 23, 2024?
Zendesk resolved the service incident on July 23, 2024, by initially increasing the capacity of the permissions service’s database instance. This provided a short-term recovery while the root cause was being identified. Once the root cause was determined, which was related to a new feature rollout, the engineers rolled back the permissions feature code change.
This rollback prevented further problems, and after additional monitoring, no further errors were detected, marking the incident as fully resolved. The team also planned remediation items to prevent future incidents, such as reducing network traffic from permissions checks and scheduling additional monitors and alerts.
More related questions
What happened during the Zendesk service incident on July 23, 2024?
On July 23, 2024, Zendesk experienced a service incident affecting customers on Pod 29. From 10:58 UTC to 14:57 UTC, users faced issues accessing Zendesk products, including the Admin Center, through the Product Tray. Approximately 1% of customer…
What was the root cause of the Zendesk incident on July 23, 2024?
The root cause of the Zendesk incident on July 23, 2024, was a new feature rollout related to managing team members' permissions. This feature allowed agents in custom roles to manage other team members and their role assignments. The rollout led…
What steps did Zendesk take to prevent future incidents like the one on July 23, 2024?
To prevent future incidents similar to the one on July 23, 2024, Zendesk planned several remediation items. These include reducing network traffic from permissions checks, which is currently in progress, and scheduling additional monitors and…
What were the main errors experienced during the Zendesk incident on July 23, 2024?
During the Zendesk incident on July 23, 2024, customers on Pod 29 experienced several errors. These included the inability to access Zendesk products through the Product Tray and receiving 503 errors when accessing authenticated features within…
Interested indeflectingover 70% of your Zendesk support tickets?
Zendesk Support Tickets
Zendesk Messaging (live chat)
Join1,000+ companies reducing their support costs and freeing up support agents for more important work
“We needed an AI agent integrated within our current tools. My AskAI was the only solution that wasn't going to disrupt our operations.”
Zeffy
“At the end of last year I was given the challenge - how can we provide the same or better service, without hiring anyone?”
Zinc
“My AskAI blew everybody else out of the water. It made the selection process very easy for us.”
Customer.io($50M+ ARR)
“It now resolves 71% of queries (over 35,000 every month), meaning more time solving complex issues and improving UX.”
Freecash
“We needed an AI agent integrated within our current tools. My AskAI was the only solution that wasn't going to disrupt our operations.”
Zeffy
“At the end of last year I was given the challenge - how can we provide the same or better service, without hiring anyone?”
Zinc
“My AskAI blew everybody else out of the water. It made the selection process very easy for us.”
Customer.io($50M+ ARR)
“It now resolves 71% of queries (over 35,000 every month), meaning more time solving complex issues and improving UX.”
Freecash
Reduce support costs.Spend more time on customer success.
