Delayed delivery across all automations and search experience

Resolved
Degraded performance
Started 2 months ago Lasted about 10 hours

Affected

app
api
Updates
  • Resolved
    Resolved

    We've fully processed all automation and indexing backlog across our systems and all customer workspaces. We've implemented several safeguards to prevent this situation and dramatically speed up processing under heavy load (5-10x backlog)

  • Monitoring
    Monitoring

    We've implemented several fixes to speed up processing and start working through the backlog.

  • Identified
    Identified

    We've detected several bottlenecks with our background processing that is contributing to the delayed automations.

  • Investigating
    Investigating

    We are currently investigating this incident after a 2-3x increase in processing.