Infrastructure issues affecting general Greenhouse Recruiting functionality

Incident Report for Greenhouse

Postmortem

At approximately 10:30 am EDT this morning, Greenhouse Recruiting experienced a partial outage lasting roughly half an hour that caused a high rate of error messages and limited a number of actions throughout the app.

The ultimate cause of this issue was operator error from a member of our engineering team while cleaning up extraneous resources that were used in a successful upgrade we completed last week. A resource that was still in use was accidentally included as well, but after identifying the issue we were able to recreate it and fully restore services. During this period, Job Boards and APIs remained unaffected and no data was lost.

We will be augmenting our deletion processes to protect against this type of outage moving forward, and we sincerely apologize for the disruption and inconvenience it caused.

Posted Sep 26, 2016 - 21:47 UTC

Resolved

This incident has been resolved. We will be providing a postmortem with more information today.
Posted Sep 26, 2016 - 15:27 UTC

Monitoring

We have implemented a fix and are monitoring the application closely.
Posted Sep 26, 2016 - 14:52 UTC

Identified

Our engineering team is currently working on a fix for an issue that is affecting the execution of background tasks in Greenhouse Recruiting, affecting many aspects of the application's functionality. We apologize for any inconvenience and will update this page shortly.
Posted Sep 26, 2016 - 14:38 UTC