Rule State and Incident Response

Rule State and Incident Response A lifecycle diagram generated by Archify. 01 / Detection + notification 02 / Responder action 03 / Outcome Inactive · condition not met · Detection + notification Inactive condition not met Pending · held for the for: duration · Detection + notification · for: 5m Pending held for the for: duration for: 5m Firing · rule condition active · Detection + notification Firing rule condition active Notified · Slack, Email, PagerDuty · Detection + notification Notified Slack, Email, PagerDuty Acknowledged · responder confirmed · Detection + notification Acknowledged responder confirmed In Progress · responder fixing the problem · Responder action In Progress responder fixing the problem Incident Closed · verify rule + signal health · Outcome Incident Closed verify rule + signal health Threshold exceeded Wait elapsed Alert sent Responder confirmed Action in progress Problem solved Returns within threshold No active series Legend inactive handled waiting resolved firing

Rule

  • • inactive / pending / firing
  • • for delays firing while the condition persists

Delivery

  • • Grouping, inhibition and reminders are configured
  • • No exactly-once delivery guarantee

Incident

  • • Acknowledgment and closure belong to the incident tool
  • • A cleared rule is not proof of recovery