Preventing Cascading Failures

Preventing Cascading Failures An architecture diagram generated by Archify. Service A · Resource exhaustion · Without Circuit Breaker · Waiting on timeouts Service A Resource exhaustion Waiting on timeouts Service B · Failure · Without Circuit Breaker Service B Failure Service C · Failure · Without Circuit Breaker Service C Failure Service D · Failure · Without Circuit Breaker Service D Failure Service A · Normal · With Circuit Breaker · Resources protected Service A Normal Resources protected Service B · Circuit Open · With Circuit Breaker Service B Circuit Open Service C · Normal · With Circuit Breaker Service C Normal Service D · Normal · With Circuit Breaker Service D Normal Slow response Accumulated timeouts Cascading failure Fast fail Normal operation Normal operation Without Circuit Breaker With Circuit Breaker Legend Healthy service Circuit Open (isolation point) Failed service

Without Circuit Breaker

  • • Wait until timeout (30s+)
  • • Thread/connection exhaustion
  • • Cascading failures occur
  • • Manual intervention to recover

With Circuit Breaker

  • • Immediate failure (1ms)
  • • Resources protected
  • • Failure isolation
  • • Automatic recovery attempts