View : 126

04/08/2026 04:12am

Preventing system failures with Circuit Breaker Pattern in Go using sonygobreaker

Golang The Series EP.166: Circuit Breaker Pattern How to Handle AI API Outages

#Golang

#Circuit Breaker

#Go API

#Cascading Failure

#Backend Development

#AI API

#Go

Welcome back to EP.166! In EP.165, we learned how to distribute workloads across multiple AI nodes using a Load Balancer to handle massive traffic. However, in a real-world production environment, networks and external services are always unpredictable.

Imagine this: The primary AI provider's API suffers an outage, or your on-premise network experiences a latency spike from 2 seconds to 30 seconds. The consequence isn't just a slow AI response; incoming requests pile up, Goroutines consume all application memory resources, and your entire backend system collapses in a domino effect known as a Cascading Failure.

To prevent our core system from crashing along with the downstream AI, we can implement an architectural pattern called the Circuit Breaker Pattern to quarantine the failure!

Understanding the 3 States of a Circuit Breaker

A Circuit Breaker acts like an electrical circuit breaker in a house. When a short circuit occurs, the breaker trips immediately to prevent a fire. It operates in three main states:

Plaintext

               ┌─────────────────────────────────────────────────────────┐
               │                                                         │
               ▼                                                         │
         ┌──────────┐      (High Error Rate Threshold)    ┌──────────┐   │ (Test Request Success)
         │  CLOSED  │ ───────────────────────────────────> │   OPEN   │   │
         │ (Closed) │                                     │  (Open)  │   │
         └──────────┘                                      └──────────┘   │
              ▲                                                  │        │
              │                                                  │        │
              │                                                  │ (Cooldown Period)
              │                                                  ▼        │
              │                (Test Request Failed)       ┌───────────┐  │
              └─────────────────────────────────────────── │ HALF-OPEN │ ─┘
                                                           │(Half-Open)│
                                                           └───────────┘

  • CLOSED (Normal State): The switch is closed, and requests flow freely to the AI API. The system tracks the failure rate in memory.

  • OPEN (Tripped / Failure State): When errors or timeouts exceed the threshold, the breaker trips open! Any incoming requests are rejected immediately (Fail Fast) without wasting time calling the failed AI API.

  • HALF-OPEN (Testing State): After a cooldown period, the breaker allows a small number of test requests to pass through to the AI API. If they succeed, it transitions back to CLOSED; if they fail, it trips back to OPEN.

Installing Dependencies with Sonygobreaker

One of the most stable, high-performance, and popular Circuit Breaker libraries in the Go ecosystem is sonygobreaker by Sony:

Bash

go get github.com/sony/gobreaker

Structure and Go Code Implementation with Circuit Breaker

We separate the mock API call function and the circuit breaker controller for clean and readable code.

Part 1: External API Call Function and Failure Simulation

Go

package main

import (
	"context"
	"errors"
	"fmt"
	"log"
	"time"

	"[github.com/sony/gobreaker](https://github.com/sony/gobreaker)"
)

// CallExternalAIAPI simulates calling an external AI API
func CallExternalAIAPI(ctx context.Context, question string, simulateFailure bool) (string, error) {
	if simulateFailure {
		// Simulate AI API outage or timeout
		time.Sleep(200 * time.Millisecond)
		return "", errors.New("503 Service Unavailable: AI Model Overloaded")
	}

	// Normal operation case
	return fmt.Sprintf("AI response for question '%s': Processing successful...", question), nil
}

Part 2: Circuit Breaker Configuration and Main Controller: main.go

Go

func main() {
	// 1. Configure the Circuit Breaker settings
	settings := gobreaker.Settings{
		Name:        "AI-API-Breaker",
		MaxRequests: 2,                // Number of requests allowed to pass during Half-Open state
		Interval:    5 * time.Second,  // Clears error statistics every 5 seconds
		Timeout:     3 * time.Second,  // Cooldown period in Open state before transitioning to Half-Open
		ReadyToTrip: func(counts gobreaker.Counts) bool {
			// Trip condition: If requests >= 3 and failure rate >= 50%, trip immediately!
			failureRatio := float64(counts.TotalFailures) / float64(counts.Requests)
			return counts.Requests >= 3 && failureRatio >= 0.5
		},
		OnStateChange: func(name string, from gobreaker.State, to gobreaker.State) {
			// Callback observer when the circuit state changes
			log.Printf("⚠️ [Circuit Breaker: %s] State changed from %s -> %s\n", name, from, to)
		},
	}

	cb := gobreaker.NewCircuitBreaker(settings)
	question := "Please summarize the annual report."

	// 2. Simulate 10 consecutive requests to observe trip behavior
	fmt.Println("🚀 Starting request simulation to AI API...")
	for i := 1; i <= 10; i++ {
		// Simulate API failure for requests 1 to 5, then recover afterwards
		simulateFailure := i <= 5

		// Execute controls whether the request goes through to the actual API
		result, err := cb.Execute(func() (interface{}, error) {
			return CallExternalAIAPI(context.Background(), question, simulateFailure)
		})

		if err != nil {
			// Check if the error is caused by the Circuit Breaker tripping
			if errors.Is(err, gobreaker.ErrOpenState) {
				fmt.Printf("🔴 [Req #%d] Circuit Breaker OPEN! (Tripped) -> [Fallback]: 'AI service is currently unavailable. Please use standard search.'\n", i)
			} else {
				fmt.Printf("❌ [Req #%d] API Error: %v -> [Fallback]: 'Sorry, unable to process response.'\n", i, err)
			}
		} else {
			fmt.Printf("🟢 [Req #%d] Success: %v\n", i, result)
		}

		time.Sleep(800 * time.Millisecond)
	}
}

Fallback Mechanism Strategies

When the Circuit Breaker trips to OPEN, the system should not leave users stranded with frozen screens or cryptic error messages. Here are strategies you can implement:

  1. Degraded Service (Model Failover): Switch to a smaller, locally-hosted model (such as Llama-3-8B) instead of the heavy cloud model.

  2. Static / Cached Answer: Fetch a basic response from Redis Cache (covered in EP.163) as an instant substitute.

  3. Graceful Degradation Message: Display a friendly notification with alternative actions, e.g., "The AI assistant is experiencing high traffic. Your query has been saved to the queue."

🎯 Daily Mission

Try running the code example above and observe the console output showing state transitions from CLOSED to OPEN and tripping immediately under failure conditions.

Challenge Question: If your organization uses two primary AI APIs—OpenAI (Primary) and Gemini (Secondary)—how would you design a Go fallback function so that when OpenAI's Circuit Breaker trips (OPEN), the system automatically routes requests to Gemini? Try designing this architecture!

Frequently Asked Questions (FAQ)

How does the Circuit Breaker know when the downstream AI API has recovered?

Once the Timeout duration expires while in the OPEN state, the switch automatically transitions to the HALF-OPEN state to test a limited number of requests (defined by MaxRequests). If successful, the system immediately switches back to CLOSED. If it still fails, a new cooldown timer starts.

What Timeout and Error Rate values should we use?

There are no fixed numbers; it depends on the AI provider's SLA and user behavior. Generally, it is recommended to set a Timeout of 3–5 seconds (to prevent users from waiting too long) and trigger ReadyToTrip when the error rate exceeds 50% out of at least 3–5 test requests, preventing premature tripping from transient faults.

How is a Circuit Breaker different from a Rate Limiter?

A Rate Limiter controls inbound request volume to protect your system from traffic spikes or quota limits. A Circuit Breaker protects your system outbound when calling external dependencies that are failing or lagging, preventing cascading failures.

Conclusion

In this article, we explored the Circuit Breaker Pattern, an essential tool for preventing Cascading Failures when external AI APIs experience outages or high latencies. We covered the three main states—CLOSED, OPEN, and HALF-OPEN—and wrote practical Go code using gobreaker to control executions while designing fallback mechanisms to keep our backend stable and deliver a great user experience even during partial system failures.

Coming up in EP.167: Our system now features Multi-LLM, Cache, Rate Limiter, Load Balancer, and Circuit Breaker. But the question is... how do we know how well everything is performing? What is the latency of each AI provider? Next time, we will install a system health monitor with "Monitoring AI Performance — Using Prometheus to Track Speed and Measure Latency". Don't miss it, Gophers!

Follow Superdev Academy on all platforms: