When your WebSocket user base grows from hundreds to tens of thousands, you’ll immediately face challenges such as load spikes, duplicate connections, and unbalanced traffic ⚡

In this episode, we’ll explore how to design a WebSocket Server that can scale automatically (Auto-Scaling) and distribute load efficiently (Load Balancing) using production-grade tools like: Kubernetes, Load Balancer, Sticky Sessions, and Redis Pub/Sub

🧩 1. Why Auto-Scaling and Load Balancing?

A production-grade WebSocket infrastructure must scale up/down automatically based on traffic demands.

⚙️ 2. Auto-Scaling WebSocket System Architecture

Client
   ↓
Load Balancer (Sticky Session)
   ↓
WebSocket Server Pods (Kubernetes)
   ↓
Redis Pub/Sub (Message Sync)
   ↓
Database / Services

🔑 Key Components

• Load Balancer → Distributes users across Pods
• Sticky Session → Keeps users connected to the same Pod
• Redis Pub/Sub → Syncs messages across all instances in real-time

🧠 3. Code Example: Syncing Messages with Redis Pub/Sub

// Install dependencies first:
// go get github.com/redis/go-redis/v9
// go get github.com/gorilla/websocket

var (
	upgrader = websocket.Upgrader{CheckOrigin: func(r *http.Request) bool { return true }}
	rdb      = redis.NewClient(&redis.Options{Addr: "localhost:6379"})
	ctx      = context.Background()
	clients  = make(map[*websocket.Conn]bool)
)

func handleConnections(w http.ResponseWriter, r *http.Request) {
	conn, _ := upgrader.Upgrade(w, r, nil)
	defer conn.Close()
	clients[conn] = true
	defer delete(clients, conn)

	for {
		_, msg, err := conn.ReadMessage()
		if err != nil {
			break
		}
		// Publish to Redis
		rdb.Publish(ctx, "chat_channel", msg)
	}
}

func handleRedisMessages() {
	subscriber := rdb.Subscribe(ctx, "chat_channel")
	ch := subscriber.Channel()
	for msg := range ch {
		for client := range clients {
			client.WriteMessage(websocket.TextMessage, []byte(msg.Payload))
		}
	}
}

func main() {
	go handleRedisMessages()
	http.HandleFunc("/ws", handleConnections)
	log.Println("WebSocket Server running on :8080")
	http.ListenAndServe(":8080", nil)
}

✅ Every message sent by any client is published to Redis, then broadcasted in real-time to all connected clients across all server instances.

🧩 4. Load Balancer + Sticky Session

Sticky Sessions are essential in a multi-instance setup to maintain consistent connections between the user and the same Pod.

Example Nginx Ingress configuration:

annotations:
  nginx.ingress.kubernetes.io/affinity: "cookie"
  nginx.ingress.kubernetes.io/session-cookie-name: "route"
  nginx.ingress.kubernetes.io/session-cookie-hash: "sha1"

👉 This helps stabilize the connection even under high load, and reduces reconnection issues and message loss.

☁️ 5. Auto-Scaling on Kubernetes

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: websocket-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: websocket-deployment
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

💡 When CPU usage exceeds 70%, the system automatically adds more Pods.

🧠 6. Best Practices for Scaling WebSocket Systems

🚀 Challenge!

Try this real-world simulation:

1. Deploy 2 WebSocket server instances
2. Connect both to Redis Pub/Sub
3. Open 2 browsers (or devices), and send a message from one

✅ If the message is instantly received on both sides → Congrats!
You’re now running a production-grade WebSocket infrastructure 🚀

🌟 Coming Up Next

📘 EP.111: Message Ordering and Event Sequence Management

Learn how to ensure perfect message sequencing in your WebSocket system, even when handling thousands of concurrent connections so every event reaches users in 100% correct order.