Golang The Series EP 130: Scalable Multi-instance WebSockets with Redis Pub/Sub

Welcome to one of the most technical and rewarding episodes in this series! If you are building a Chat application, a Real-time Notification system, or a Live Dashboard that needs to support hundreds of thousands of users, you will eventually hit a wall: Horizontal Scaling.

Traditional WebSocket implementations are Stateful—they keep the connection alive in the memory of a specific server. Today, we will break that barrier and build an architecture that allows your WebSockets to scale infinitely across multiple instances.

1. The Challenge: Stateful Hurdles in a Stateless World

Standard HTTP requests are Stateless (request-response-done). WebSockets, however, are Stateful; they maintain a persistent TCP connection between the client and a specific server instance.

Imagine this scenario:

1. We have two servers (Instance A and Instance B) behind a Load Balancer.
2. Alice connects to Instance A.
3. Bob connects to Instance B.
4. When Alice sends a message to Bob, Instance A looks for Bob in its own memory and fails because Bob’s connection is held by Instance B.

Without a central "brain" to sync these instances, Alice's message disappears into a black hole.

2. Why Redis Pub/Sub?

To bridge the gap between instances, we need a communication layer that is incredibly fast and supports broadcasting. Redis Pub/Sub is the perfect fit for several reasons:

• In-Memory Speed: Operating entirely in RAM, it offers sub-millisecond latency.
• Pub/Sub Pattern: One instance Publishes a message, and every other instance that is Subscribed receives it instantly.
• Simplicity: It is far easier to set up and maintain than heavy-duty message brokers like Kafka or RabbitMQ for real-time messaging needs.

3. The "Message Backplane" Architecture

We will treat Redis as the Backplane (the central nervous system) of our system:

1. Publishing: When an instance receives a message from its own client, it doesn't just look locally; it Publishes the message to a Redis Channel.
2. Subscribing: Every running instance Subscribes to that same Redis Channel.
3. Dispatching: When a message arrives via Redis, every instance receives it. Each instance then checks: "Is the recipient connected to me?" If yes, it pushes the message through the local WebSocket.

4. Implementation: Production-Ready Go Code

We will use gorilla/websocket and go-redis/v9. We will also incorporate slog for structured logging, as discussed in EP 128.

A. The Hub and Client Structure

Go
type Hub struct {
    // Registered clients connected specifically to THIS instance
    clients    map[*Client]bool
    broadcast  chan []byte
    register   chan *Client
    unregister chan *Client
    
    // Redis Client for cross-instance communication
    rdb        *redis.Client
}

B. The Redis Listener (The Subscriber)

We need a dedicated Goroutine to listen for messages from Redis indefinitely:

Go
func (h *Hub) listenToRedis(ctx context.Context) {
    pubsub := h.rdb.Subscribe(ctx, "chat_room_global")
    defer pubsub.Close()

    ch := pubsub.Channel()

    for msg := range ch {
        // Upon receiving a message from Redis, send it to the local broadcast channel
        h.broadcast <- []byte(msg.Payload)
    }
}

C. Handling Incoming Messages (The Publisher)

When a client sends a message to the server, we broadcast it to the entire cluster via Redis:

Go
func (c *Client) readPump() {
    defer func() {
        c.hub.unregister <- c
        c.conn.Close()
    }()

    for {
        _, message, err := c.conn.ReadMessage()
        if err != nil {
            slog.Error("Read error", "error", err)
            break
        }
        
        // Instead of local loop, Publish to Redis to reach all instances
        err = c.hub.rdb.Publish(context.Background(), "chat_room_global", message).Err()
        if err != nil {
            slog.Error("Failed to publish to Redis", "error", err)
        }
    }
}

5. Advanced Scaling Tactics

• Sticky Sessions: Even with a backplane, it is highly recommended to enable Sticky Sessions (e.g., ip_hash in Nginx) to minimize unnecessary re-handshaking and reduce CPU overhead.
• JSON Serialization: Never send raw strings. Use structured JSON with a target_user_id or room_id. This allows instances to quickly filter out messages that aren't meant for their local clients.
• Resource Cleanup: Ensure that when a client disconnects, they are immediately unregistered from memory to prevent memory leaks in long-running processes.

Summary

By implementing a Redis Pub/Sub backplane, you transform a collection of isolated WebSocket servers into a powerful, Distributed Real-time System. You can now scale out your instances to meet any demand, while your users enjoy a seamless experience as if they were all connected to a single machine.

"Stateful doesn't have to mean unscalable." Proper architecture makes all the difference.

In the Next Episode (EP 131): We will take this further by integrating our WebSocket system into a Microservices Architecture. We’ll cover Authentication, API Gateways, and how WebSockets live inside a Service Mesh. Stay tuned!