Golang The Series EP.148: Handling Streams - Building Real-time Chat with Go Channels

Welcome to EP.148! In our previous episode, we learned how to receive 100% accurate Structured JSON data. However, in real-world applications, we often encounter a major roadblock: Latency (slowness). The more we ask the AI to analyze, the longer it takes to process. Waiting for a massive JSON chunk to complete can take several seconds, leaving users wondering if the system has frozen or the app has crashed.

Today, we are going to eliminate this bottleneck by leveraging Go's absolute superpowers—Channels and Concurrency. We will manage a Streaming system that transmits data in tiny pieces called Chunks. This allows responses to render on the screen in real-time the moment the AI generates its very first word, delivering that smooth, interactive experience we all love in ChatGPT.

Why Use Go Channels to Handle Streams?

When you enable stream mode with an AI API (whether it's OpenAI or Ollama), the data transmission behavior changes completely. Instead of waiting for a single, massive payload, the API breaks the response down into Chunks—tiny fragments of text that continuously flow through the network connection.

Using Go Channels as an intermediary pipe to receive and forward these small data packets offers 3 distinct advantages over writing a standard, monolithic loop:

Decoupling (Separation of Concerns): Channels allow us to cleanly separate responsibilities. One side is solely responsible for "fetching data from the AI," while the other side only cares about "rendering data for the user." Neither side needs to know the internal workings of the other; they communicate exclusively through the channel.
Non-blocking Execution: Go can immediately process chunk #1 (e.g., sending it to be displayed on the screen) while simultaneously waiting for chunk #2 to arrive over the network via concurrent execution. This completely eliminates idle waiting time, keeping the system highly efficient.
Type Safety: We define exactly what data type our channel accepts (such as chan string or chan MyStruct). This ensures that whatever data flows through the pipeline always conforms strictly to our designed schema, minimizing runtime errors.

💡 Insight for Gophers:

Think of using a channel in this scenario as building a conveyor belt in a factory. The AI API is the machine at the starting line, dropping items onto the belt one by one, and your UI is the worker at the end, picking them up to display. Without this conveyor belt (Channel), you would have to wait for the machine to finish manufacturing every single item before walking over to pick them up all at once—and that is exactly why standard systems feel so slow!

Go Streaming Structure

We will design a function that acts as a Producer. It returns a Receive-only Channel (<-chan), which is like handing over a pipe that can only be used to listen. This design prevents the consumer from accidentally sending data back into the pipe, ensuring a one-way, clean data flow.

Implementation Example:

// Returns a <-chan to signify: "This pipe is for receiving data only"
func StreamAIResponse(ctx context.Context, client *openai.Client, prompt string) <-chan string {
    out := make(chan string)

    // Run a separate Goroutine so we don't block the main function execution
    go func() {
        // Crucial: Always close the channel when finished to tell the receiver "no more data"
        defer close(out) 
        
        req := openai.ChatCompletionRequest{
            Model:  openai.GPT4o,
            Stream: true, // The game changer: Enable streaming to let the API spit out chunks
            Messages: []openai.ChatCompletionMessage{
                {Role: "user", Content: prompt},
            },
        }

        stream, err := client.CreateChatCompletionStream(ctx, req)
        if err != nil {
            // In production, consider sending the error through the channel or logging it
            return 
        }
        defer stream.Close()

        for {
            response, err := stream.Recv()
            // io.EOF is the signal that the AI has finished its sentence
            if errors.Is(err, io.EOF) {
                return 
            }
            if err != nil {
                return
            }
            
            // Immediately push the "Delta" (newly received fragment) into the pipe
            if len(response.Choices) > 0 {
                out <- response.Choices[0].Delta.Content
            }
        }
    }()

    return out
}

🔍 A Deep Dive into the Code:

go func() { ... }(): We offload the API data listening to the background (Goroutine). This allows the StreamAIResponse function to return the channel to the user immediately, without having to wait for the AI to finish its entire response.
defer close(out): This is essential channel etiquette. If we don't close the pipe, the receiver waiting for data (using a range loop) will wait forever, leading to a Deadlock.
stream.Recv(): This function blocks execution inside the Goroutine while waiting for the next data fragment from the AI. Once a chunk arrives, we quickly shove it into the pipe using out <-.

Consumption (Usage)

At the Main or Controller level (the end of the pipe), we don't need to worry about API complexity or managing Goroutines. Our only job is to wait for the data flowing out of the channel using the for range keyword.

Usage Example:

// 1. Call the function and receive the Channel
responseChan := StreamAIResponse(ctx, client, "Explain Go Channels to me")

// 2. Use range to pull data from the pipe until close() is called
for msg := range responseChan {
    // 3. Display each character/word immediately as it arrives
    fmt.Print(msg) 
}

fmt.Println("\n--- Streaming Finished ---")

Why is this approach excellent?

Smooth UX: The text doesn't stutter or arrive in a massive block. Instead, it appears gradually with a "Typing Effect," significantly reducing user frustration during long AI processing times.
Memory Efficiency: We don't have to store long strings in memory until the process finishes. We process (in this case, print) the data bit by bit as it arrives.
Automatic Termination: When the background Goroutine finishes and calls close(out), the for range loop exits gracefully and safely without us having to write complex break conditions.

Critical Precautions for Streaming

While Go handles concurrency beautifully, developers must be mindful of two key areas to ensure system safety:

1. Context Cancellation

This is the most common mistake! If a user clicks Stop or closes their browser while the AI is mid-sentence, and you haven't passed the ctx (Context) into CreateChatCompletionStream, your backend will continue running the loop and receiving data until the AI finishes.

The Consequences: Wasted tokens (losing money!) and high server load for results no one is watching.
The Solution: Always verify that the ctx is still active and ensure stream.Close() is triggered immediately upon context cancellation.

2. Buffer Size & Backpressure

Typically, for chat applications, we use an Unbuffered Channel (size 0) to ensure data freshness. We want the information to flow from the API to the user's screen instantly without sitting in a queue.

When to use a Buffered Channel?: If the "Consumer" (receiver) processes data slower than the sender (e.g., if you need to run data through a filter before displaying it), using a Buffered Channel (e.g., make(chan string, 10)) allows the sender to keep moving without waiting for the receiver, reducing stream stuttering.

💡 Rule of Thumb for Gophers:

"If you open it, you must close it." — Every time you use go func and channel, you must be able to answer: When will this Goroutine end? and When will this channel be closed? If you can't answer those, you likely have a Goroutine Leak, which will slowly consume your server's memory.

Daily Mission

To deepen your understanding of data flow, let's upgrade our stream function to a professional level. Instead of sending raw strings, we will transmit a rich context through the channel.

Challenge: Create a data structure and refactor your function to communicate using this struct:

type StreamResponse struct {
    Content      string
    FinishReason string
    Err          error
}

// Homework: Refactor StreamAIResponse to return <-chan StreamResponse

Food for Thought:

Try to "Mental Benchmark" these two scenarios:

Standard Mode: A 5-second wait, then a massive wall of text appears all at once.
Streaming Mode: A 0.5-second wait, and the first word starts "typing" out until it finishes at the 5-second mark.

In which scenario is a user more likely to hit Refresh or close the app? This is Perceived Performance—the speed the user feels. In the world of UX, the feeling of speed is just as critical as the actual execution time of your code.

Conclusion

Implementing Streaming with Go Channels transforms your application from a "frozen" interface into a living, breathing experience. Go's clean concurrency model allows us to turn complex real-time logic into something simple, readable, and safe.

But with great speed comes great responsibility...

Coming Up Next | EP.149:

Now that your system is smooth and your users are happy, there’s one thing that might make you unhappy: the API bill at the end of the month! We’re diving deep into budget control in "Token Management: How to Count Tokens and Calculate API Costs on the Backend."

We'll explore how AI sees our text as numbers and how we can "predict" the cost before hitting that API. If you don't want a heart attack when your invoice arrives... don't miss this one!

See you in the next episode, Gophers!

Follow Superdev Academy on all platforms:

🔵 Facebook: Superdev Academy Thailand
🎬 YouTube: Superdev Academy Channel
📸 Instagram: @superdevacademy
🎬 TikTok: @superdevacademy
🌐 Website: superdevacademy.com