Golang The Series EP.162: Goroutines for Multi-LLM - Querying Multiple AIs in Parallel for Result Comparison

Hello Gophers! Welcome back to EP.162. Lately, anyone building backends for AI systems probably faces a similar issue: we don't want to tie our system to just one LLM provider. In some use cases, we want to "ask all of them at once"—whether it's OpenAI, Gemini, Claude, or even a local model like Ollama—to use the answers for a fact-checking voting system or to let users compare them side-by-side.

The problem is, if we write code to call each API one by one (Sequential), users will be waiting forever before everyone finishes (assume 3 seconds per provider, 3 providers would take almost 10 seconds). Today, we're going to use Go's specialty, Goroutines and Channels, to perform Concurrent Querying. We'll fire off the question to all providers simultaneously. Whoever is the slowest, that becomes our maximum wait time!

Multi-LLM Architecture via Concurrent Querying

The idea is to spawn Goroutines as independent Workers to call each AI's API. We'll use sync.WaitGroup to orchestrate and wait for all of them to finish. Then, we use Go Channels to collect the results centrally, preventing any Data Race issues.

Plaintext

┌──> [Goroutine 1] ──> Call OpenAI API (1.5s) ──┐
                  │                                             │
[User Question]   ├──> [Goroutine 2] ──> Call Gemini API (2.5s) ──┼──> [Go Channel] ──> Aggregated Result (max wait 3.5s)
                  │                                             │
                  └──> [Goroutine 3] ──> Call Ollama Local (3.5s) ┘

Data Structure and Worker (Types & Function)

To keep our code clean, we separate the Data Structure for the results and create a specific Worker function for querying each provider.

Go

package main

import (
	"context"
	"fmt"
	"strings"
	"sync"
	"time"
)

// LLMResponse is a struct to store results and stats from each AI provider
type LLMResponse struct {
	Provider string
	Answer   string
	Duration time.Duration
	Error    error
}

This function acts as a Worker that calls the API. (In this example, we'll simulate the latency using time.After).

Go

// callLLM acts as a Worker to independently call each Provider's API
func callLLM(ctx context.Context, provider string, delay time.Duration, question string, ch chan<- LLMResponse, wg *sync.WaitGroup) {
	defer wg.Done()
	startTime := time.Now()

	// Simulate Network Latency
	select {
	case <-time.After(delay):
		// Simulate the model's response
		answer := fmt.Sprintf("[%s]: After analyzing the question '%s', I found that...", provider, question)
		
		ch <- LLMResponse{
			Provider: provider,
			Answer:   answer,
			Duration: time.Since(startTime),
			Error:    nil,
		}
	case <-ctx.Done():
		// Case where Context times out or gets canceled
		ch <- LLMResponse{
			Provider: provider,
			Error:    ctx.Err(),
		}
	}
}

Main Orchestrator

In main.go, we run 3 Workers in parallel and set an overall Timeout of 4 seconds to prevent the system from hanging endlessly.

Go

func main() {
	question := "What should be the focus of SEO trends this year?"
	
	// Set an overall timeout of 4 seconds
	ctx, cancel := context.WithTimeout(context.Background(), 4*time.Second)
	defer cancel()

	var wg sync.WaitGroup
	// Create a Buffered Channel with a size equal to the number of providers
	responseChan := make(chan LLMResponse, 3)

	// 1. Distribute parallel AI queries
	wg.Add(1)
	go callLLM(ctx, "OpenAI (GPT-4o)", 1500*time.Millisecond, question, responseChan, &wg)

	wg.Add(1)
	go callLLM(ctx, "Google Gemini", 2500*time.Millisecond, question, responseChan, &wg)

	wg.Add(1)
	go callLLM(ctx, "Ollama (Llama 3)", 3500*time.Millisecond, question, responseChan, &wg)

	// 2. Run a Background Worker to wait and close the Channel once everyone is done
	go func() {
		wg.Wait()
		close(responseChan)
	}()

	fmt.Printf("🚀 Sending the question: \"%s\" to all AI providers simultaneously...\n\n", question)

	// 3. Loop to fetch incoming results from the Channel
	for resp := range responseChan {
		if resp.Error != nil {
			fmt.Printf("❌ [%s] Failed or Timed out: %v\n", resp.Provider, resp.Error)
			fmt.Println(strings.Repeat("-", 50))
			continue
		}
		fmt.Printf("🤖 %s (Took: %v)\n", resp.Provider, resp.Duration)
		fmt.Printf("📝 Answer: %s\n", resp.Answer)
		fmt.Println(strings.Repeat("-", 50))
	}
}

Bonus Technique: First Past the Post

For some projects where we don't need to compare results but just want "maximum speed" (whoever answers first wins), we can easily adapt this code. Just change the loop condition: once we successfully pull the first data from responseChan, call cancel() immediately. This signal will tell other Goroutines waiting for APIs to abort their requests, saving API tokens and significantly reducing server load.

🎯 Daily Mission

Try running this code and push the delay of some providers up to 5000*time.Millisecond. You'll notice that those specific providers get cut off with a Timeout Error instead.

Challenge: If the requirement changes to "Must wait for all 3 answers, then send all of them to a Judge LLM to summarize the best one," how would you design a Slice to receive values from the Channel and pass them on? Try picturing the architecture in your head!

Conclusion & Next Episode (EP.163)

Goroutines and Channels are the true heroes that make Go handle I/O-Bound tasks—like concurrently calling multiple AI APIs—efficiently and safely.

In the next episode (EP.163): Even though this parallel querying is faster, it still incurs redundant API costs for the "same old questions" users frequently ask. Next time, we'll reduce costs by implementing Caching AI Responses with Redis, allowing our system to answer popular questions in milliseconds without bothering the AI. Stay tuned, Gophers!

Follow Superdev Academy on all platforms:

• 🔵 Facebook: Superdev Academy Thailand

• 🎬 YouTube: Superdev Academy Channel

• 📸 Instagram: @superdevacademy

• 🎬 TikTok: @superdevacademy