Golang The Series EP.160: Building an Enterprise Internal AI Tool (RAG) with Qdrant and Gin

Welcome back, Superdev Academy! Welcome to EP.160, our major hands-on workshop for this season. Throughout this series, we’ve covered everything from environment setup, text embeddings, strategic data chunking, to advanced RAG (Retrieval-Augmented Generation) concepts like Hybrid Search and Context Injection.

Today, it's time to bring all these pieces together to build a robust Internal AI Tool for an enterprise using Go (Golang). We will create an intelligent Q&A server capable of querying internal knowledge bases, enforcing department-level access control via Hybrid Search, and streaming real-time responses to users using Server-Sent Events (SSE).

Project Structure

To ensure our code remains modular, maintainable, and adheres to clean architecture principles, we will organize our project using a Layered Architecture:

Plaintext

internal-ai-tool/
├── config/
│   └── qdrant.go      # Manages Qdrant Vector DB connections
├── handlers/
│   └── qa.go          # HTTP Handlers for incoming requests and SSE streaming
├── services/
│   ├── ai.go          # Wrapper for OpenAI API (Embeddings & Chat Stream)
│   └── vector.go      # Handles Hybrid Search queries on Qdrant
├── go.mod
├── go.sum
└── main.go            # Entry point and dependency injection setup

Let's initialize our Go module and install the required core dependencies:

Bash

go mod init internal-ai-tool
go get github.com/gin-gonic/gin
go get github.com/sashabaranov/go-openai
go get github.com/qdrant/go-client/qdrant
go get github.com/google/uuid

Configuration & Services Layer

In this layer, we isolate the database connection logic and core service engines following the Separation of Concerns principle.

1. Vector DB Connection Setup (config/qdrant.go)

We will implement an initialization function to establish a connection with the Qdrant Vector Database. This client will be injected into our services later via the main.go file.

Go

package config

import (
	"log"
	"github.com/qdrant/go-client/qdrant"
)

// NewQdrantClient establishes and returns a new client connection for Qdrant Vector DB
func NewQdrantClient(host string, port int) *qdrant.Client {
	client, err := qdrant.NewClient(&qdrant.Config{
		Host: host,
		Port: port,
	})
	if err != nil {
		log.Fatalf("❌ Failed to connect to Qdrant: %v", err)
	}
	return client
}

2. OpenAI Service Integration (services/ai.go)

This service handles all communication with the OpenAI API, specifically generating text embeddings for user queries and initiating a Chat Completion Stream using the gpt-4o model.

Go

package services

import (
	"context"
	"github.com/sashabaranov/go-openai"
)

type AIService struct {
	client *openai.Client
}

func NewAIService(apiKey string) *AIService {
	return &AIService{client: openai.NewClient(apiKey)}
}

// CreateEmbedding converts raw input text into a high-dimensional semantic vector
func (s *AIService) CreateEmbedding(ctx context.Context, text string) ([]float32, error) {
	req := openai.EmbeddingRequest{
		Input: []string{text},
		Model: openai.SmallEmbedding3Small, // Cost-efficient and highly performant model
	}
	resp, err := s.client.CreateEmbeddings(ctx, req)
	if err != nil {
		return nil, err
	}
	return resp.Data[0].Embedding, nil
}

// GetChatStream sends a completion request to OpenAI and returns a real-time stream
func (s *AIService) GetChatStream(ctx context.Context, systemPrompt, userPrompt string) (*openai.ChatCompletionStream, error) {
	return s.client.CreateChatCompletionStream(ctx, openai.ChatCompletionRequest{
		Model: openai.GPT4o,
		Messages: []openai.ChatCompletionMessage{
			{Role: openai.ChatMessageRoleSystem, Content: systemPrompt},
			{Role: openai.ChatMessageRoleUser, Content: userPrompt},
		},
	})
}

3. Secure Vector Search Service (services/vector.go)

The core of our RAG mechanism lies here. We utilize Qdrant's querying features to perform a Hybrid Search with a strict security filter, ensuring employees can only retrieve active documents matching their specific department.

Go

package services

import (
	"context"
	"github.com/qdrant/go-client/qdrant"
)

type VectorService struct {
	client *qdrant.Client
}

func NewVectorService(client *qdrant.Client) *VectorService {
	return &VectorService{client: client}
}

// SearchRelevantContext queries Qdrant using a vector while enforcing a department-level security filter
func (s *VectorService) SearchRelevantContext(ctx context.Context, vector []float32, department string) ([]string, error) {
	searchLimit := uint64(3) // Fetch top 3 most relevant context chunks
	
	// Security Filter: Only match active documents belonging to the user's department
	searchFilters := &qdrant.Filter{
		Must: []*qdrant.Condition{
			qdrant.NewMatchKeyword("department", department),
			qdrant.NewMatchKeyword("status", "active"),
		},
	}

	resp, err := s.client.Query(ctx, &qdrant.QueryPoints{
		CollectionName: "ai_knowledge_base",
		Query:          qdrant.NewQuery(vector...),
		Filter:         searchFilters,
		Limit:          &searchLimit,
	})
	if err != nil {
		return nil, err
	}

	var chunks []string
	for _, point := range resp {
		if contentVal, exists := point.Payload["content"]; exists {
			chunks = append(chunks, contentVal.GetStringValue())
		}
	}
	return chunks, nil
}

API Endpoint Layer & Server-Sent Events (SSE) Streaming

This layer manages client requests and coordinates data processing between our services. We use the Gin Gonic framework to create our HTTP API endpoint and stream tokenized responses back to the frontend in real time using Server-Sent Events (SSE).

1. Request Struct & Validation

First, we set up our Data Transfer Object (DTO) structure to capture incoming payloads and enforce required parameters.

Go

package handlers

import (
	"errors"
	"fmt"
	"io"
	"net/http"
	"strings"

	"internal-ai-tool/services"
	"github.com/gin-gonic/gin"
)

// QARequest binds incoming JSON payloads from the frontend client
type QARequest struct {
	Question   string `json:"question" binding:"required"`
	Department string `json:"department" binding:"required"` // Used for access control filtering
}

2. Query Retrieval & Context Injection

Next, inside our HandleQAStream handler, we take the user's question, turn it into an embedding vector, look up authorized document chunks, and inject them cleanly into our LLM prompt template.

Go

// HandleQAStream controls the overall RAG pipeline execution flow
func HandleQAStream(ai *services.AIService, vector *services.VectorService) gin.HandlerFunc {
	return func(c *gin.Context) {
		var req QARequest
		if err := c.ShouldBindJSON(&req); err != nil {
			c.JSON(http.StatusBadRequest, gin.H{"error": "Invalid request parameters"})
			return
		}

		ctx := c.Request.Context()

		// Step 1: Convert user question into a semantic text embedding vector
		queryVector, err := ai.CreateEmbedding(ctx, req.Question)
		if err != nil {
			c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to generate embedding vector"})
			return
		}

		// Step 2: Retrieve authorized knowledge base chunks with security filters applied
		chunks, err := vector.SearchRelevantContext(ctx, queryVector, req.Department)
		if err != nil {
			c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to retrieve knowledge base context"})
			return
		}

		// Step 3: Perform Context Injection to ground the LLM's response scope
		contextText := ""
		if len(chunks) > 0 {
			contextText = strings.Join(chunks, "\n\n")
		} else {
			contextText = "No relevant context found for this department scope."
		}

		systemPrompt := "You are an internal corporate AI assistant. Your sole job is to answer employee questions using the provided Context only. If the answer cannot be found in the Context, state clearly that you cannot find it. Never hallucinate."
		userPrompt := fmt.Sprintf("Context:\n%s\n\nQuestion: %s\nAnswer:", contextText, req.Question)

3. Server-Sent Events (SSE) Streaming Output

Finally, we configure the HTTP response headers to stay open and feed the response pieces to the frontend interface as soon as they are computed.

Go

		// Step 4: Open Chat Completion Stream and configure SSE response pipeline
		stream, err := ai.GetChatStream(ctx, systemPrompt, userPrompt)
		if err != nil {
			c.JSON(http.StatusInternalServerError, gin.H{"error": "Failed to initialize completion stream"})
			return
		}
		defer stream.Close()

		// Set HTTP headers to support persistent text/event-stream connections
		c.Header("Content-Type", "text/event-stream")
		c.Header("Cache-Control", "no-cache")
		c.Header("Connection", "keep-alive")

		// Continuously stream response text tokens out to the frontend client in real-time
		c.Stream(func(w io.Writer) bool {
			resp, err := stream.Recv()
			if errors.Is(err, io.EOF) {
				c.SSEvent("message", "[DONE]") // Emit closing signal to inform the UI that streaming is finished
				return false
			}
			if err != nil {
				return false
			}

			if len(resp.Choices) > 0 {
				content := resp.Choices[0].Delta.Content
				if content != "" {
					c.SSEvent("message", content)
				}
			}
			return true
		})
	}
}

Assembling the Application (main.go)

The main.go file acts as the central control room. It loads configuration values, handles secure environment variables, performs Dependency Injection (DI) to wire up our handlers and services, and spins up our engine using Gin Gonic.

1. System Initialization & Dependency Injection

We configure the entry point to load sensitive keys and securely instantiate our architecture layout.

Go

package main

import (
	"log"
	"os"

	"internal-ai-tool/config"
	"internal-ai-tool/handlers"
	"internal-ai-tool/services"
	"github.com/gin-gonic/gin"
)

func main() {
	// 1. Fetch credentials from Environment Variables adhering to enterprise safety standards
	openAIKey := os.Getenv("OPENAI_API_KEY")
	if openAIKey == "" {
		log.Fatal("ERROR: Environment variable OPENAI_API_KEY is not set")
	}

	// 2. Initialize connection to Qdrant Vector DB
	qdrantClient := config.NewQdrantClient("localhost", 6334)
	defer qdrantClient.Close() // Ensure connections close safely when server terminates

	// 3. Execute Dependency Injection (DI) to assemble service instances
	aiService := services.NewAIService(openAIKey)
	vectorService := services.NewVectorService(qdrantClient)

2. Routing Setup & Server Initialization

We bind our handler to an API route path and hook it up to standard execution loops.

Go

	// 4. Create an HTTP routing instances using Gin Gonic
	r := gin.Default()

	// Register secure POST route path for streaming internal context answers
	r.POST("/api/v1/qa/stream", handlers.HandleQAStream(aiService, vectorService))

	// Run application listening on predefined port interfaces
	log.Println("🚀 Enterprise Internal AI Tool server running smoothly on port :8080...")
	r.Run(":8080")
}

🎯 Daily Mission

Congratulations! You now have a production-ready, clean, and secure enterprise RAG backend architecture ready to interface with a frontend UI.

Your Mission: Spin up this server and use Postman or curl to send a test request. Try swapping the "department" field in your JSON request body between "Human Resources" and "Engineering" using the same question.

Observe how the backend Security Filter handles the authorization rules. Does the AI limit or tailor its response strictly based on the department privileges defined in the Qdrant query? Test it out and prove your backend skills!

FAQ

Why use Server-Sent Events (SSE) instead of WebSockets for streaming AI responses?

AI chat streaming is inherently a one-way downstream process (server-to-client). SSE runs on standard HTTP protocols, making it lightweight, easier to scale, and more efficient than WebSockets, which are designed for persistent bi-directional communication.

Will query performance degrade as the Qdrant database grows?

Without optimization, latency might increase. We highly recommend setting up Payload Indexing on fields used in your filters (e.g., department and status) to keep Qdrant vector queries running within single-digit milliseconds.

Summary

In this episode, we successfully covered:

• Designing a layered RAG architecture using Go (Golang).

• Managing vector database connections and filtering access scopes with Qdrant Vector DB.

• Implementing Context Injection to ground OpenAI GPT-4o models with relevant company facts.

• Building an asynchronous, real-time streaming HTTP endpoint via Server-Sent Events (SSE) in Gin Gonic.

Coming Up Next (EP.161): We've successfully wrapped up our fundamental RAG architecture workshop. But what happens in a real-world enterprise when hundreds of employees upload massive procurement guidelines or onboarding manuals simultaneously? Processing those heavy workloads synchronously over single API calls will inevitably cause timeouts and crash your server.

In the next episode, we’ll step into advanced systems design with "Async AI Tasks: Using Worker Pools to Handle Large-Scale AI Data Extraction." If you're building highly scalable backend architectures, you won't want to miss this one. See you next time, Gophers of Superdev Academy!

Follow Superdev Academy on all platforms:

• 🔵 Facebook: Superdev Academy Thailand

• 🎬 YouTube: Superdev Academy Channel

• 📸 Instagram: @superdevacademy

• 🎬 TikTok: @superdevacademy

• 🌐 Website: superdevacademy.com