View : 106

16/06/2026 04:00am

Go code implementation using go-openai library to convert text into Vector Embeddings array via OpenAI API

Golang The Series EP.152: Intro to Embeddings — Converting Text into Vectors with Go

#Vector Embeddings

#Go OpenAI

#Go

#Text Embedding

#Go Concurrency

#Data Pipeline

#RAG Backend

Welcome to EP.152! In our previous episode, we discussed how RAG allows an AI to take an "open-book exam." However, as backend developers, the critical question we must answer next is: "How do we search through that book to find the exact sentences that match our user's intent?"

If we rely on traditional SQL queries like WHERE content LIKE '%payment%', and an indie user asks, "How do I fork over the cash?", our system will fail to retrieve the document. Even though the intent is identical, the actual words share zero characters in common.

To solve this, the AI world uses Embeddings. It is the process of converting human language into a structured array of numbers (Vectors), enabling computers to mathematically calculate and understand word definitions. Today, we’re going to build this pipeline using Go!

What are Vector Embeddings?

An Embedding takes a piece of text (a word, a sentence, or a whole article) and processes it through a specialized AI model (like OpenAI's text-embedding-3-small). The model outputs a fixed-length array of floating-point numbers ([]float32), usually spanning 1,536 dimensions.

The brilliance behind this lies in two aspects:

  • Semantic Closeness: Words with similar meanings, such as "cat" and "kitten," or "payment" and "cash out," will yield coordinate numbers that are positioned very close to one another in the semantic space.

  • Math Over Language: Computers do not understand words; they understand mathematics. By turning text into numbers, we can use simple geometric formulas (like Cosine Similarity) to calculate whether two distinct sentences are talking about the same topic.

Implementing OpenAI Embeddings in Go

We will use our trusted go-openai library to pass our strings to the API and retrieve the vectors.

Go

package main

import (
	"context"
	"fmt"
	"log"
	"os"

	"github.com/sashabaranov/go-openai"
)

func main() {
	// Retrieve API Key from Environment Variables
	apiKey := os.Getenv("OPENAI_API_KEY")
	if apiKey == "" {
		log.Fatal("Embedding API Key is required")
	}

	client := openai.NewClient(apiKey)

	// The text we want to process semantic meaning for
	inputText := "I want to pay with my credit card"

	// 1. Construct the Embedding Request
	req := openai.EmbeddingRequest{
		Input: []string{inputText},
		Model: openai.SmallEmbedding3Small, // Popular 1,536-dimension model: fast and cost-effective
	}

	// 2. Send request to OpenAI API
	resp, err := client.CreateEmbeddings(context.Background(), req)
	if err != nil {
		log.Fatalf("Embedding failed: %v", err)
	}

	// 3. Extract the Vector result ([]float32)
	vector := resp.Data[0].Embedding

	fmt.Printf("Text: '%s'\n", inputText)
	fmt.Printf("Vector Dimensions: %d\n", len(vector))
	fmt.Printf("First 5 Dimensions Sample: %v\n", vector[:5])
}

Production Data Structure

In a production RAG application, once you generate these vectors, you typically store them alongside the raw source text inside a struct. This sets them up for distance calculations later:

Go

type DocumentChunk struct {
	ID        string    `json:"id"`
	Content   string    `json:"content"`   // Raw text, e.g., "7-Day Return Policy Manual"
	Embedding []float32 `json:"embedding"` // The 1,536-dimension array from the API
}

Why Golang Excels at Embedding Pipelines

When scaling an enterprise RAG application, you may need to convert hundreds of thousands of document pages into vectors. This workload is heavily I/O-bound (API requests) and compute-bound (data processing) simultaneously. Go dominates this playing field:

  • Goroutine Worker Pools: You can easily read documents, chunk them, and dispatch hundreds of concurrent embedding API requests via Go's concurrency model, shrinking processing times from hours to minutes.

  • Memory Efficiency: Managing millions of []float32 arrays can cause massive memory spikes if handled poorly. Go’s low runtime overhead and predictable memory management ensure your infrastructure remains light and stable.

🎯 Daily Mission: Challenge Yourself

Initialize 3 sample sentences inside your Go codebase:

  1. "How do I change my account password?"

  2. "Steps to reset my credentials."

  3. "It is raining heavily in Bangkok today."

Your Task: Modify your program to generate embeddings for all 3 sentences simultaneously. Loop through the output to inspect the values. For an extra engineering challenge, write a basic function to calculate the distance (like a Dot Product) between [1 and 2] versus [1 and 3]. Which pair yields a closer mathematical score?

❓ FAQ: Frequently Asked Questions about Embeddings

What does a 1,536-dimensional array actually represent?

Think of it as a physical map. While a standard chart uses 2D coordinates (X, Y), an AI model maps concepts onto a 1,536-dimensional matrix. Each dimension tracks a specific semantic feature—such as formality, financial context, or animal relations. More dimensions mean deeper semantic comprehension.

If I embed a Thai sentence and an English sentence with the same meaning, will their vectors match?

They won't match identically, but they will be remarkably close! Modern models like text-embedding-3-small are multilingual. They align meanings across languages, so words like "แมว" and "Cat" map out to near-identical coordinates in the vector space.

📌 Conclusion

Vector Embeddings act as the ultimate bridge translating human conversations into mathematical structures that computers can reason with. They elevate your search capabilities from strict word matching to full semantic understanding. When backed by Go's concurrent architecture, handling high-throughput pipeline transformations becomes an effortlessly scalable task.

In the Next Episode (EP.153): Once you have millions of []float32 vectors floating around your system, where do you store them to search and compare them in milliseconds? Next time, we enter the world of "Vector Databases 101: Meet Pinecone, Weaviate, and Milvus". Get ready to upgrade your stack!

Follow Superdev Academy on all platforms: