[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"academy-blogs-en-1-1-all-golang-the-series-ep156-semantic-search-vector-qdrant-all--*":3,"academy-blog-translations-or1jhl03h1oifdj":85},{"data":4,"page":71,"perPage":71,"totalItems":71,"totalPages":71},[5],{"alt":6,"collectionId":7,"collectionName":8,"content":9,"cover_image":10,"cover_image_path":11,"created":12,"created_by":13,"expand":14,"id":79,"keywords":80,"locale":51,"published_at":81,"scheduled_at":67,"school_blog":75,"short_description":82,"status":73,"title":83,"updated":84,"updated_by":13,"slug":76,"views":78},"Semantic Search Pipeline Diagram using Go, OpenAI Embedding, and Qdrant DB","sclblg987654321","school_blog_translations","\u003Cp>Welcome to \u003Cstrong>EP.156\u003C\u002Fstrong>! In our last episode, we discussed strategic chunking to keep our text segments contextually rich. Today, all the pieces we’ve been gathering since the start of this season finally come together to build \u003Cstrong>Semantic Search\u003C\u002Fstrong>—an intelligent search system that understands the actual intent and meaning behind human language, rather than just matching raw keywords.\u003C\u002Fp>\u003Cp>Imagine a user searches for \u003Cem>\"how to fix internet connection issues\"\u003C\u002Fem>. A traditional \u003Cstrong>Keyword Search\u003C\u002Fstrong> would completely miss a troubleshooting guide titled \u003Cem>\"Router Setup and Wi-Fi Dropouts Guide\"\u003C\u002Fem> simply because the exact words \"internet\" or \"connection\" never appear in it. \u003Cstrong>Semantic Search\u003C\u002Fstrong>, however, immediately recognizes that both phrases are talking about the exact same problem—all thanks to vector embeddings.\u003C\u002Fp>\u003Cp>Let’s dive into Go and build this intelligent search pipeline.\u003C\u002Fp>\u003Ch2>The Semantic Search Pipeline Under the Hood\u003C\u002Fh2>\u003Cp>When a query hits our backend, the system processes it through 3 core steps:\u003C\u002Fp>\u003Col>\u003Cli>\u003Cp>\u003Cstrong>Query Embedding:\u003C\u002Fstrong> The raw user query (e.g., \u003Cem>\"how do I reset my password\"\u003C\u002Fem>) is sent to the Embedding API, converting it into a 1,536-dimensional vector (\u003Ccode>[]float32\u003C\u002Fcode>).\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>Vector Similarity Search:\u003C\u002Fstrong> That query vector is sent directly to our Qdrant Vector Database using a \u003Cem>Nearest Neighbor\u003C\u002Fem> search to pinpoint the text chunks closest to the query's conceptual meaning.\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>Payload Extraction:\u003C\u002Fstrong> We extract the raw text and metadata stored within Qdrant's payload to serve it back to the user or pass it to an LLM for response generation in the next step.\u003C\u002Fp>\u003C\u002Fli>\u003C\u002Fol>\u003Ch2>Building Semantic Search in Go with Qdrant\u003C\u002Fh2>\u003Cp>Here is a practical Go example that takes a user query, converts it into an embedding, and queries the Qdrant collection we set up back in EP.154.\u003C\u002Fp>\u003Cp>Go\u003C\u002Fp>\u003Cpre>\u003Ccode>package main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\n\t\"github.com\u002Fqdrant\u002Fgo-client\u002Fqdrant\"\n\t\"github.com\u002Fsashabaranov\u002Fgo-openai\"\n)\n\nfunc main() {\n\tctx := context.Background()\n\n\t\u002F\u002F 1. Initialize OpenAI and Qdrant clients (Use env variables for production!)\n\topenaiClient := openai.NewClient(\"YOUR_OPENAI_API_KEY\")\n\tqdrantClient, err := qdrant.NewClient(&qdrant.Config{\n\t\tHost: \"localhost\",\n\t\tPort: 6334, \u002F\u002F Utilizing gRPC for maximum speed\n\t})\n\tif err != nil {\n\t\tlog.Fatalf(\"Failed to connect to Qdrant: %v\", err)\n\t}\n\tdefer qdrantClient.Close()\n\n\t\u002F\u002F Simulate a user query\n\tuserQuery := \"อยากเปลี่ยนพาสเวิร์ดระบบต้องทำตรงไหน\"\n\n\t\u002F\u002F 2. Transform the user query into a vector (Query Embedding)\n\tembReq := openai.EmbeddingRequest{\n\t\tInput: []string{userQuery},\n\t\tModel: openai.SmallEmbedding3Small, \u002F\u002F Popular 1,536-dimension model\n\t}\n\tembResp, err := openaiClient.CreateEmbeddings(ctx, embReq)\n\tif err != nil {\n\t\tlog.Fatalf(\"Failed to create query embedding: %v\", err)\n\t}\n\tqueryVector := embResp.Data[0].Embedding\n\n\t\u002F\u002F 3. Execute Semantic Search in Qdrant\n\tsearchLimit := uint64(3) \u002F\u002F Fetch top 3 closest matches\n\tsearchResp, err := qdrantClient.Query(ctx, &qdrant.QueryPoints{\n\t\tCollectionName: \"ai_knowledge_base\",\n\t\tQuery:          qdrant.NewQuery(queryVector...), \u002F\u002F Unpack the float32 array\n\t\tLimit:          &searchLimit,\n\t})\n\tif err != nil {\n\t\tlog.Fatalf(\"Qdrant search query failed: %v\", err)\n\t}\n\n\t\u002F\u002F 4. Extract and display the payload data\n\tfmt.Printf(\"🔍 Search results for: '%s'\\n\\n\", userQuery)\n\tfor i, point := range searchResp {\n\t\tpayloadMap := point.Payload\n\t\t\n\t\t\u002F\u002F Safely extract the original text from the \"content\" key\n\t\tcontentValue, exists := payloadMap[\"content\"]\n\t\tif !exists {\n\t\t\tcontinue\n\t\t}\n\t\t\n\t\tcontent := contentValue.GetStringValue()\n\t\tscore := point.Score \u002F\u002F Cosine Similarity score (Closer to 1.0 means higher relevance)\n\n\t\tfmt.Printf(\"[%d] Similarity Score: %.4f\\n\", i+1, score)\n\t\tfmt.Printf(\"   Document Content: %s\\n\\n\", content)\n\t}\n}\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Ch2>Why Semantic Search with Go is Incredibly Powerful\u003C\u002Fh2>\u003Cul>\u003Cli>\u003Cp>\u003Cstrong>Low-Latency Serialization:\u003C\u002Fstrong> Qdrant's Go client communicates natively over \u003Cstrong>gRPC\u003C\u002Fstrong>. This serializes massive vector arrays into a compact binary format right from the source, eliminating the overhead of parsing large arrays of floating-point numbers into JSON string text. Your system gets responses back in single-digit milliseconds.\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>Ready for RAG:\u003C\u002Fstrong> The raw text chunks we pulled based on their similarity scores are the exact \"reference material\" we need. In the upcoming RAG phase, we will bundle these chunks with the original user query and feed them to an LLM to generate precise, grounded answers.\u003C\u002Fp>\u003C\u002Fli>\u003C\u002Ful>\u003Ch2>🎯 Daily Mission\u003C\u002Fh2>\u003Cp>Try integrating this Semantic Search logic into the \u003Cstrong>Gin Web Server\u003C\u002Fstrong> we built together back in Workshop EP.150. Instead of having an endpoint that serves static or randomized text, turn it into an intelligent document retrieval API.\u003C\u002Fp>\u003Cp>\u003Cstrong>💡 Food for Thought:\u003C\u002Fstrong> Test out queries that match your documents word-for-word versus queries using synonyms, and watch how the \u003Cstrong>Score\u003C\u002Fstrong> changes. If you were deploying this to a production environment, what minimum similarity score (Threshold) would you enforce to filter out irrelevant noise? Go ahead and experiment!\u003C\u002Fp>\u003Ch2>💬 FAQ\u003C\u002Fh2>\u003Ch3>What similarity score should I consider \"relevant enough\" for real-world use?\u003C\u002Fh3>\u003Cp>For OpenAI's \u003Ccode>text-embedding-3-small\u003C\u002Fcode> using Cosine Similarity, a safe production threshold for Thai context typically floats around \u003Cstrong>0.45 - 0.60+\u003C\u002Fstrong> (highly dependent on your chunk sizes). We recommend setting an initial threshold at \u003Ccode>0.50\u003C\u002Fcode>. Anything below that can generally be discarded as irrelevant.\u003C\u002Fp>\u003Ch3>Can we combine traditional keyword search and semantic search in a live system?\u003C\u002Fh3>\u003Cp>\u003Cstrong>Absolutely.\u003C\u002Fstrong> In fact, this is the enterprise gold standard known as \u003Cstrong>Hybrid Search\u003C\u002Fstrong>. You run a classic keyword-matching algorithm (like BM25) alongside your Vector Search, then merge and re-rank the results using RRF (Reciprocal Rank Fusion). Qdrant actually supports this natively out of the box!\u003C\u002Fp>\u003Cdiv data-type=\"horizontalRule\">\u003Chr>\u003C\u002Fdiv>\u003Ch2>📝 Wrap-up\u003C\u002Fh2>\u003Cp>Stepping into semantic search brings us closer to building context-aware AI applications that genuinely understand what users are looking for, moving far beyond fragile keyword matching.\u003C\u002Fp>\u003Cp>\u003Cstrong>Coming up next in EP.157:\u003C\u002Fstrong> Our search system is ready and our vector base is smart, but in reality, nobody manually types sentences into a payload map. Company documentation lives in dense, multi-page PDFs and Word files. In the next episode, we will build an automated \u003Cstrong>\"Document Ingestion Pipeline: Ingesting Data from PDFs\u002FWord Documents Directly to Your Vector DB\"\u003C\u002Fstrong>. Stay tuned!\u003C\u002Fp>\u003Cp>\u003Cstrong>Follow Superdev Academy on all platforms:\u003C\u002Fstrong>\u003C\u002Fp>\u003Cul>\u003Cli>\u003Cp>\u003Cstrong>🔵 Facebook: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener\" class=\"ng-star-inserted\" href=\"https:\u002F\u002Fwww.facebook.com\u002Fsuperdev.academy.th\">\u003Cstrong>Superdev Academy Thailand\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>🎬 YouTube: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener\" class=\"ng-star-inserted\" href=\"https:\u002F\u002Fwww.youtube.com\u002F@SuperdevAcademy\">\u003Cstrong>Superdev Academy Channel\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>📸 Instagram: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener\" class=\"ng-star-inserted\" href=\"https:\u002F\u002Fwww.instagram.com\u002Fsuperdevacademy\u002F\">\u003Cstrong>@superdevacademy\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>🎬 TikTok: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener\" class=\"ng-star-inserted\" href=\"https:\u002F\u002Fwww.tiktok.com\u002F@superdevacademy?lang=th-TH\">\u003Cstrong>@superdevacademy\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>🌐 Website: \u003C\u002Fstrong>\u003Ca rel=\"noopener noreferrer\" href=\"https:\u002F\u002Fsuperdevacademy.com\">\u003Cstrong>superdevacademy.com\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003C\u002Ful>\u003Cp>\u003C\u002Fp>","325b52o5gi65_72aqkc0vnm.png","https:\u002F\u002Ftwsme-r2.tumwebsme.com\u002Fsclblg987654321\u002Fnus58zdklg1b12u\u002F325b52o5gi65_72aqkc0vnm.png","2026-06-30 03:51:18.701Z","76qprkevbgfdps8",{"keywords":15,"locale":45,"school_blog":55},[16,23,27,32,36,41],{"collectionId":17,"collectionName":18,"created":19,"created_by":13,"id":20,"name":21,"updated":22,"updated_by":13},"sclkey987654321","school_keywords","2026-03-04 08:20:14.253Z","ah6lvy4x8qe08l5","Golang","2026-06-07 06:45:08.193Z",{"collectionId":17,"collectionName":18,"created":24,"created_by":13,"id":25,"name":26,"updated":24,"updated_by":13},"2026-06-11 16:14:22.575Z","gluay8aj98wheus","RAG",{"collectionId":17,"collectionName":18,"created":28,"created_by":13,"id":29,"name":30,"updated":31,"updated_by":13},"2026-05-11 04:12:24.718Z","zo53ndb3rj4jxci","Vector Database","2026-06-07 06:49:11.382Z",{"collectionId":17,"collectionName":18,"created":33,"created_by":13,"id":34,"name":35,"updated":33,"updated_by":13},"2026-06-30 03:40:49.549Z","lz4vxehenzo8oum","Qdrant",{"collectionId":17,"collectionName":18,"created":37,"created_by":13,"id":38,"name":39,"updated":40,"updated_by":13},"2026-03-04 08:20:11.547Z","ey3puyme01a9bsw","Go","2026-06-07 06:45:07.798Z",{"collectionId":17,"collectionName":18,"created":42,"created_by":13,"id":43,"name":44,"updated":42,"updated_by":13},"2026-06-30 03:41:08.262Z","i0hgq247989xph0","Semantic Search",{"code":46,"collectionId":47,"collectionName":48,"created":49,"flag":50,"id":51,"is_default":52,"label":53,"updated":54},"en","pbc_1989393366","locales","2026-01-22 11:00:02.726Z","twemoji:flag-united-states","qv9c1llfov2d88z",false,"English","2026-04-10 15:42:46.825Z",{"category":56,"collectionId":57,"collectionName":58,"created":59,"expand":60,"id":75,"slug":76,"updated":77,"views":78},"wqxt7ag2gn7xcmk","pbc_2105096300","school_blogs","2026-06-30 03:39:56.012Z",{"category":61},{"blogIds":62,"collectionId":63,"collectionName":64,"created":65,"created_by":13,"id":56,"image":66,"image_alt":67,"image_path":68,"label":69,"name":70,"priority":71,"publish_at":72,"scheduled_at":67,"status":73,"updated":74,"updated_by":13},[],"sclcatblg987654321","school_category_blogs","2026-03-04 08:33:53.210Z","59ty92ns80w_15oc1implw.png","","https:\u002F\u002Ftwsme-r2.tumwebsme.com\u002Fsclcatblg987654321\u002Fwqxt7ag2gn7xcmk\u002F59ty92ns80w_15oc1implw.png",{"en":70,"th":70},"Golang The Series",1,"2026-03-16 04:39:38.440Z","published","2026-06-07 06:45:03.856Z","or1jhl03h1oifdj","golang-the-series-ep156-semantic-search-vector-qdrant","2026-06-30 06:27:02.106Z",112,"nus58zdklg1b12u",[20,25,29,34,38,43],"2026-06-30 04:01:39.395Z","Stop matching keywords! Learn how to build an intelligent Semantic Search system based on human context using Go, OpenAI, and Qdrant Vector DB via high-speed gRPC.","Golang The Series EP.156: Semantic Search with Qdrant & OpenAI","2026-06-30 04:01:39.396Z",{"th":76,"en":76}]