[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"academy-blogs-en-1-1-all-golang-ep159-hybrid-search-sql-vector-qdrant-all--*":3,"academy-blog-translations-ftwq5mc0lm5y4sp":95},{"data":4,"page":81,"perPage":81,"totalItems":81,"totalPages":81},[5],{"alt":6,"collectionId":7,"collectionName":8,"content":9,"cover_image":10,"cover_image_path":11,"created":12,"created_by":13,"expand":14,"id":89,"keywords":90,"locale":61,"published_at":91,"scheduled_at":77,"school_blog":85,"short_description":92,"status":83,"title":93,"updated":94,"updated_by":13,"slug":86,"views":88},"\"Article cover image for 'Golang The Series EP.159: Hybrid Search Blending SQL and Vector Search in Go', featuring a blue AI data network background","sclblg987654321","school_blog_translations","\u003Cp>Hello Gophers! Welcome to EP.159.\u003C\u002Fp>\u003Cp>Following our previous episode on Context Injection, where we successfully built a RAG system capable of smoothly retrieving answers from documents. However, when deploying this to a real production environment, I believe many of you might encounter the same \"pain point.\"\u003C\u002Fp>\u003Cp>The issue is that most enterprise data doesn't just consist of raw, unstructured document files that we convert into vectors. There's also a massive amount of structured data resting in traditional Relational Databases like PostgreSQL or MySQL—whether it's employee departments, access permission levels, system update dates, or document statuses (Active\u002FDraft).\u003C\u002Fp>\u003Cp>Imagine a requirement from the business side: \u003Cem>\"We want employees to be able to search manuals using AI, but restrict it so that it only retrieves the latest version of documents, and the user asking must have the correct permissions (Role = Admin).\"\u003C\u002Fem>\u003C\u002Fp>\u003Cp>If we rely solely on a Vector DB without considering other conditions, there's a high chance that confidential cross-department data could leak. This is exactly where \u003Cstrong>Hybrid Search\u003C\u002Fstrong>, or the technique of blending the SQL world with Vector Search, comes into play in our Go applications!\u003C\u002Fp>\u003Ch2>Hybrid Search Design Concepts in Go Projects\u003C\u002Fh2>\u003Cp>When we need to filter data across these two architectures, we generally have two main approaches:\u003C\u002Fp>\u003Col>\u003Cli>\u003Cp>\u003Cstrong>Two-Stage Query (The Classic Approach: SQL First, then Vector)\u003C\u002Fstrong>\u003C\u002Fp>\u003Cp>This approach involves using the Go Backend to query the SQL Database first to retrieve a list of \u003Cem>Document IDs\u003C\u002Fem> that meet the conditions (e.g., correct permissions, active status). Then, we send this list of IDs as a filter to the Vector DB for the actual search. This method clearly separates responsibilities, but if the ID list is enormous, it can introduce significant latency.\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>Integrated Filtering or Metadata Pre-filtering (Recommended Approach: One-Shot)\u003C\u002Fstrong>\u003C\u002Fp>\u003Cp>This method shifts the necessary conditional data (Metadata) to be saved into the Payload section of the Vector Database (like Qdrant) right from the data extraction phase. From there, we use Qdrant's Payload Indexing feature to perform a vector search alongside the conditions in a single query.\u003C\u002Fp>\u003Cp>\u003Cem>And the Qdrant Go SDK provides highly efficient functions for this, drastically reducing network latency!\u003C\u002Fem>\u003C\u002Fp>\u003C\u002Fli>\u003C\u002Fol>\u003Ch2>Go Example for Hybrid Search (Vector Search + Metadata Filter)\u003C\u002Fh2>\u003Cp>Instead of writing all our code inside \u003Ccode>main()\u003C\u002Fcode>, let's divide the structure into two main parts to keep the code clean and easy to extend:\u003C\u002Fp>\u003Col>\u003Cli>\u003Cp>\u003Cstrong>Text Conversion (Embedding)\u003C\u002Fstrong>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>Data Search and Filtering (Hybrid Search)\u003C\u002Fstrong>\u003C\u002Fp>\u003C\u002Fli>\u003C\u002Fol>\u003Ch3>1. Function to Convert Queries to Vectors (Embedding)\u003C\u002Fh3>\u003Cp>We will extract the OpenAI API interaction into a separate function that takes text input and returns vector coordinates (\u003Ccode>[]float32\u003C\u002Fcode>).\u003C\u002Fp>\u003Cp>Go\u003C\u002Fp>\u003Cpre>\u003Ccode>\u002F\u002F generateEmbedding takes a query and converts it into vector coordinates via OpenAI\nfunc generateEmbedding(ctx context.Context, client *openai.Client, text string) ([]float32, error) {\n\tembReq := openai.EmbeddingRequest{\n\t\tInput: []string{text},\n\t\tModel: openai.SmallEmbedding3Small, \u002F\u002F Using standard 1,536 dimensions\n\t}\n\tresp, err := client.CreateEmbeddings(ctx, embReq)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"embedding failed: %w\", err)\n\t}\n\treturn resp.Data[0].Embedding, nil\n}\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Ch3>2. Function to Execute Hybrid Search Query on Qdrant\u003C\u002Fh3>\u003Cp>This function takes the vector coordinates from step 1 along with the \"conditions\" (Metadata) we want to filter by, such as department and document status, to construct the search query.\u003C\u002Fp>\u003Cp>Go\u003C\u002Fp>\u003Cpre>\u003Ccode>\u002F\u002F searchDocuments performs a semantic search with Metadata filtering (Department, Status)\nfunc searchDocuments(ctx context.Context, client *qdrant.Client, queryVector []float32, dept, status string) ([]*qdrant.ScoredPoint, error) {\n\tsearchLimit := uint64(3)\n\t\n\t\u002F\u002F Create filter specifications by matching keywords in the Payload\n\tsearchFilters := &qdrant.Filter{\n\t\tMust: []*qdrant.Condition{\n\t\t\tqdrant.NewMatchKeyword(\"department\", dept),\n\t\t\tqdrant.NewMatchKeyword(\"status\", status),\n\t\t},\n\t}\n\n\t\u002F\u002F Execute the query for both semantic coordinates and Metadata filtering in one go\n\treturn client.Query(ctx, &qdrant.QueryPoints{\n\t\tCollectionName: \"ai_knowledge_base\",\n\t\tQuery:          qdrant.NewQuery(queryVector...),\n\t\tFilter:         searchFilters,\n\t\tLimit:          &searchLimit,\n\t})\n}\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Ch3>3. Assembling in the \u003Ccode>main()\u003C\u002Fcode> Function\u003C\u002Fh3>\u003Cp>By separating the functions, our \u003Ccode>main()\u003C\u002Fcode> code becomes as readable as a book. It clearly simulates the workflow.\u003C\u002Fp>\u003Cp>Go\u003C\u002Fp>\u003Cpre>\u003Ccode>package main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com\u002Fqdrant\u002Fgo-client\u002Fqdrant\"\n\t\"github.com\u002Fsashabaranov\u002Fgo-openai\"\n)\n\nfunc main() {\n\tctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)\n\tdefer cancel()\n\n\t\u002F\u002F --- [Setup Clients] ---\n\tapiKey := os.Getenv(\"OPENAI_API_KEY\")\n\tif apiKey == \"\" {\n\t\tlog.Fatal(\"Please set your OPENAI_API_KEY environment variable\")\n\t}\n\topenaiClient := openai.NewClient(apiKey)\n\n\tqdrantClient, err := qdrant.NewClient(&qdrant.Config{\n\t\tHost: \"localhost\",\n\t\tPort: 6334,\n\t})\n\tif err != nil {\n\t\tlog.Fatalf(\"Qdrant connection failed: %v\", err)\n\t}\n\tdefer qdrantClient.Close()\n\n\t\u002F\u002F --- [Define Query & Filters] ---\n\tuserQuery := \"What are the medical reimbursement regulations?\"\n\t\n\t\u002F\u002F Simulated values retrieved from the core system's SQL Database\n\ttargetDepartment := \"Human Resources\"\n\tdocumentStatus := \"active\"\n\n\t\u002F\u002F --- [Step 1: Embedding] ---\n\tqueryVector, err := generateEmbedding(ctx, openaiClient, userQuery)\n\tif err != nil {\n\t\tlog.Fatalf(\"Error generating embedding: %v\", err)\n\t}\n\n\t\u002F\u002F --- [Step 2: Hybrid Search] ---\n\tsearchResp, err := searchDocuments(ctx, qdrantClient, queryVector, targetDepartment, documentStatus)\n\tif err != nil {\n\t\tlog.Fatalf(\"Hybrid Search query failed: %v\", err)\n\t}\n\n\t\u002F\u002F --- [Step 3: Display Results] ---\n\tfmt.Printf(\"🔍 Hybrid Search Results for: '%s'\\n\", userQuery)\n\tfmt.Printf(\"📋 [System Conditions] Department: %s | Status: %s\\n\\n\", targetDepartment, documentStatus)\n\t\n\tfor i, point := range searchResp {\n\t\tpayloadMap := point.Payload\n\t\t\n\t\t\u002F\u002F Prevent Panics by checking keys before retrieving values\n\t\tvar content, dept, status string\n\t\tif val, ok := payloadMap[\"content\"]; ok { content = val.GetStringValue() }\n\t\tif val, ok := payloadMap[\"department\"]; ok { dept = val.GetStringValue() }\n\t\tif val, ok := payloadMap[\"status\"]; ok { status = val.GetStringValue() }\n\t\t\n\t\tfmt.Printf(\"[%d] Score: %.4f | Department: %s | Status: %s\\n\", i+1, point.Score, dept, status)\n\t\tfmt.Printf(\"   Reference Content: %s\\n\\n\", content)\n\t}\n}\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Ch2>Why is Go Perfect for Hybrid Search Architecture?\u003C\u002Fh2>\u003Cp>Having the backend talk to two databases (both SQL and Vector) inevitably introduces latency concerns if the workflow isn't managed well. Fortunately, we're writing in Go! Go has great features that make handling this a breeze:\u003C\u002Fp>\u003Cul>\u003Cli>\u003Cp>\u003Cstrong>Goroutines for Concurrent Querying:\u003C\u002Fstrong> If your requirements force you into the Two-Stage Query approach (fetching data from separate silos), you can simply run them in parallel using \u003Cem>Goroutines\u003C\u002Fem> and synchronize them with \u003Ccode>sync.WaitGroup\u003C\u002Fcode>. Once you have results from both sides, perform data intersection in Go. This significantly cuts down processing time compared to waiting for sequential queries.\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>Type-Safety with Go Structs:\u003C\u002Fstrong> When we receive payload data back from the Vector DB (often as JSON or Maps) alongside SQL records, we can unmarshal them into strictly typed \u003Cem>Go Structs\u003C\u002Fem> right in memory. This allows for confident data transformation, completely eliminating the minor runtime errors typically found in dynamically typed languages.\u003C\u002Fp>\u003C\u002Fli>\u003C\u002Ful>\u003Ch2>🎯 Challenge (Daily Mission)\u003C\u002Fh2>\u003Cp>Try applying the filter-attached query structure from this article to your own project. The challenge is: \u003Cem>\"Retrieve only documents that are no older than a specified duration\"\u003C\u002Fem> (using Range Filters).\u003C\u002Fp>\u003Cp>\u003Cem>Food for thought:\u003C\u002Fem> Try modifying the \u003Ccode>searchFilters\u003C\u002Fcode> variable to support numeric or Unix Timestamp comparisons (e.g., finding documents where the \u003Ccode>created_at\u003C\u002Fcode> field is greater than 30 days ago). \u003Cem>Hint:\u003C\u002Fem> Check out the Qdrant Go Client documentation to see how \u003Ccode>qdrant.NewRange(...)\u003C\u002Fcode> handles numeric parameter passing, run the query, and observe the results in your terminal!\u003C\u002Fp>\u003Ch2>💡 FAQ (Frequently Asked Questions)\u003C\u002Fh2>\u003Ch3>Is Qdrant strictly required for Hybrid Search?\u003C\u002Fh3>\u003Cp>Not necessarily. Other leading Vector Databases like Milvus, Pinecone, Weaviate, or even \u003Cem>pgvector\u003C\u002Fem> in PostgreSQL also support Metadata Filtering. However, in this series, we chose Qdrant because of its easy-to-use, high-performance Go SDK and excellent support for Payload Indexing.\u003C\u002Fp>\u003Ch3>If Metadata (like access permissions) changes very frequently, which approach is better?\u003C\u002Fh3>\u003Cp>If access permissions change on a per-second basis (Highly Dynamic), using the \u003Cem>Two-Stage Query\u003C\u002Fem> approach (checking SQL for permissions first, then passing IDs to Vector) might be better for Data Consistency. But if changes aren't that frequent, the \u003Cem>Metadata Pre-filtering\u003C\u002Fem> approach discussed here offers much better speed performance (you just need to build a mechanism to keep the Qdrant Payload synced with SQL).\u003C\u002Fp>\u003Ch3>Does Payload Indexing consume more system resources?\u003C\u002Fh3>\u003Cp>Creating indexes will consume slightly more RAM and Storage. But the tradeoff is a massive leap in query speed. For production environments, sacrificing a little storage for lower latency is a highly worthwhile investment.\u003C\u002Fp>\u003Cdiv data-type=\"horizontalRule\">\u003Chr>\u003C\u002Fdiv>\u003Ch2>Summary\u003C\u002Fh2>\u003Cp>In this article, we've explored \u003Cem>how to integrate Relational Databases (SQL) and Vector Databases via Hybrid Search\u003C\u002Fem>. This is crucial for building enterprise AI search systems that are secure, permission-aware, and retrieve only ready-to-use data using the Metadata Pre-filtering technique efficiently completed in a single query via the Go SDK.\u003C\u002Fp>\u003Cp>\u003Cstrong>In the next episode (EP.160): \u003C\u002Fstrong>Congratulations! The theoretical foundation and advanced components for our AI Integration development are now 100% ready. Next time, we'll conclude the season with a grand finale: \u003Cstrong>\"Workshop 2: Internal Tool Document Q&A System.\"\u003C\u002Fstrong> We will combine the Gin Web Framework, PDF Extraction, the RAG Pipeline, and Hybrid Search to build a real enterprise application. Don't miss it, Gophers!\u003C\u002Fp>\u003Cp>\u003Cstrong>Follow Superdev Academy on all platforms:\u003C\u002Fstrong>\u003C\u002Fp>\u003Cul>\u003Cli>\u003Cp>\u003Cstrong>🔵 Facebook: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener\" class=\"ng-star-inserted\" href=\"https:\u002F\u002Fwww.facebook.com\u002Fsuperdev.academy.th\">\u003Cstrong>Superdev Academy Thailand\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>🎬 YouTube: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener\" class=\"ng-star-inserted\" href=\"https:\u002F\u002Fwww.youtube.com\u002F@SuperdevAcademy\">\u003Cstrong>Superdev Academy Channel\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>📸 Instagram: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener\" class=\"ng-star-inserted\" href=\"https:\u002F\u002Fwww.instagram.com\u002Fsuperdevacademy\u002F\">\u003Cstrong>@superdevacademy\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>🎬 TikTok: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener\" class=\"ng-star-inserted\" href=\"https:\u002F\u002Fwww.tiktok.com\u002F@superdevacademy?lang=th-TH\">\u003Cstrong>@superdevacademy\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>🌐 Website: \u003C\u002Fstrong>\u003Ca rel=\"noopener noreferrer\" href=\"https:\u002F\u002Fsuperdevacademy.com\">\u003Cstrong>superdevacademy.com\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003C\u002Ful>\u003Cp>\u003C\u002Fp>","38fcahddlju7_m50jnyem36.png","https:\u002F\u002Ftwsme-r2.tumwebsme.com\u002Fsclblg987654321\u002Fp3itkx95ngmlkit\u002F38fcahddlju7_m50jnyem36.png","2026-07-13 01:23:15.697Z","76qprkevbgfdps8",{"keywords":15,"locale":55,"school_blog":65},[16,23,28,32,36,41,45,50],{"collectionId":17,"collectionName":18,"created":19,"created_by":13,"id":20,"name":21,"updated":22,"updated_by":13},"sclkey987654321","school_keywords","2026-03-04 08:20:14.253Z","ah6lvy4x8qe08l5","Golang","2026-06-07 06:45:08.193Z",{"collectionId":17,"collectionName":18,"created":24,"created_by":13,"id":25,"name":26,"updated":27,"updated_by":13},"2026-03-04 08:20:11.547Z","ey3puyme01a9bsw","Go","2026-06-07 06:45:07.798Z",{"collectionId":17,"collectionName":18,"created":29,"created_by":13,"id":30,"name":31,"updated":29,"updated_by":13},"2026-07-13 01:22:24.796Z","jakeeeq3nrsn8q6","Hybrid Search",{"collectionId":17,"collectionName":18,"created":33,"created_by":13,"id":34,"name":35,"updated":33,"updated_by":13},"2026-07-13 01:22:28.308Z","p1xcuxg9elg8xnt","Vector Search",{"collectionId":17,"collectionName":18,"created":37,"created_by":13,"id":38,"name":39,"updated":40,"updated_by":13},"2026-03-04 08:49:14.997Z","o07u79nqhhz3zbn","SQL","2026-06-07 06:48:09.540Z",{"collectionId":17,"collectionName":18,"created":42,"created_by":13,"id":43,"name":44,"updated":42,"updated_by":13},"2026-06-30 03:40:49.549Z","lz4vxehenzo8oum","Qdrant",{"collectionId":17,"collectionName":18,"created":46,"created_by":13,"id":47,"name":48,"updated":49,"updated_by":13},"2026-03-04 08:31:29.142Z","hrqdq7kjl5lzjmi","AI","2026-06-07 06:45:27.147Z",{"collectionId":17,"collectionName":18,"created":51,"created_by":13,"id":52,"name":53,"updated":54,"updated_by":13},"2026-03-04 08:32:51.346Z","tmzmy6jyz1n35rr","Go Programming","2026-06-07 06:45:51.178Z",{"code":56,"collectionId":57,"collectionName":58,"created":59,"flag":60,"id":61,"is_default":62,"label":63,"updated":64},"en","pbc_1989393366","locales","2026-01-22 11:00:02.726Z","twemoji:flag-united-states","qv9c1llfov2d88z",false,"English","2026-04-10 15:42:46.825Z",{"category":66,"collectionId":67,"collectionName":68,"created":69,"expand":70,"id":85,"slug":86,"updated":87,"views":88},"wqxt7ag2gn7xcmk","pbc_2105096300","school_blogs","2026-07-13 01:22:48.450Z",{"category":71},{"blogIds":72,"collectionId":73,"collectionName":74,"created":75,"created_by":13,"id":66,"image":76,"image_alt":77,"image_path":78,"label":79,"name":80,"priority":81,"publish_at":82,"scheduled_at":77,"status":83,"updated":84,"updated_by":13},[],"sclcatblg987654321","school_category_blogs","2026-03-04 08:33:53.210Z","59ty92ns80w_15oc1implw.png","","https:\u002F\u002Ftwsme-r2.tumwebsme.com\u002Fsclcatblg987654321\u002Fwqxt7ag2gn7xcmk\u002F59ty92ns80w_15oc1implw.png",{"en":80,"th":80},"Golang The Series",1,"2026-03-16 04:39:38.440Z","published","2026-06-07 06:45:03.856Z","ftwq5mc0lm5y4sp","golang-ep159-hybrid-search-sql-vector-qdrant","2026-07-13 05:58:38.243Z",135,"p3itkx95ngmlkit",[20,25,30,34,38,43,47,52],"2026-07-13 03:49:12.424Z","Deep dive into Hybrid Search in Go. Learn how to combine Relational SQL with Vector Databases (Qdrant) to build secure, permission-aware AI search systems using Metadata Pre-filtering.","Golang The Series EP.159: Hybrid Search - Combining SQL and Vector Search in Go","2026-07-13 03:49:12.427Z",{"th":86,"en":86}]