[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"academy-blogs-en-1-1-all-golang-ai-batch-processing-worker-pool-all--*":3,"academy-blog-translations-2nqska91xchwzgb":96},{"data":4,"page":82,"perPage":82,"totalItems":82,"totalPages":82},[5],{"alt":6,"collectionId":7,"collectionName":8,"content":9,"cover_image":10,"cover_image_path":11,"cover_image_s_url":12,"created":13,"created_by":14,"expand":15,"id":90,"keywords":91,"locale":61,"published_at":92,"scheduled_at":77,"school_blog":86,"short_description":93,"status":84,"title":94,"updated":95,"updated_by":14,"slug":87,"views":89},"Cover image showing AI Batch Processing architecture with Golang Worker Pool","sclblg987654321","school_blog_translations","\u003Cp>Welcome to EP.170! We have finally arrived at the grand finale workshop for the AI API Infrastructure chapter in the Superdev Academy series. After thoroughly learning about Multi-LLM management, Redis Cache, Rate Limiting, Load Balancing, Circuit Breaker, Prometheus Metrics, as well as Error Handling and Concurrency.\u003C\u002Fp>\u003Cp>In the real enterprise world, AI workloads aren't just about replying to single chat messages (Transactional Queries). We often have to deal with massive \u003Cem>\"Batch Workloads\"\u003C\u002Fem>, such as summarizing 10,000 customer feedback reviews, extracting table of contents data from thousands of PDF documents, or running weekly Sentiment Analysis.\u003C\u002Fp>\u003Cp>If we send data one by one using a Synchronous Loop, the system could take days, and if a crash occurs midway, all the processed data might instantly vanish! Today, we will use Go to build a \u003Cem>High-Performance AI Batch Processor\u003C\u002Fem> that can process thousands of records in mere minutes, using a secure, stable, and fully monitored Worker Pool architecture!\u003C\u002Fp>\u003Ch2>AI Batch Processor System Architecture\u003C\u002Fh2>\u003Cp>Our system is divided into 4 main parts, working together via Goroutines and Channels:\u003C\u002Fp>\u003Cp>Plaintext\u003C\u002Fp>\u003Cpre>\u003Ccode>+------------------+      +-------------------+      +-------------------+      +------------------+\n|  Job Dispatcher  | ---> |    Worker Pool    | ---> |   AI Processing   | ---> |  Result Handler  |\n| (Reads data &    |      | (Runs n parallel  |      |   (Calls API +    |      | (Saves results & |\n| feeds Job Channel|      |      workers)     |      |  Circuit Breaker) |      |   Monitoring)    |\n+------------------+      +-------------------+      +-------------------+      +------------------+\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Cul>\u003Cli>\u003Cp>\u003Cstrong>Job Dispatcher:\u003C\u002Fstrong> Distributes jobs into a Buffered Channel (\u003Ccode>JobQueue\u003C\u002Fcode>).\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>Worker Pool:\u003C\u002Fstrong> A specified number of Goroutines (e.g., 50 workers) pull jobs to process concurrently.\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>AI Processing:\u003C\u002Fstrong> Simulates\u002Fcalls the LLM API with built-in timing and Error Tracking.\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>Result Handler & Collector:\u003C\u002Fstrong> A separate Goroutine that receives results from the \u003Ccode>ResultQueue\u003C\u002Fcode> to save into a Database\u002FLog without interrupting the Workers.\u003C\u002Fp>\u003C\u002Fli>\u003C\u002Ful>\u003Ch2>Workshop Project Code: AI Batch Processing System\u003C\u002Fh2>\u003Cp>We will divide the code into 3 main parts for easier understanding. Create a \u003Ccode>main.go\u003C\u002Fcode> file and let's write it step-by-step.\u003C\u002Fp>\u003Ch3>Part 1: Data Structures (Structs & Initialization)\u003C\u002Fh3>\u003Cp>The first part is preparing the structures to store job data, results, and the Worker Pool controller.\u003C\u002Fp>\u003Cp>Go\u003C\u002Fp>\u003Cpre>\u003Ccode>package main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"math\u002Frand\"\n\t\"sync\"\n\t\"sync\u002Fatomic\"\n\t\"time\"\n)\n\n\u002F\u002F Job represents a single piece of work for the AI to process\ntype Job struct {\n\tID      int\n\tPayload string\n}\n\n\u002F\u002F Result represents the outcome of the processing (success or failure)\ntype Result struct {\n\tJobID  int\n\tOutput string\n\tErr    error\n}\n\n\u002F\u002F BatchProcessor controls the Worker Pool system\ntype BatchProcessor struct {\n\tWorkerCount  int\n\tJobQueue     chan Job\n\tResultQueue  chan Result\n\tSuccessCount uint64\n\tFailureCount uint64\n}\n\n\u002F\u002F NewBatchProcessor creates an instance and initializes queue sizes\nfunc NewBatchProcessor(workerCount int, queueSize int) *BatchProcessor {\n\treturn &BatchProcessor{\n\t\tWorkerCount: workerCount,\n\t\tJobQueue:    make(chan Job, queueSize),\n\t\tResultQueue: make(chan Result, queueSize),\n\t}\n}\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Ch3>Part 2: AI Simulation and Running the Worker Pool\u003C\u002Fh3>\u003Cp>The \u003Ccode>callAIModel\u003C\u002Fcode> function simulates calling an AI API (with simulated delays and random errors). The \u003Ccode>StartWorkers\u003C\u002Fcode> function creates the specified number of Goroutines to wait for jobs from the \u003Ccode>JobQueue\u003C\u002Fcode>.\u003C\u002Fp>\u003Cp>Go\u003C\u002Fp>\u003Cpre>\u003Ccode>\u002F\u002F callAIModel simulates sending data to be processed by an AI API\nfunc (bp *BatchProcessor) callAIModel(ctx context.Context, job Job) (string, error) {\n\t\u002F\u002F Simulate AI processing time (200ms - 600ms)\n\tprocessTime := time.Duration(200+rand.Intn(400)) * time.Millisecond\n\n\tselect {\n\tcase <-time.After(processTime):\n\t\t\u002F\u002F Simulate a 5% random chance of failure\n\t\tif rand.Float32() < 0.05 {\n\t\t\treturn \"\", fmt.Errorf(\"AI Provider Error on Job #%d\", job.ID)\n\t\t}\n\t\treturn fmt.Sprintf(\"Summary for Item %d: '%s' [Success]\", job.ID, job.Payload), nil\n\tcase <-ctx.Done():\n\t\treturn \"\", ctx.Err()\n\t}\n}\n\n\u002F\u002F StartWorkers spins up the specified number of Goroutine Workers\nfunc (bp *BatchProcessor) StartWorkers(ctx context.Context, wg *sync.WaitGroup) {\n\tfor i := 1; i <= bp.WorkerCount; i++ {\n\t\twg.Add(1)\n\t\tgo func(workerID int) {\n\t\t\tdefer wg.Done()\n\t\t\tfor job := range bp.JobQueue {\n\t\t\t\t\u002F\u002F Execute AI Processing\n\t\t\t\toutput, err := bp.callAIModel(ctx, job)\n\t\t\t\t\n\t\t\t\t\u002F\u002F Use atomic for thread-safe updates to counters (prevents Race Conditions)\n\t\t\t\tif err != nil {\n\t\t\t\t\tatomic.AddUint64(&bp.FailureCount, 1)\n\t\t\t\t} else {\n\t\t\t\t\tatomic.AddUint64(&bp.SuccessCount, 1)\n\t\t\t\t}\n\t\t\t\t\n\t\t\t\t\u002F\u002F Send the result to the queue for handling\n\t\t\t\tbp.ResultQueue <- Result{JobID: job.ID, Output: output, Err: err}\n\t\t\t}\n\t\t}(i)\n\t}\n}\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Ch3>Part 3: Main Function (Assembly and Execution)\u003C\u002Fh3>\u003Cp>In the final part, we will simulate feeding 1,000 jobs and collecting the results through the Collector to summarize the execution.\u003C\u002Fp>\u003Cp>Go\u003C\u002Fp>\u003Cpre>\u003Ccode>func main() {\n\tctx, cancel := context.WithCancel(context.Background())\n\tdefer cancel()\n\n\ttotalJobs := 1000  \u002F\u002F Total 1,000 jobs\n\tworkerCount := 50  \u002F\u002F Run 50 parallel workers\n\n\tprocessor := NewBatchProcessor(workerCount, totalJobs)\n\n\tvar workerWg sync.WaitGroup\n\tvar resultWg sync.WaitGroup\n\n\tstartTime := time.Now()\n\n\t\u002F\u002F 1. Start Worker Pool\n\tprocessor.StartWorkers(ctx, &workerWg)\n\n\t\u002F\u002F 2. Result Collector Goroutine\n\tresultWg.Add(1)\n\tgo func() {\n\t\tdefer resultWg.Done()\n\t\tfor result := range processor.ResultQueue {\n\t\t\tif result.Err != nil {\n\t\t\t\t\u002F\u002F Real world: Save to Error Log or send to retry queue\n\t\t\t} else {\n\t\t\t\t\u002F\u002F Real world: Save to Database or export as CSV\u002FJSON\n\t\t\t}\n\t\t}\n\t}()\n\n\t\u002F\u002F 3. Dispatcher: Feed 1,000 jobs into the Queue\n\tfmt.Printf(\"🚀 Starting to dispatch %d jobs into the Worker Pool (%d Workers)...\\n\", totalJobs, workerCount)\n\tfor i := 1; i <= totalJobs; i++ {\n\t\tprocessor.JobQueue <- Job{\n\t\t\tID:      i,\n\t\t\tPayload: fmt.Sprintf(\"Customer Feedback #%d\", i),\n\t\t}\n\t}\n\tclose(processor.JobQueue) \u002F\u002F Close Job Channel to notify workers that there are no more jobs\n\n\t\u002F\u002F 4. Wait for all Workers to finish, then close Result Queue\n\tworkerWg.Wait()\n\tclose(processor.ResultQueue)\n\n\t\u002F\u002F 5. Wait for the Collector to finish gathering results\n\tresultWg.Wait()\n\n\ttotalDuration := time.Since(startTime)\n\n\t\u002F\u002F 6. Summarize the processing report\n\tfmt.Println(\"\\n==============================================\")\n\tfmt.Println(\"📊 AI Batch Processing Execution Summary\")\n\tfmt.Println(\"==============================================\")\n\tfmt.Printf(\"⏱️ Total Time Elapsed : %v\\n\", totalDuration)\n\tfmt.Printf(\"✅ Successful Jobs    : %d items\\n\", processor.SuccessCount)\n\tfmt.Printf(\"❌ Failed Jobs        : %d items\\n\", processor.FailureCount)\n\tfmt.Printf(\"⚡ Throughput (RPS)   : %.2f Requests\u002Fsec\\n\", float64(totalJobs)\u002FtotalDuration.Seconds())\n\tfmt.Println(\"==============================================\")\n}\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Ch2>Results and Performance Analysis\u003C\u002Fh2>\u003Cp>When you run the code above with \u003Ccode>go run main.go\u003C\u002Fcode>, you will see the following processing statistics:\u003C\u002Fp>\u003Cp>Plaintext\u003C\u002Fp>\u003Cpre>\u003Ccode>🚀 Starting to dispatch 1000 jobs into the Worker Pool (50 Workers)...\n\n==============================================\n📊 AI Batch Processing Execution Summary\n==============================================\n⏱️ Total Time Elapsed : 8.12s\n✅ Successful Jobs    : 952 items\n❌ Failed Jobs        : 48 items\n⚡ Throughput (RPS)   : 123.15 Requests\u002Fsec\n==============================================\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Ch2>Why is this method so powerful?\u003C\u002Fh2>\u003Cul>\u003Cli>\u003Cp>\u003Cstrong>Reduces time by over 50x:\u003C\u002Fstrong> If processed sequentially, 1,000 items × 400ms would take 400 seconds (~6.6 minutes). But with 50 Go Workers, we finish the processing in just \u003Cstrong>8 seconds\u003C\u002Fstrong>!\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>Memory Constrained:\u003C\u002Fstrong> Using Bounded Channels and Atomic Counters helps control RAM usage from spiking, even when hundreds of thousands of jobs are queued up.\u003C\u002Fp>\u003C\u002Fli>\u003C\u002Ful>\u003Ch2>🎯 Daily Mission\u003C\u002Fh2>\u003Cp>Try running this Workshop code on your machine and tweak the \u003Ccode>workerCount\u003C\u002Fcode> (e.g., from 50 to 10 or 100) to compare the speed results.\u003C\u002Fp>\u003Cp>\u003Cstrong>Thought Experiment Homework:\u003C\u002Fstrong> If during Batch Processing, a Rate Limit from the AI provider cuts you off (causing a \u003Cem>429 Too Many Requests\u003C\u002Fem> error), how would you wrap the \u003Ca rel=\"noopener noreferrer\" href=\"http:\u002F\u002Fgolang.org\u002Fx\u002Ftime\u002Frate\">\u003Ccode>golang.org\u002Fx\u002Ftime\u002Frate\u003C\u002Fcode>\u003C\u002Fa> package (which we learned in EP.164) around the workers' job-fetching phase to prevent exceeding the quota? Try tweaking the code!\u003C\u002Fp>\u003Ch2>🙋‍♂️ Frequently Asked Questions (FAQ)\u003C\u002Fh2>\u003Ch3>Why use a Channel to distribute jobs instead of creating a slice and using a \u003Ccode>go func()\u003C\u002Fcode> loop directly?\u003C\u002Fh3>\u003Cp>Spawning an unlimited (Unbounded) number of Goroutines, like calling \u003Ccode>go func()\u003C\u002Fcode> 1,000 times concurrently, can lead to Out of Memory issues or Connection Exhaustion. Using a Worker Pool + Channel allows us to control Concurrency at an optimal level (like 50), making the system much more stable and reliable.\u003C\u002Fp>\u003Ch3>If I want the system to automatically \"Retry\" failed jobs, what should I do?\u003C\u002Fh3>\u003Cp>You can enhance the Result Collector. If it checks and finds \u003Ccode>result.Err != nil\u003C\u002Fcode>, instead of logging the error immediately, it can send \u003Ccode>result.JobID\u003C\u002Fcode> back into the \u003Ccode>JobQueue\u003C\u002Fcode> (or create a separate \u003Ccode>RetryQueue\u003C\u002Fcode>). But be careful to limit the maximum number of retries to prevent Infinite Loops.\u003C\u002Fp>\u003Ch3>What is the optimal number for \u003Ccode>workerCount\u003C\u002Fcode>?\u003C\u002Fh3>\u003Cp>For I\u002FO Bound tasks like calling APIs or Database queries (like in this Workshop), the worker count can be set high (e.g., 50, 100, or 200), depending on the API provider's Rate Limit. But if it's a CPU Bound task (heavy computations on your machine), it should be set close to the number of CPU cores on your server.\u003C\u002Fp>\u003Cdiv data-type=\"horizontalRule\">\u003Chr>\u003C\u002Fdiv>\u003Ch2>Conclusion\u003C\u002Fh2>\u003Cp>In this article, we learned how to design and build an Enterprise-grade \u003Cem>AI Batch Processing\u003C\u002Fem> system using the Worker Pool architecture in Go, allowing us to handle massive workloads efficiently. We saw the power of Concurrency via Goroutines, safe data passing via Channels, and Asynchronous result handling. These are the core reasons why Go has become a highly popular language for backend systems.\u003C\u002Fp>\u003Cp>\u003Cstrong>Coming up next (EP.171):\u003C\u002Fstrong> Congratulations! You have successfully completed the entire AI Backend Infrastructure curriculum. In the next episode, we will step into a \u003Cem>\"New Series \u002F Advanced Topic\"\u003C\u002Fem>: the world of AI Autonomous Agents. In \u003Cem>\"EP.171: Intro to AI Agents - When AI can decide to use tools on its own\"\u003C\u002Fem>, we will explore how an LLM transforms from merely answering questions into a system that thinks, analyzes, runs code, or calls APIs to solve problems for us. Don't miss it, Gophers!\u003C\u002Fp>\u003Cp>\u003Cstrong>Follow Superdev Academy on all platforms:\u003C\u002Fstrong>\u003C\u002Fp>\u003Cul>\u003Cli>\u003Cp>\u003Cstrong>🔵 Facebook: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener\" class=\"ng-star-inserted\" href=\"https:\u002F\u002Fwww.facebook.com\u002Fsuperdev.academy.th\">\u003Cstrong>Superdev Academy Thailand\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>🎬 YouTube: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener\" class=\"ng-star-inserted\" href=\"https:\u002F\u002Fwww.youtube.com\u002F@SuperdevAcademy\">\u003Cstrong>Superdev Academy Channel\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>📸 Instagram: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener\" class=\"ng-star-inserted\" href=\"https:\u002F\u002Fwww.instagram.com\u002Fsuperdevacademy\u002F\">\u003Cstrong>@superdevacademy\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>🎬 TikTok: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener\" class=\"ng-star-inserted\" href=\"https:\u002F\u002Fwww.tiktok.com\u002F@superdevacademy?lang=th-TH\">\u003Cstrong>@superdevacademy\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp>\u003Cstrong>🌐 Website: \u003C\u002Fstrong>\u003Ca rel=\"noopener noreferrer\" href=\"https:\u002F\u002Fsuperdevacademy.com\">\u003Cstrong>superdevacademy.com\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003C\u002Ful>\u003Cp>\u003C\u002Fp>","604qt6u7xdyy_562dxnaazk_4t6ykzpem9.webp","https:\u002F\u002Ftwsme-r2.tumwebsme.com\u002Fsclblg987654321\u002F9diiaifdwadw4js\u002F604qt6u7xdyy_562dxnaazk_4t6ykzpem9.webp","https:\u002F\u002Ftwsme-r2.tumwebsme.com\u002Fsclblg987654321\u002F9diiaifdwadw4js\u002Fs\u002F604qt6u7xdyy_562dxnaazk_4t6ykzpem9.webp","2026-08-18 04:03:39.711Z","76qprkevbgfdps8",{"keywords":16,"locale":55,"school_blog":65},[17,24,28,32,36,41,46,50],{"collectionId":18,"collectionName":19,"created":20,"created_by":14,"id":21,"name":22,"updated":23,"updated_by":14},"sclkey987654321","school_keywords","2026-03-04 08:20:14.253Z","ah6lvy4x8qe08l5","Golang","2026-06-07 06:45:08.193Z",{"collectionId":18,"collectionName":19,"created":25,"created_by":14,"id":26,"name":27,"updated":25,"updated_by":14},"2026-08-18 04:00:01.660Z","gqyw7xfjsynf4ou","AI Batch Processing",{"collectionId":18,"collectionName":19,"created":29,"created_by":14,"id":30,"name":31,"updated":29,"updated_by":14},"2026-08-18 04:00:22.775Z","l3zpz0114sq4tk1","Go Worker Pool",{"collectionId":18,"collectionName":19,"created":33,"created_by":14,"id":34,"name":35,"updated":33,"updated_by":14},"2026-08-18 04:03:21.418Z","tt7q2p4y9dbfduk","Concurrency in Go",{"collectionId":18,"collectionName":19,"created":37,"created_by":14,"id":38,"name":39,"updated":40,"updated_by":14},"2026-03-04 08:33:58.044Z","nb6p1r8sfqlsxf8","Goroutines","2026-06-07 06:45:54.913Z",{"collectionId":18,"collectionName":19,"created":42,"created_by":14,"id":43,"name":44,"updated":45,"updated_by":14},"2026-04-03 10:57:34.421Z","azixuoag5jisout","Backend Development","2026-06-07 06:49:02.435Z",{"collectionId":18,"collectionName":19,"created":47,"created_by":14,"id":48,"name":49,"updated":47,"updated_by":14},"2026-08-18 04:03:32.336Z","itnnbb4oxw6wysh","AI API integration",{"collectionId":18,"collectionName":19,"created":51,"created_by":14,"id":52,"name":53,"updated":54,"updated_by":14},"2026-05-19 09:09:15.823Z","fbj34lco59k2lc0","Go Channels","2026-06-07 06:49:16.397Z",{"code":56,"collectionId":57,"collectionName":58,"created":59,"flag":60,"id":61,"is_default":62,"label":63,"updated":64},"en","pbc_1989393366","locales","2026-01-22 11:00:02.726Z","twemoji:flag-united-states","qv9c1llfov2d88z",false,"English","2026-04-10 15:42:46.825Z",{"category":66,"collectionId":67,"collectionName":68,"created":69,"expand":70,"id":86,"slug":87,"updated":88,"views":89},"wqxt7ag2gn7xcmk","pbc_2105096300","school_blogs","2026-08-17 10:43:48.709Z",{"category":71},{"blogIds":72,"collectionId":73,"collectionName":74,"created":75,"created_by":14,"id":66,"image":76,"image_alt":77,"image_path":78,"image_s_url":79,"label":80,"name":81,"priority":82,"publish_at":83,"scheduled_at":77,"status":84,"updated":85,"updated_by":14},[],"sclcatblg987654321","school_category_blogs","2026-03-04 08:33:53.210Z","59ty92ns80w_15oc1implw_bj2o4tjgh4.webp","","https:\u002F\u002Ftwsme-r2.tumwebsme.com\u002Fsclcatblg987654321\u002Fwqxt7ag2gn7xcmk\u002F59ty92ns80w_15oc1implw_bj2o4tjgh4.webp","https:\u002F\u002Ftwsme-r2.tumwebsme.com\u002Fsclcatblg987654321\u002Fwqxt7ag2gn7xcmk\u002Fs\u002F59ty92ns80w_15oc1implw_bj2o4tjgh4.webp",{"en":81,"th":81},"Golang The Series",1,"2026-03-16 04:39:38.440Z","published","2026-08-18 11:12:33.729Z","2nqska91xchwzgb","golang-ai-batch-processing-worker-pool","2026-08-18 12:48:55.491Z",130,"9diiaifdwadw4js",[21,26,30,34,38,43,48,52],"2026-08-18 04:22:15.212Z","Learn how to build a high-performance AI Batch Processing system using Golang Worker Pool. Efficiently handle thousands of requests with concurrency and error handling.","Golang The Series EP.170: Building an AI Batch Processing System with Golang Worker Pool","2026-08-18 10:59:56.247Z",{"th":87,"en":87}]