Ep.19 Go and Worker Pool: Enhancing Concurrent Processing!

Go and Worker Pool: Enhancing Concurrent Processing!

What is a Worker Pool?

A Worker Pool is a structure that divides tasks into multiple parts and distributes them to "Workers" that operate concurrently. This allows the program to process tasks more quickly without waiting for one Worker to finish before starting the next task.

Creating a Worker Pool in Go

We create a Worker Pool using Goroutines and Channels to communicate between tasks and Workers.

Steps to Create a Simple Worker Pool:

1. Create Channels for Tasks and Results

We will create a Channel to send tasks to Workers and another Channel to receive results sent back from Workers.

jobs := make(chan int, 5)     // ช่องรับงาน
results := make(chan int, 5)  // ช่องรับผลลัพธ์

2. Create a Worker Function

The Worker will perform tasks received from the Channel and send results back through the results Channel.

func worker(id int, jobs <-chan int, results chan<- int) {
    for job := range jobs {
        fmt.Printf("Worker %d started job %d\n", id, job)
        time.Sleep(time.Second) // จำลองเวลาทำงาน
        fmt.Printf("Worker %d finished job %d\n", id, job)
        results <- job * 2 // ส่งผลลัพธ์กลับไปยัง results
    }
}

3. Start Multiple Workers Concurrently

We can create multiple Workers (Goroutines) to receive tasks from the Channel.

for w := 1; w <= 3; w++ { // สร้าง Worker 3 ตัว
    go worker(w, jobs, results)
}

4. Send Tasks into the Task Channel

We will send tasks into the jobs Channel for Workers to process.

for j := 1; j <= 5; j++ {
    jobs <- j
}
close(jobs) // ปิด Channel ของงานเมื่อส่งงานครบ

5. Receive Results from the Results Channel

Receive results from the results Channel, which the Workers have sent back.

for a := 1; a <= 5; a++ {
    result := <-results
    fmt.Println("Result:", result)
}

Full Example Code of a Worker Pool

The program creates 3 Workers and sends 5 tasks for them to handle. Each Worker performs its assigned tasks concurrently and sends the results back to the results Channel.

package main

import (
    "fmt"
    "time"
)

func worker(id int, jobs <-chan int, results chan<- int) {
    for job := range jobs {
        fmt.Printf("Worker %d started job %d\n", id, job)
        time.Sleep(time.Second) // จำลองเวลาทำงาน
        fmt.Printf("Worker %d finished job %d\n", id, job)
        results <- job * 2 // ส่งผลลัพธ์
    }
}

func main() {
    jobs := make(chan int, 5)
    results := make(chan int, 5)

    for w := 1; w <= 3; w++ {
        go worker(w, jobs, results)
    }

    for j := 1; j <= 5; j++ {
        jobs <- j
    }
    close(jobs)

    for a := 1; a <= 5; a++ {
        fmt.Println("Result:", <-results)
    }
}

Benefits of a Worker Pool:

• Resource Efficiency: Perform multiple tasks concurrently using a limited number of Workers.
• Improved Performance: Reduced wait times as tasks are divided into smaller parts.
• Easier Management: Suitable for repetitive tasks in large volumes.

In Summary:

• Create Channels for sending tasks and receiving results.
• Use Goroutines to create multiple Workers.
• Send tasks to Workers through Channels and receive results when completed.