View : 0

12/04/2026 18:16pm

JS2GO EP.39 Using Channels and Pipelines in Go for Data Processing

JS2GO EP.39 Using Channels and Pipelines in Go for Data Processing

#Channels

#JavaScript

#Golang

#Go

#Pipelines

#Concurrency

#Data Processing

When working with large-scale data such as:

  • Reading multiple files concurrently
  • Processing data from multiple APIs
  • Or transforming datasets through multiple stages (Transform → Filter → Save)

 

you need parallel processing techniques to increase system throughput and optimize resource utilization.

 

Both JavaScript and Go support asynchronous processing, but their approaches are fundamentally different:

 

LanguageCore Concept
JavaScriptPromises / async-await / Event Loop
Go (Golang)Goroutines + Channels + Pipelines

 

1. Concurrency Concept: JavaScript vs Go

 

FeatureJavaScriptGo
Core ModelAsync / Event LoopGoroutines + Channels
Execution ModelSingle-threadedLightweight Multi-threaded
Inter-process CommunicationCallbacks / EventsChannels
StrengthsEasy to use, ideal for I/O-bound tasksHigh performance, great for CPU-bound tasks

 

2. Basic Example: Concurrency in JavaScript

 

In JavaScript, concurrent execution can be achieved using async/await or Promise.all().

 

async function processData() {
  const data = [1, 2, 3, 4, 5];

  const results = await Promise.all(
    data.map(async (n) => {
      await new Promise((r) => setTimeout(r, 500)); // simulate processing
      return n * 2;
    })
  );

  console.log("✅ Processed Data:", results);
}

processData();

 

Output:
✅ Processed Data: [2, 4, 6, 8, 10]

 

Explanation:
JavaScript creates a separate Promise for each task and runs them concurrently in the background via the Event Loop.

 

3. Basic Example: Concurrency in Go

 

In Go, you can easily run functions concurrently using goroutines.

 

package main

import (
	"fmt"
	"time"
)

func process(n int, ch chan int) {
	time.Sleep(500 * time.Millisecond) // simulate processing
	ch <- n * 2
}

func main() {
	data := []int{1, 2, 3, 4, 5}
	ch := make(chan int)

	for _, n := range data {
		go process(n, ch)
	}

	for i := 0; i < len(data); i++ {
		fmt.Println("✅ Result:", <-ch)
	}
}

 

Output:
✅ Result: 2
✅ Result: 4
✅ Result: 6
✅ Result: 8
✅ Result: 10

 

Explanation:

  • The go process(n, ch) command launches a goroutine, running concurrently with the main function.
  • The channel (ch) acts as a safe communication pipeline between goroutines for passing results back.

 

4. Understanding Channels

 

A channel in Go is a "communication pipe" that allows goroutines to send and receive data without locks or shared memory.

 

messages := make(chan string)

go func() {
	messages <- "Hello from goroutine"
}()

msg := <-messages
fmt.Println(msg)

 

Output:
Hello from goroutine

 

Summary:
✅ Channels allow goroutines to communicate safely without using locks or mutexes.
✅ They are the foundation of Go’s concurrent programming model.

 

5. The Pipeline Pattern in Go

 

A Pipeline connects multiple goroutines via channels
each stage receives input from one channel, processes it, and passes results to the next.

 

🔹 Example: Simple Pipeline

 

package main

import "fmt"

// Stage 1: Generate data
func generate(nums ...int) <-chan int {
	out := make(chan int)
	go func() {
		for _, n := range nums {
			out <- n
		}
		close(out)
	}()
	return out
}

// Stage 2: Process data (multiply by 2)
func multiply(in <-chan int) <-chan int {
	out := make(chan int)
	go func() {
		for n := range in {
			out <- n * 2
		}
		close(out)
	}()
	return out
}

// Stage 3: Print results
func main() {
	nums := generate(1, 2, 3, 4, 5)
	results := multiply(nums)

	for v := range results {
		fmt.Println("✅ Result:", v)
	}
}

 

Output:
✅ Result: 2
✅ Result: 4
✅ Result: 6
✅ Result: 8
✅ Result: 10

 

Explanation:

  • generate() produces data and sends it through a channel.
  • multiply() processes the input concurrently.
  • Each stage connects seamlessly, enabling true parallel processing.

 

6. Multi-stage Pipelines

 

You can extend a pipeline into multiple processing stages, for example:
generate → square → filter → aggregate → output

Example: 3-stage Pipeline

 

package main

import "fmt"

func generate(nums ...int) <-chan int {
	out := make(chan int)
	go func() {
		for _, n := range nums {
			out <- n
		}
		close(out)
	}()
	return out
}

func square(in <-chan int) <-chan int {
	out := make(chan int)
	go func() {
		for n := range in {
			out <- n * n
		}
		close(out)
	}()
	return out
}

func filterEven(in <-chan int) <-chan int {
	out := make(chan int)
	go func() {
		for n := range in {
			if n%2 == 0 {
				out <- n
			}
		}
		close(out)
	}()
	return out
}

func main() {
	nums := generate(1, 2, 3, 4, 5)
	squared := square(nums)
	filtered := filterEven(squared)

	for v := range filtered {
		fmt.Println("✅ Filtered Result:", v)
	}
}

 

Output:
✅ Filtered Result: 4
✅ Filtered Result: 16

 

7. Comparison with JavaScript

 

JavaScript can also implement pipelines using Array methods, async generators, or RxJS.

 

async function* generate(nums) {
  for (const n of nums) yield n;
}

async function* square(input) {
  for await (const n of input) yield n * n;
}

async function* filterEven(input) {
  for await (const n of input) if (n % 2 === 0) yield n;
}

for await (const val of filterEven(square(generate([1,2,3,4,5])))) {
  console.log("✅ Filtered Result:", val);
}

 

Advantages of Go:
✅ More efficient concurrency via channels
✅ Fine-grained control over flow and memory
✅ No overhead from an event loop

 

Advantages of JavaScript:
✅ Simpler syntax and easier to learn
✅ Excellent for I/O-bound workloads such as API requests or streaming

 

8. Best Practices

 

💡 Use Buffered Channels (e.g., make(chan int, 10)) to avoid blocking.
💡 Always close channels when the goroutine finishes (close(out)).
💡 Use select statements to handle timeouts or cancellations.
💡 For complex pipelines, define each stage as a separate, clearly named function.

 


 

Summary

 

AspectJavaScriptGo
Async ModelEvent Loop + PromisesGoroutines + Channels
Parallel ExecutionSimulated via asyncTrue concurrent threads
Pipeline DesignFunctional / Stream-basedChannel-based Pipeline
PerformanceExcellent for I/O-bound tasksOutstanding for CPU-bound tasks
ComplexityEasier to write and readRequires understanding of goroutines and channels

 

Recommendation:

  • For I/O-heavy systems (API, streaming, frontend) → use JavaScript
  • For high-performance concurrent systems → choose Go

 

Next Episode

 

In EP.40 of JS2GO, we’ll explore Code Performance Optimization: Go vs JavaScript —
comparing advanced techniques such as Garbage Collection, Memory Profiling, Parallel Execution, and Benchmark Tools to find out which language truly performs faster in the real world 🧠💨

 

Read more

🔵 Facebook: Superdev Academy

🔴 YouTube: Superdev Academy

📸 Instagram: Superdev Academy

🎬 TikTok: https://www.tiktok.com/@superdevacademy?lang=th-TH

🌐 Website: https://www.superdevacademy.com/en