[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"academy-blogs-en-1-1-all-js2go-ep38-buffer-and-stream-in-nodejs-and-go-all--*":3,"academy-blog-translations-sa6yrtkxl6r63no":80},{"data":4,"page":79,"perPage":79,"totalItems":79,"totalPages":79},[5],{"alt":6,"collectionId":7,"collectionName":8,"content":9,"cover_image":10,"cover_image_path":11,"created":12,"created_by":13,"expand":14,"id":73,"keywords":74,"locale":49,"published_at":75,"scheduled_at":13,"school_blog":71,"short_description":76,"slug":77,"status":69,"title":6,"updated":78,"updated_by":13,"views":72},"JS2GO EP.38 Buffer and Stream Management in Node.js and Go","sclblg987654321","school_blog_translations","\u003Cp>In a production environment, working with large-scale data — such as\u003C\u002Fp>\u003Cul>\u003Cli>Reading massive log files\u003C\u002Fli>\u003Cli>Downloading data over a network\u003C\u002Fli>\u003Cli>Or sending API responses\u003C\u002Fli>\u003C\u002Ful>\u003Cp> \u003C\u002Fp>\u003Cp>requires careful consideration of performance, memory efficiency, and system stability.\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Cp>Both JavaScript (Node.js) and Go (Golang) provide intelligent solutions for handling large data efficiently through two core concepts: Buffer and Stream.\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Cp>In this article, we’ll explore how to use Buffer and Stream effectively in both Node.js and Go, along with real-world code examples and best practices for production-ready systems. 🚀\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Ch2>1. What is a Buffer?\u003C\u002Fh2>\u003Cp> \u003C\u002Fp>\u003Cp>A Buffer is a temporary memory space used to store data before or during read\u002Fwrite operations.\u003Cbr>It allows applications to process large data efficiently without loading everything into memory at once (for example, a 1GB file).\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Ch3>🔹 Buffer in Node.js\u003C\u002Fh3>\u003Cp>In Node.js, the \u003Ccode inline=\"\">Buffer\u003C\u002Fcode> class handles raw binary data directly.\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Cpre>\u003Ccode class=\"language-javascript\">const fs = require('fs');\n\nfs.readFile('example.txt', (err, data) => {\n  if (err) throw err;\n  console.log(data);            \u002F\u002F Display raw buffer\n  console.log(data.toString()); \u002F\u002F Convert to string\n});\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp> \u003C\u002Fp>\u003Cp>Creating your own buffer:\u003C\u002Fp>\u003Cpre>\u003Ccode class=\"language-javascript\">const buf = Buffer.from('Hello Superdev!');\nconsole.log(buf);            \u002F\u002F <Buffer 48 65 6c 6c 6f ...>\nconsole.log(buf.toString()); \u002F\u002F Hello Superdev!\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp> \u003C\u002Fp>\u003Cp>Advantages of Buffer in Node.js:\u003Cbr>✅ Handles binary data directly\u003Cbr>✅ Great for file and network I\u002FO operations\u003Cbr>✅ More memory-efficient than loading entire data at once\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Ch3>🔹 Buffer in Go\u003C\u002Fh3>\u003Cp>In Go, you can use \u003Ccode inline=\"\">bytes.Buffer\u003C\u002Fcode> from the \u003Ccode inline=\"\">bytes\u003C\u002Fcode> package — a dynamic data structure that efficiently stores byte streams.\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Cpre>\u003Ccode class=\"language-plaintext language-go\">package main\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n)\n\nfunc main() {\n\tvar buffer bytes.Buffer\n\tbuffer.WriteString(\"Hello Superdev!\")\n\tfmt.Println(buffer.String())\n}\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp> \u003C\u002Fp>\u003Cp>Advantages of Buffer in Go:\u003Cbr>✅ Efficient memory usage\u003Cbr>✅ Can append data continuously without creating new strings\u003Cbr>✅ Ideal for sequential data processing like log aggregation or network streaming\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Ch2>2. What is a Stream?\u003C\u002Fh2>\u003Cp> \u003C\u002Fp>\u003Cp>A Stream processes data piece by piece (chunk by chunk) instead of loading it all at once. This approach allows systems to handle large datasets efficiently while minimizing memory consumption.\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Ch3>🔹 Stream in Node.js\u003C\u002Fh3>\u003Cp>In Node.js, almost every I\u002FO operation (File, Network, HTTP) can be performed as a Stream.\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Cpre>\u003Ccode class=\"language-javascript\">const fs = require('fs');\n\nconst readable = fs.createReadStream('bigfile.txt', { encoding: 'utf8' });\n\nreadable.on('data', chunk => {\n  console.log('📦 Received chunk:', chunk.length);\n});\n\nreadable.on('end', () => {\n  console.log('✅ Done reading file');\n});\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp> \u003C\u002Fp>\u003Cp>Explanation:\u003C\u002Fp>\u003Cul>\u003Cli>Node.js reads files in chunks (around 64KB per read).\u003C\u002Fli>\u003Cli>No need to load the entire file into memory.\u003C\u002Fli>\u003Cli>Perfect for logs, media, or API streaming tasks.\u003C\u002Fli>\u003C\u002Ful>\u003Cp> \u003C\u002Fp>\u003Cp>Writing data as a stream:\u003C\u002Fp>\u003Cpre>\u003Ccode class=\"language-javascript\">const writeStream = fs.createWriteStream('output.txt');\nwriteStream.write('Superdev Stream Example\\n');\nwriteStream.end('✅ Finished writing!');\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp> \u003C\u002Fp>\u003Ch3>🔹 Stream in Go\u003C\u002Fh3>\u003Cp>Go uses the \u003Ccode inline=\"\">io.Reader\u003C\u002Fcode> and \u003Ccode inline=\"\">io.Writer\u003C\u002Fcode> interfaces to handle data streaming.\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Cpre>\u003Ccode class=\"language-plaintext language-go\">package main\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n)\n\nfunc main() {\n\tfile, err := os.Open(\"bigfile.txt\")\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\tdefer file.Close()\n\n\tbuffer := make([]byte, 1024)\n\tfor {\n\t\tn, err := file.Read(buffer)\n\t\tif err == io.EOF {\n\t\t\tbreak\n\t\t}\n\t\tif err != nil {\n\t\t\tpanic(err)\n\t\t}\n\t\tfmt.Printf(\"📦 Read %d bytes\\n\", n)\n\t}\n\tfmt.Println(\"✅ Done reading file\")\n}\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp> \u003C\u002Fp>\u003Cp>Explanation:\u003C\u002Fp>\u003Cul>\u003Cli>Reads data in chunks of 1024 bytes.\u003C\u002Fli>\u003Cli>Gives developers full control over memory usage.\u003C\u002Fli>\u003Cli>The same principle applies to network connections (e.g., \u003Ccode inline=\"\">net.Conn\u003C\u002Fcode>).\u003C\u002Fli>\u003C\u002Ful>\u003Cp> \u003C\u002Fp>\u003Cp>Writing to a stream:\u003C\u002Fp>\u003Cpre>\u003Ccode class=\"language-plaintext language-go\">output, _ := os.Create(\"output.txt\")\ndefer output.Close()\noutput.Write([]byte(\"Superdev Stream Example\\n\"))\noutput.Write([]byte(\"✅ Finished writing!\"))\n\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp> \u003C\u002Fp>\u003Ch2>3. Comparing Buffer and Stream\u003C\u002Fh2>\u003Cp> \u003C\u002Fp>\u003Cfigure class=\"table\">\u003Ctable>\u003Cthead>\u003Ctr>\u003Cth>Feature\u003C\u002Fth>\u003Cth>Node.js\u003C\u002Fth>\u003Cth>Go\u003C\u002Fth>\u003C\u002Ftr>\u003C\u002Fthead>\u003Ctbody>\u003Ctr>\u003Ctd>Buffer Management\u003C\u002Ftd>\u003Ctd>Uses \u003Ccode inline=\"\">Buffer\u003C\u002Fcode> class\u003C\u002Ftd>\u003Ctd>Uses \u003Ccode inline=\"\">bytes.Buffer\u003C\u002Fcode>\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>File Streaming (Read)\u003C\u002Ftd>\u003Ctd>\u003Ccode inline=\"\">fs.createReadStream()\u003C\u002Fcode>\u003C\u002Ftd>\u003Ctd>\u003Ccode inline=\"\">io.Reader\u003C\u002Fcode>\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>File Streaming (Write)\u003C\u002Ftd>\u003Ctd>\u003Ccode inline=\"\">fs.createWriteStream()\u003C\u002Fcode>\u003C\u002Ftd>\u003Ctd>\u003Ccode inline=\"\">io.Writer\u003C\u002Fcode>\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>Network Handling\u003C\u002Ftd>\u003Ctd>\u003Ccode inline=\"\">net.Socket\u003C\u002Fcode> (TCP\u002FHTTP)\u003C\u002Ftd>\u003Ctd>\u003Ccode inline=\"\">net.Conn\u003C\u002Fcode>\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>Memory Control\u003C\u002Ftd>\u003Ctd>Automatic\u003C\u002Ftd>\u003Ctd>Fine-grained control\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>I\u002FO Performance\u003C\u002Ftd>\u003Ctd>Excellent\u003C\u002Ftd>\u003Ctd>Outstanding — more stable at scale\u003C\u002Ftd>\u003C\u002Ftr>\u003C\u002Ftbody>\u003C\u002Ftable>\u003C\u002Ffigure>\u003Cp> \u003C\u002Fp>\u003Ch2>4. Best Practices for Production Systems\u003C\u002Fh2>\u003Cp> \u003C\u002Fp>\u003Cp>💡 Use Buffer when:\u003C\u002Fp>\u003Cul>\u003Cli>Data size is small enough to fit into memory\u003C\u002Fli>\u003Cli>You’re working with strings, JSON, or small log files\u003C\u002Fli>\u003C\u002Ful>\u003Cp> \u003C\u002Fp>\u003Cp>⚡ Use Stream when:\u003C\u002Fp>\u003Cul>\u003Cli>Data size is large (e.g., file uploads\u002Fdownloads, network transfers)\u003C\u002Fli>\u003Cli>You want to minimize memory footprint\u003C\u002Fli>\u003Cli>You’re handling real-time or continuous data such as video streaming or API responses\u003C\u002Fli>\u003C\u002Ful>\u003Cp> \u003C\u002Fp>\u003Cp>Node.js:\u003Cbr>Perfect for API servers, file streaming, and real-time applications.\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Cp>Go:\u003Cbr>Ideal for backend systems requiring high throughput and low latency, such as data pipelines or log processors.\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Ch2>5. Real-World Use Cases\u003C\u002Fh2>\u003Cp> \u003C\u002Fp>\u003Cfigure class=\"table\">\u003Ctable>\u003Cthead>\u003Ctr>\u003Cth>Use Case\u003C\u002Fth>\u003Cth>Recommended Language\u003C\u002Fth>\u003Cth>Approach\u003C\u002Fth>\u003C\u002Ftr>\u003C\u002Fthead>\u003Ctbody>\u003Ctr>\u003Ctd>File Upload \u002F Download\u003C\u002Ftd>\u003Ctd>Node.js\u003C\u002Ftd>\u003Ctd>Use \u003Ccode inline=\"\">fs.createReadStream()\u003C\u002Fcode> + \u003Ccode inline=\"\">.pipe()\u003C\u002Fcode>\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>Real-time Data Processing\u003C\u002Ftd>\u003Ctd>Go\u003C\u002Ftd>\u003Ctd>Use \u003Ccode inline=\"\">io.Reader\u003C\u002Fcode> and \u003Ccode inline=\"\">io.Writer\u003C\u002Fcode>\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>Network Proxy\u003C\u002Ftd>\u003Ctd>Go\u003C\u002Ftd>\u003Ctd>Use \u003Ccode inline=\"\">net.Conn\u003C\u002Fcode> + \u003Ccode inline=\"\">bufio\u003C\u002Fcode> for streaming\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>Video Streaming\u003C\u002Ftd>\u003Ctd>Node.js\u003C\u002Ftd>\u003Ctd>Use Stream API + HTTP Chunked Responses\u003C\u002Ftd>\u003C\u002Ftr>\u003C\u002Ftbody>\u003C\u002Ftable>\u003C\u002Ffigure>\u003Cp> \u003C\u002Fp>\u003Chr>\u003Cp> \u003C\u002Fp>\u003Ch2>Conclusion\u003C\u002Fh2>\u003Cp> \u003C\u002Fp>\u003Cp>Understanding Buffer and Stream is essential for developers working with large-scale data systems.\u003Cbr>By mastering both concepts, you can build software that runs faster, uses less memory, and handles heavier loads efficiently.\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Cp>✅ Use Stream for large I\u002FO operations\u003Cbr>✅ Use Buffer for small or intermediate data handling\u003Cbr>💻 Node.js: Flexible, simple, and great for real-time use cases\u003Cbr>⚙️ Go: High-performance, concurrency-friendly, and ideal for enterprise systems\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Ch2>Next Episode\u003C\u002Fh2>\u003Cp> \u003C\u002Fp>\u003Cp>In EP.39 of JS2GO, we’ll explore Channels and Pipelines in Go for Data Processing —\u003Cbr>diving deep into asynchronous and parallel processing concepts, and how to design pipelines that perform multiple tasks concurrently with maximum efficiency ⚡\u003C\u002Fp>\u003Cp> \u003C\u002Fp>\u003Cp data-start=\"498\" data-end=\"834\">\u003Cstrong>Read more\u003C\u002Fstrong>\u003C\u002Fp>\u003Cul>\u003Cli>\u003Cp data-start=\"498\" data-end=\"834\">\u003Ca target=\"_blank\" rel=\"noopener noreferrer\" href=\"https:\u002F\u002Fwww.superdev.school\u002Fblogs\u002Fcategories\u002FGolang\">\u003Cstrong>Golang The Series\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp data-start=\"498\" data-end=\"834\">\u003Ca target=\"_blank\" rel=\"noopener noreferrer\" href=\"https:\u002F\u002Fwww.superdev.school\u002Fblogs\u002Fcategories\u002FJS2GO\">\u003Cstrong>JS2GO\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003Cli>\u003Cp data-start=\"498\" data-end=\"834\">\u003Ca target=\"_blank\" rel=\"noopener noreferrer\" href=\"https:\u002F\u002Fwww.superdev.school\u002Fen\u002Fblogs\u002Fcategories\u002FTailwind%20CSS\">\u003Cstrong>10 Eps That Will Make You a Pro Tailwind CSS Overnight\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003C\u002Fli>\u003C\u002Ful>\u003Cp>\u003Cstrong>🔵 Facebook: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener noreferrer\" href=\"https:\u002F\u002Fwww.facebook.com\u002Fsuperdev.academy.th\">\u003Cstrong>Superdev Academy\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003Cp>\u003Cstrong>🔴 YouTube: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener noreferrer\" href=\"https:\u002F\u002Fwww.youtube.com\u002Fchannel\u002FUC2eI2RxcA2zbTqZyHyWIfRg\">\u003Cstrong>Superdev Academy\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003Cp>\u003Cstrong>📸 Instagram: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener noreferrer\" href=\"https:\u002F\u002Fwww.instagram.com\u002Fsuperdevschool\u002F\">\u003Cstrong>Superdev Academy\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003Cp>\u003Cstrong>🎬 TikTok: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener noreferrer\" href=\"https:\u002F\u002Fwww.tiktok.com\u002F@superdevacademy?lang=th-TH\">\u003Cstrong>https:\u002F\u002Fwww.tiktok.com\u002F@superdevacademy?lang=th-TH\u003C\u002Fstrong>\u003C\u002Fa>\u003C\u002Fp>\u003Cp>\u003Cstrong>🌐 Website: \u003C\u002Fstrong>\u003Ca target=\"_blank\" rel=\"noopener noreferrer\" href=\"https:\u002F\u002Fwww.superdevacademy.com\u002Fen\">\u003Cstrong>https:\u002F\u002Fwww.superdevacademy.com\u002Fen\u003C\u002Fstrong>\u003C\u002Fa>\u003Cstrong> \u003C\u002Fstrong>\u003C\u002Fp>","76_11zon_82cz71256m.webp","https:\u002F\u002Ftwsme-r2.tumwebsme.com\u002Fsclblg987654321\u002Fnwfy0uzqz75u69b\u002F76_11zon_82cz71256m.webp","2026-03-04 08:45:30.569Z","",{"keywords":15,"locale":43,"school_blog":53},[16,23,28,33,38],{"collectionId":17,"collectionName":18,"created":19,"created_by":13,"id":20,"name":21,"updated":22,"updated_by":13},"sclkey987654321","school_keywords","2026-03-04 08:20:14.253Z","ah6lvy4x8qe08l5","Golang","2026-04-10 16:07:26.172Z",{"collectionId":17,"collectionName":18,"created":24,"created_by":13,"id":25,"name":26,"updated":27,"updated_by":13},"2026-03-04 08:20:11.547Z","ey3puyme01a9bsw","Go","2026-04-10 16:07:25.893Z",{"collectionId":17,"collectionName":18,"created":29,"created_by":13,"id":30,"name":31,"updated":32,"updated_by":13},"2026-03-04 08:24:40.405Z","05u3tysava2z6ga","Node.js","2026-04-10 16:07:29.504Z",{"collectionId":17,"collectionName":18,"created":34,"created_by":13,"id":35,"name":36,"updated":37,"updated_by":13},"2026-03-04 08:45:29.064Z","ppd9qfd57kr4d9v","Stream","2026-04-10 16:12:58.890Z",{"collectionId":17,"collectionName":18,"created":39,"created_by":13,"id":40,"name":41,"updated":42,"updated_by":13},"2026-03-04 08:45:29.466Z","ikxxjlf4gp9demr","Buffer","2026-04-10 16:12:59.000Z",{"code":44,"collectionId":45,"collectionName":46,"created":47,"flag":48,"id":49,"is_default":50,"label":51,"updated":52},"en","pbc_1989393366","locales","2026-01-22 11:00:02.726Z","twemoji:flag-united-states","qv9c1llfov2d88z",false,"English","2026-04-10 15:42:46.825Z",{"category":54,"collectionId":55,"collectionName":56,"expand":57,"id":71,"views":72},"hsa1afr8fcnd6qb","pbc_2105096300","school_blogs",{"category":58},{"blogIds":59,"collectionId":60,"collectionName":61,"created":62,"created_by":13,"id":54,"image":63,"image_alt":13,"image_path":64,"label":65,"name":66,"priority":67,"publish_at":68,"scheduled_at":13,"status":69,"updated":70,"updated_by":13},[],"sclcatblg987654321","school_category_blogs","2026-03-04 08:24:37.986Z","js2_go_2_11zon_y6paxmuz32.webp","https:\u002F\u002Ftwsme-r2.tumwebsme.com\u002Fsclcatblg987654321\u002Fhsa1afr8fcnd6qb\u002Fjs2_go_2_11zon_y6paxmuz32.webp",{"en":66,"th":66},"JS2GO",10,"2025-08-11 03:41:08.820Z","published","2026-04-25 02:32:14.339Z","sa6yrtkxl6r63no",212,"nwfy0uzqz75u69b",[20,25,30,35,40],"2025-11-12 02:36:52.979Z","Learn how to handle large-scale data efficiently using Buffer and Stream in Node.js and Go. This article covers real-world use cases, code examples, and best practices for building high-performance production systems.","js2go-ep38-buffer-and-stream-in-nodejs-and-go","2026-04-25 02:47:53.937Z",1,{"en":77}]