What Is an API Gateway? How It Differs from a Load Balancer

Picture a food-delivery app that started life as a single backend. Everything lived on one server, and the app called one URL. Then the app grew, and the team split the system into five services: user accounts, restaurants, orders, payments, and rider tracking. One day the frontend developer opens the config file and finds five sets of URLs. Each service checks logins its own way. Some block clients that call too often, some do not. And every time the backend team moves or splits a service, the app has to change with it.

The obvious question is whether there could be one place for the app to talk to, with that place passing each request to the right service. In short: an API gateway is a single entry point that sits in front of many backend services, receives every request from clients, checks who is calling, limits how often they can call, and forwards each request to the correct service. Today Superdev Academy walks through how it works, what it does, how it differs from a load balancer and a reverse proxy, and then builds a tiny gateway in Node.js so you can watch it return 429 when it is called too often.

If terms like API or endpoint are still new to you, read What Is an API? first and come back. This article will be much easier.

What is an API gateway?

An API gateway is an intermediary server that every client (a mobile app, a website, a partner's system) sends its requests to. The gateway then decides which service behind it should handle each request.

Microsoft's Azure Architecture Center describes an API gateway as a centralized entry point that acts as a reverse proxy, routes client requests to the appropriate services, and takes on cross-cutting tasks such as authentication, SSL termination, and rate limiting. AWS calls Amazon API Gateway a "front door" for applications to reach data and business logic in backend services.

Put simply, it is the reception desk of an office building. Visitors do not need to know which floor each department is on. They tell the desk what they came for, the desk checks their ID, and points them to the right room.

How does an API gateway work?

Put a gateway in front of the food-delivery app above, and a single request travels like this:

1. The app sends GET /orders to one URL: the gateway's.

2. The gateway checks whether the request carries a valid token or API key. If not, it answers 401 straight away, and the services behind it never see the request.

3. The gateway checks whether this client has gone over its quota. If so, it answers 429.

4. The gateway looks at the path and matches it against its routing rules: /orders goes to orders-service.

5. The service responds, the gateway writes a log line, and the response goes back to the app.

What changed is that the app now knows exactly one URL. Moving, splitting, or merging services behind it becomes a change to the gateway's rules, not a new app release.

What does an API gateway do?

The jobs below are the core ones that Microsoft's and AWS's documentation agree on. Not every gateway turns on every one, and each product supports each job to a different degree.

Routing: sending each request to the right service

This is the most basic job. The gateway looks at the path, a header, or the host, and forwards the request according to its rules: /users goes to users-service, /orders goes to orders-service. Microsoft's Gateway Routing pattern notes that this routing happens at Layer 7, the HTTP layer, which is why it can decide based on a URL or a header.

Authentication: checking identity once, at the door

Instead of every service writing its own token-checking code, the gateway checks once for all of them. Amazon API Gateway supports access control through IAM policies, Lambda authorizers, and Amazon Cognito user pools. Keeping the API keys themselves safe on the server side is a separate topic, covered in our article on storing API keys in .env.

Rate limiting: blocking clients that call too often, with 429

Rate limiting caps how many calls a client can make in a given time window, so that a single client cannot eat the whole system's capacity. The status code that says "you have been limited" is 429 Too Many Requests, defined in RFC 6585, which adds that the response may include a Retry-After header telling the client how long to wait.

Each vendor counts differently. AWS states that Amazon API Gateway throttles requests with the token bucket algorithm, and that once requests exceed both the steady-state rate and the burst limit, clients may receive 429 responses. NGINX uses the limit_req module, which uses the leaky bucket method, and here is the detail many people miss: the default value of limit_req_status is 503, not 429. If you want 429, you have to set it yourself.

Fun Fact: for the other 4xx/5xx codes gateways like to return, such as 401, 502, and 503, see HTTP Status Codes Explained.

TLS termination: decrypting HTTPS at the door

TLS termination (also called SSL termination) means the gateway holds the certificates and decrypts HTTPS traffic from clients, then talks to the services behind it on the internal network. The benefit is managing certificates in one place instead of installing them on every service. If you also need encryption inside the network, Microsoft lists mutual TLS as another job that can be offloaded to the gateway.

Request/response transformation: reshaping data on the way through

Some gateways can rewrite headers, change paths, or reshape payloads before forwarding them. For example, a gateway can add a header that identifies the user for the service behind it, or combine responses from several services into one. Microsoft calls the second case the Gateway Aggregation pattern, and it cuts down the number of requests the app has to make.

Logging and monitoring: every request in one place

Because every request passes through the gateway, it is the best place to collect logs and metrics. Amazon API Gateway can send access logs and execution logs to CloudWatch and raise alarms, so you can see which endpoint is slow and who keeps hitting 429.

API gateway example: building one in Node.js

Enough theory. Let's run the real thing. This example uses only the node:http module that ships with Node.js, so there is nothing to install. We tested it on Node.js v26.7.0, curl 8.4.0, and macOS 14.3.1 on 1 October 2026.

The first file is two mock services: one returns users, the other returns orders.

// services.js — two tiny backend services
const http = require('node:http');

function service(name, port, payload) {
  http.createServer((req, res) => {
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify({ service: name, path: req.url, data: payload }));
  }).listen(port, () => console.log(`${name} on :${port}`));
}

service('users-service', 4001, [{ id: 1, name: 'Nok' }]);
service('orders-service', 4002, [{ id: 101, item: 'Pad Thai' }]);

The second file is the gateway. It does four things: checks the API key, allows 3 requests per 10 seconds, routes by path, and writes a log line.

// gateway.js — one entry point in front of two services
const http = require('node:http');

const routes = {
  '/users': 'http://localhost:4001',
  '/orders': 'http://localhost:4002',
};

const LIMIT = 3;            // requests allowed...
const WINDOW_MS = 10_000;   // ...per 10 seconds, per client
const hits = new Map();     // client -> { count, start }

function rateLimited(client) {
  const now = Date.now();
  const entry = hits.get(client);
  if (!entry || now - entry.start >= WINDOW_MS) {
    hits.set(client, { count: 1, start: now });
    return false;
  }
  entry.count += 1;
  return entry.count > LIMIT;
}

http.createServer((req, res) => {
  // 1) authentication: one check, for every service behind the gateway
  const key = req.headers['x-api-key'];
  if (key !== 'demo-key') {
    res.writeHead(401, { 'Content-Type': 'application/json' });
    return res.end(JSON.stringify({ error: 'missing or wrong x-api-key' }));
  }

  // 2) rate limiting: per API key
  if (rateLimited(key)) {
    res.writeHead(429, { 'Content-Type': 'application/json', 'Retry-After': '10' });
    return res.end(JSON.stringify({ error: 'Too Many Requests' }));
  }

  // 3) routing: match the path prefix, forward to the right service
  const prefix = Object.keys(routes).find((p) => req.url.startsWith(p));
  if (!prefix) {
    res.writeHead(404, { 'Content-Type': 'application/json' });
    return res.end(JSON.stringify({ error: 'no route' }));
  }

  const upstream = new URL(req.url, routes[prefix]);
  const proxyReq = http.request(upstream, { method: req.method, headers: req.headers }, (up) => {
    // 4) logging: one line per request, in one place
    console.log(`${req.method} ${req.url} -> ${routes[prefix]} ${up.statusCode}`);
    res.writeHead(up.statusCode, up.headers);
    up.pipe(res);
  });
  proxyReq.on('error', () => {
    res.writeHead(502, { 'Content-Type': 'application/json' });
    res.end(JSON.stringify({ error: 'Bad Gateway' }));
  });
  req.pipe(proxyReq);
}).listen(8080, () => console.log('gateway on :8080'));

Open two terminals, run node services.js and node gateway.js, then call it with curl. This is the actual output from our machine:

$ curl -s localhost:8080/users -H "x-api-key: demo-key"
{"service":"users-service","path":"/users","data":[{"id":1,"name":"Nok"}]}

$ curl -s localhost:8080/orders -H "x-api-key: demo-key"
{"service":"orders-service","path":"/orders","data":[{"id":101,"item":"Pad Thai"}]}

$ curl -s -w ' %{http_code}\n' localhost:8080/users
{"error":"missing or wrong x-api-key"} 401

There is one URL, localhost:8080, yet /users and /orders reach different services, and the request without a key is turned away at the gateway without ever reaching a service. Next, wait for the 10-second window to reset and fire five calls in a row:

$ for i in 1 2 3 4 5; do curl -s -o /dev/null -w "call $i: %{http_code}\n" localhost:8080/orders -H "x-api-key: demo-key"; done
call 1: 200
call 2: 200
call 3: 200
call 4: 429
call 5: 429

$ curl -si localhost:8080/orders -H "x-api-key: demo-key" | grep -E "HTTP|Retry"
HTTP/1.1 429 Too Many Requests
Retry-After: 10

The first three calls pass. From the fourth call on, the gateway answers 429 with Retry-After: 10, as RFC 6585 suggests. The gateway's log has exactly five lines for the requests it forwarded (the first two, plus the three that passed in the loop). The requests rejected with 401 or 429 never reached a service, so they do not appear here.

gateway on :8080
GET /users -> http://localhost:4001 200
GET /orders -> http://localhost:4002 200
GET /orders -> http://localhost:4002 200
GET /orders -> http://localhost:4002 200
GET /orders -> http://localhost:4002 200

To be clear, this code exists to show the mechanism, not to go to production. The rate-limit counter lives in the memory of one process, so two gateway instances would each count separately. Real systems usually keep the counter in a shared store such as Redis, or use an off-the-shelf gateway. For a more detailed rate-limiting implementation in Go, read Golang The Series EP.164: Rate Limiting AI Requests.

What API gateways are there?

In real projects almost nobody writes a gateway from scratch. The common choices include both self-hosted software and cloud services.

What is Kong API Gateway?

Kong's documentation describes Kong Gateway as a lightweight, cloud-native API gateway that runs in front of any RESTful API and can be extended through modules and plugins, such as rate limiting or authentication. You can run it on your own servers or use the managed version through Konnect.

What is AWS API Gateway?

Amazon API Gateway is the AWS service for creating, publishing, maintaining, monitoring, and securing REST, HTTP, and WebSocket APIs, and it is very often paired with AWS Lambda. One detail worth knowing: a REST API's integration timeout can be set from 50 milliseconds to 29 seconds, and when the service behind it does not respond in time, the gateway returns a 504 (the INTEGRATION_TIMEOUT response type in the AWS documentation).

NGINX and general-purpose reverse proxies

Microsoft lists NGINX and HAProxy as open-source reverse proxies that can serve as an API gateway, because they already support load balancing, SSL termination, and Layer 7 routing. On the Azure side there are Azure API Management and Azure Application Gateway, depending on the job.

API gateway vs load balancer vs reverse proxy: what is the difference?

These three get confused more than any other terms here, because all three sit between clients and servers. The most precise way to tell them apart is to ask what question each one was designed to answer.

• Reverse proxy: the underlying mechanism, a server that accepts requests on behalf of the backend and forwards them. The NGINX documentation says proxying is typically used to distribute load among several servers, show content from different websites, or pass requests to application servers over protocols other than HTTP.

• Load balancer: answers "which machine is free?" AWS describes Elastic Load Balancing as distributing incoming traffic across multiple targets, monitoring their health, and routing traffic only to healthy targets. It is usually used for many copies of the same service.

• API gateway: answers "who is this request from, are they allowed, and which service should get it?" Kong puts it plainly: an API gateway is a reverse proxy that lets you manage, configure, and route requests to your APIs. In other words, a reverse proxy with API-level jobs added on top.

So the relationship is: an API gateway is always a kind of reverse proxy, but a reverse proxy is not necessarily an API gateway, and the roles can be stacked. Microsoft notes, for example, that Azure API Management does not do load balancing itself, so it should be used together with a load balancer such as Azure Application Gateway. In real systems you will see a load balancer in front of several gateway instances, and also a gateway forwarding to a load balancer for each service. The order is not fixed; it depends on the architecture. For the load-balancing side in hands-on Go, Golang The Series EP.165 covers it too.

Does a small system need an API gateway?

Honestly, not yet, and often it should not have one.

If your app has a single backend, the app already calls one URL. Adding a gateway adds one more network hop and one more system to maintain, without solving any problem you have. Microsoft's own Gateway Routing pattern says the pattern might not be suitable when you have a simple application that uses only one or two services.

You should also know the gateway has a cost. The same document warns that the gateway can become a single point of failure (the one part whose outage takes down everything) and a bottleneck if it cannot handle the load. The fixes are to run more than one gateway instance, load-test it, and stop the services behind it from accepting outside traffic directly, so that everything has to pass through the gateway.

The signals that it is time are exactly what the food-delivery app showed at the start: the app has to remember many URLs, every service repeats the same login-checking code, and moving one service means changing the app.

Summary: what an API gateway is, and when to start using one

An API gateway is a single entry point in front of many backend services. It handles routing, authentication, rate limiting with 429, TLS termination, request reshaping, and logging in one place. It is a kind of reverse proxy with API-level jobs added, and it solves a different problem from a load balancer, which spreads traffic across many machines.

If your system still has one backend today, there is no rush. But take the 57-line gateway above and play with it: add a third route, change the rate-limit ceiling, and watch how the 429s change. When your app does split into many services, you will know immediately what a real gateway like Kong or Amazon API Gateway is doing for you.

Follow more articles and tips for people starting out as programmers on our channels.

Facebook: Superdev Academy Thailand (https://www.facebook.com/superdev.academy.th)

YouTube: Superdev Academy Channel (https://www.youtube.com/@SuperdevAcademy)

Instagram: @superdevacademy (https://www.instagram.com/superdevacademy/)

TikTok: @superdevacademy (https://www.tiktok.com/@superdevacademy?lang=th-TH)