Load Balancing Explained: Layer 4 vs Layer 7, Algorithms, Health Checks, Nginx, HAProxy and Cloud Load Balancers

Modern applications cannot rely indefinitely on a single server.

A simple application may begin with:

Internet
   │
   ▼
Application Server

But as traffic increases, that architecture creates an obvious limitation:

                    All Traffic
                        │
                        ▼
                   Single Server
                        │
          ┌─────────────┼─────────────┐
          │             │             │
       CPU Load     Memory Load    Connections
          │             │             │
          └─────────────┼─────────────┘
                        ▼
                     Failure

The server eventually becomes either a performance bottleneck or a single point of failure.

A common solution is horizontal scaling:

Server A
Server B
Server C
Server D

But adding servers creates another problem:

Which server should receive each request?

This is the problem solved by load balancing.

A load balancer sits between clients and backend resources and distributes incoming traffic according to health, capacity, routing rules, connection state, or a selected balancing algorithm.

                     Internet
                        │
                        ▼
                  Load Balancer
                        │
            ┌───────────┼───────────┐
            ▼           ▼           ▼
         Server A    Server B    Server C

Load balancing is therefore a fundamental component of high availability, horizontal scaling, cloud architecture, Kubernetes, microservices, API infrastructure, databases, and distributed systems.

This guide explains load balancing from first principles through production architecture, including Layer 4 vs Layer 7 load balancing, Round Robin, Least Connections, hashing, weighted balancing, health checks, session persistence, failover, Nginx, HAProxy, AWS Application Load Balancer, containers, Kubernetes, observability, and troubleshooting.

If you have not already covered the traffic layer immediately before this topic, read Reverse Proxy Explained. For the caching layer commonly placed ahead of application infrastructure, see CDN Caching Explained.

What Is Load Balancing?

Load balancing is the process of distributing network connections or application requests across multiple computing resources.

Without load balancing:

Users
  │
  ▼
Server A

With load balancing:

                   Users
                     │
                     ▼
                Load Balancer
                     │
          ┌──────────┼──────────┐
          ▼          ▼          ▼
       Server A   Server B   Server C

The objective is not necessarily to distribute every request equally.

A production load balancer may consider:

Server Availability
Current Connections
Server Capacity
Request Type
Hostname
URL Path
Client Identity
Response Time
Geographic Location

before selecting a destination.

The real objective is therefore:

Send traffic to an appropriate healthy backend while maintaining application performance and availability.

Why Load Balancing Is Necessary

Suppose one server can reliably process:

1,000 requests/second

but traffic increases to:

4,000 requests/second

You could attempt to replace it with a more powerful machine.

That is vertical scaling.

Or you could deploy multiple application servers:

Server A → 1,000 req/s
Server B → 1,000 req/s
Server C → 1,000 req/s
Server D → 1,000 req/s

This is horizontal scaling.

But the infrastructure now needs a mechanism to distribute requests.

4,000 req/s
     │
     ▼
Load Balancer
     │
     ├── 1,000 → A
     ├── 1,000 → B
     ├── 1,000 → C
     └── 1,000 → D

The exact distribution will depend on workload and algorithm, but the principle remains the same.

Vertical Scaling vs Horizontal Scaling

Vertical Scaling

Vertical scaling increases the capacity of one machine.

4 CPU
8 GB RAM
   │
   ▼
16 CPU
64 GB RAM

Advantages:

• simple architecture;
• fewer machines;
• fewer distributed-system concerns.

Limitations:

• hardware ceilings;
• potentially expensive upgrades;
• large failure domain;
• maintenance can affect the entire application.

Horizontal Scaling

Horizontal scaling adds more machines or instances.

Application
     │
     ▼
Server A

becomes:

Application
     │
     ▼
Load Balancer
     │
 ┌───┼───┐
 ▼   ▼   ▼
 A   B   C

This architecture can provide both:

Capacity
+
Redundancy

But it introduces distributed-system concerns such as:

Session State
Health Checks
Synchronization
Deployment Coordination
Caching
Database Scaling
Observability

Load balancing is therefore only one part of horizontal scaling.

Basic Load Balancer Architecture

A basic HTTP architecture looks like:

                     Internet
                        │
                        ▼
                   Load Balancer
                        │
             ┌──────────┼──────────┐
             ▼          ▼          ▼
          App 1      App 2      App 3
             │          │          │
             └──────────┼──────────┘
                        ▼
                    Database

The application servers should ideally be interchangeable.

A request reaching:

App 1

should normally work just as well when sent to:

App 2

or:

App 3

This principle becomes important when discussing stateless applications.

Layer 4 vs Layer 7 Load Balancing

One of the most important distinctions is between Layer 4 and Layer 7 load balancing.

These names refer to the OSI model layers involved in routing decisions.

Layer 4 Load Balancing

Layer 4 operates primarily using transport-layer information.

The load balancer can make decisions using information such as:

Source IP
Destination IP
TCP Port
UDP Port
Protocol

Architecture:

Client
   │
   │ TCP Connection
   ▼
Layer 4 Load Balancer
   │
   ├── Backend A
   ├── Backend B
   └── Backend C

The load balancer does not necessarily need to understand application semantics such as:

/products
/api/users
Host: api.example.com
Cookie: session=...

This can make Layer 4 balancing efficient and applicable to many TCP-based services.

Layer 7 Load Balancing

Layer 7 operates at the application layer.

For HTTP traffic, it can inspect information such as:

Hostname
URL
Path
HTTP Method
Headers
Cookies
Query Parameters

Example:

                    Layer 7 LB
                         │
        ┌────────────────┼────────────────┐
        │                │                │
 /api/* │       /images/*│      /admin/*  │
        ▼                ▼                ▼
     API Pool         Media Pool       Admin Pool

This enables intelligent application routing.

AWS describes its Application Load Balancer as operating at Layer 7 and evaluating listener rules before selecting a target from the corresponding target group.

Layer 4 vs Layer 7

Feature	Layer 4	Layer 7
Main awareness	TCP/UDP connections	Application protocols
IP awareness	Yes	Yes
Port awareness	Yes	Yes
HTTP path routing	No	Yes
Hostname routing	Limited/not application-aware	Yes
Header routing	No	Yes
Cookie-aware routing	No	Possible
Protocol flexibility	High	Application-specific
HTTP intelligence	Low	High

Neither layer is universally better.

The correct choice depends on the application.

Reverse Proxy vs Load Balancer

A reverse proxy and a load balancer often appear together because one system can perform both roles.

A reverse proxy:

Client
   │
   ▼
Proxy
   │
   ▼
Backend

represents backend services to clients.

A load balancer:

Client
   │
   ▼
Load Balancer
   │
   ├── Backend A
   ├── Backend B
   └── Backend C

distributes workloads.

Nginx and HAProxy can provide both capabilities.

The distinction is architectural:

Reverse Proxy
→ Intermediates traffic

Load Balancer
→ Distributes traffic

For the proxy layer itself, see Reverse Proxy Explained.

Round Robin Load Balancing

Round Robin is one of the simplest balancing algorithms.

Requests rotate through available servers.

Request 1 → Server A
Request 2 → Server B
Request 3 → Server C
Request 4 → Server A
Request 5 → Server B
Request 6 → Server C

Nginx uses Round Robin as its default HTTP load-balancing method when another method is not specified.

Round Robin works particularly well when:

Servers have similar capacity
+
Requests have similar processing cost

The Limitation of Round Robin

Consider:

Request 1 → 20 ms
Request 2 → 8 seconds
Request 3 → 30 ms

Round Robin treats them simply as three requests.

But the second request occupies its server much longer.

Eventually:

Server A → 5 active requests
Server B → 100 active requests
Server C → 10 active requests

Round Robin may still send another request to Server B.

This is where connection-aware algorithms become useful.

Weighted Round Robin

Servers do not always have identical hardware.

Suppose:

Server A
16 CPU
32 GB RAM

Server B
8 CPU
16 GB RAM

A weighted strategy could assign:

Server A → weight 2
Server B → weight 1

Conceptually:

Request 1 → A
Request 2 → A
Request 3 → B
Request 4 → A
Request 5 → A
Request 6 → B

Nginx supports server weights when distributing requests through upstream groups.

Weighted balancing is useful during:

• heterogeneous server deployments;
• hardware upgrades;
• migrations;
• gradual capacity changes.

Least Connections

Least Connections sends a new request or connection toward the server currently handling fewer active connections.

Example:

Server A → 80 connections
Server B → 15 connections
Server C → 45 connections

Next connection:

→ Server B

Nginx’s least_conn method selects the server with the fewest active connections while accounting for server weights.

This can be useful when request durations vary significantly.

Round Robin vs Least Connections

Consider three requests:

Request A → 100 ms
Request B → 30 seconds
Request C → 200 ms

Round Robin focuses on request order.

Least Connections focuses on current connection load.

Therefore:

Short, similar requests
        │
        ▼
Round Robin can work well

while:

Long-lived / uneven connections
        │
        ▼
Least Connections can be more appropriate

Algorithm selection should follow workload behavior rather than popularity.

Hash-Based Load Balancing

Another approach uses a value to deterministically select a server.

For example:

hash(client IP)
      │
      ▼
Backend

The objective can be to send requests from the same client to the same server.

Nginx provides ip_hash, which uses the client address as the hashing key for backend selection.

Conceptually:

Client A → hash → Server 2
Client B → hash → Server 1
Client C → hash → Server 3
Client A → hash → Server 2

This can provide basic session persistence.

Session Persistence

Suppose a user logs in through Server A.

User
 │
 ▼
Server A
 │
 ▼
Session stored locally

The next request reaches Server B:

User
 │
 ▼
Server B
 │
 ▼
Session missing

The user may appear logged out.

One solution is session persistence:

User A → Server A
User A → Server A
User A → Server A

This is also called:

Sticky Sessions
Session Affinity

Why Sticky Sessions Can Become a Problem

Sticky sessions introduce dependency on a specific backend.

Suppose:

User A
   │
   ▼
Server A
   │
   X

If Server A fails, the session may disappear.

A more scalable application architecture stores session state externally:

                    Load Balancer
                         │
             ┌───────────┼───────────┐
             ▼           ▼           ▼
          Server A    Server B    Server C
             │           │           │
             └───────────┼───────────┘
                         ▼
                  Shared Session Store

Possible shared state systems include:

Redis
Database
Distributed Cache
External Identity Service

Now any server can process any request.

Stateless Applications

Stateless application servers are easier to scale horizontally.

Instead of:

Server A
  │
  └── User A session

use:

Application Servers
       │
       ▼
External State

The servers become interchangeable:

Request 1 → A
Request 2 → C
Request 3 → B

without breaking the application session.

This is one of the most important principles behind scalable application architecture.

Health Checks

Load balancing is not useful if traffic is sent to broken servers.

Suppose:

Server A ✓
Server B ✗
Server C ✓

A naive traffic distributor might continue:

Request → Server B → Failure

A health-aware load balancer instead removes the failed target:

Load Balancer
     │
     ├── Server A ✓
     ├── Server B ✗
     └── Server C ✓

and routes new requests only to healthy targets.

AWS Application Load Balancer performs health checks on registered targets and routes requests to healthy targets according to target-group health behavior.

TCP Health Checks

A simple health check can ask:

Can I establish a TCP connection?

For example:

Load Balancer
      │
      ▼
10.0.10.25:8080
      │
      ├── Connection succeeds → Healthy?
      └── Connection fails → Unhealthy

This verifies that something is listening.

But it does not prove the application is actually functional.

HTTP Health Checks

A better application check may request:

GET /health

and expect:

200 OK

Architecture:

Load Balancer
     │
     ▼
GET /health
     │
     ▼
Application
     │
     ├── 200 → Healthy
     └── Failure → Unhealthy

HAProxy supports active checks that periodically connect to a server or send an HTTP request; after configured failure thresholds are reached, the server can be removed from rotation and later restored after successful checks.

What Should a Health Endpoint Check?

A poor endpoint simply returns:

200 OK

whenever the process is running.

But the application may depend on:

Database
Redis
Message Queue
Storage
Critical Internal Service

This creates an architectural question:

Should /health fail when one dependency fails?

The answer depends on the dependency and the purpose of the check.

A useful distinction is between:

Liveness

and:

Readiness

Liveness asks:

Is the process alive?

Readiness asks:

Is this instance currently capable of serving traffic?

These are different questions.

Health Check Thresholds

A server should not necessarily be removed after one transient failure.

Production health checks often use thresholds.

Example:

Check every 10 seconds

3 failures
    │
    ▼
Mark unhealthy

2 successful checks
    │
    ▼
Return to service

This prevents minor transient problems from constantly adding and removing servers.

Passive Health Checks

A proxy can also learn from real production traffic.

For example:

Request → Backend
             │
             X Connection Refused

After repeated failures, the backend can temporarily be considered unavailable.

Nginx’s open-source HTTP load-balancing functionality includes passive/in-band health behavior through parameters such as max_fails and fail_timeout.

Active and passive checking can serve different purposes.

Failover

Load balancing improves capacity.

Failover improves availability.

Consider:

Server A ✓
Server B ✓
Server C ✓

then:

Server B ✗

A resilient architecture becomes:

Traffic
   │
   ▼
Load Balancer
   │
   ├── A ✓
   ├── B ✗
   └── C ✓

Users continue through A and C.

This is graceful degradation.

Nginx Load Balancing

Nginx can define a backend pool using an upstream block.

Example:

upstream application {
    server 10.0.10.11:3000;
    server 10.0.10.12:3000;
    server 10.0.10.13:3000;
}

server {
    listen 80;

    location / {
        proxy_pass http://application;
    }
}

The architecture becomes:

                 Nginx
                   │
        ┌──────────┼──────────┐
        ▼          ▼          ▼
     App 1      App 2      App 3

With no alternative balancing method specified, Nginx uses Round Robin.

Nginx Least Connections

Configuration:

upstream application {
    least_conn;

    server 10.0.10.11:3000;
    server 10.0.10.12:3000;
    server 10.0.10.13:3000;
}

Now backend selection considers active connection counts.

Nginx Weighted Servers

Example:

upstream application {
    server 10.0.10.11:3000 weight=3;
    server 10.0.10.12:3000 weight=1;
}

Conceptually:

Server A
   │
   └── Larger Share

Server B
   │
   └── Smaller Share

This can represent unequal server capacity.

Nginx IP Hash

Configuration:

upstream application {
    ip_hash;

    server 10.0.10.11:3000;
    server 10.0.10.12:3000;
    server 10.0.10.13:3000;
}

Requests from the same client address are normally mapped to the same backend while that backend remains available.

This can provide basic persistence, but external session storage is often a better long-term application design.

HAProxy Load Balancing

HAProxy is designed specifically around proxying and load balancing workloads.

A simplified configuration can look like:

frontend
     │
     ▼
backend
     │
 ┌───┼───┐
 ▼   ▼   ▼
S1  S2  S3

HAProxy supports HTTP and TCP proxying modes, allowing Layer 7 HTTP decisions as well as lower-level TCP load balancing.

A basic backend configuration might resemble:

backend web_servers
    mode http
    balance roundrobin

    server web1 10.0.10.11:80 check
    server web2 10.0.10.12:80 check
    server web3 10.0.10.13:80 check

HAProxy supports multiple balancing strategies, including Round Robin, Least Connections, hashing, and other algorithms.

Nginx vs HAProxy

Both can perform reverse proxying and load balancing, but they are often selected for different operational reasons.

Nginx combines:

Web Server
+
Reverse Proxy
+
Caching
+
Load Balancing

HAProxy has a particularly strong focus on:

Proxying
+
Load Balancing
+
Traffic Management
+
Health Checking

The correct choice depends on architecture rather than a universal winner.

For a simple web stack:

Nginx
   │
   ▼
Application

may be extremely convenient.

For dedicated traffic-management infrastructure, HAProxy may be attractive.

AWS Application Load Balancer

Cloud environments frequently provide managed load balancing.

On AWS, one important service is the Application Load Balancer.

Its conceptual architecture is:

                     Internet
                        │
                        ▼
                       ALB
                        │
                  Listener Rules
                        │
           ┌────────────┼────────────┐
           ▼            ▼            ▼
      Target Group A Target Group B Target Group C
           │            │            │
           ▼            ▼            ▼
        Targets       Targets      Targets

AWS ALB operates at Layer 7 and evaluates listener rules before routing traffic to targets.

AWS ALB Target Groups

A target group contains resources that can receive traffic.

Depending on architecture, targets can represent supported resources such as instances or IP addresses.

Conceptually:

Listener
   │
   ▼
Target Group
   │
   ├── Target A
   ├── Target B
   └── Target C

Health checks are configured at the target-group level, and the load balancer continually monitors registered targets.

Host-Based Routing

Suppose:

www.example.com
api.example.com
admin.example.com

share one load balancer.

Layer 7 routing can implement:

www.example.com
       │
       ▼
Frontend Pool

api.example.com
       │
       ▼
API Pool

admin.example.com
       │
       ▼
Admin Pool

This allows one entry point to serve multiple application components.

Path-Based Routing

The same hostname can route by URL path:

example.com/
     │
     ▼
Frontend

example.com/api/
     │
     ▼
API

example.com/images/
     │
     ▼
Media Service

This is extremely useful for microservices.

Load Balancer TLS Termination

A Layer 7 load balancer can often terminate HTTPS.

Architecture:

Client
   │ HTTPS
   ▼
Load Balancer
   │
   ├── TLS Certificate
   └── Routing
   │
   ▼
Backend

Backend traffic can then use:

HTTP

or:

HTTPS

depending on security requirements.

This centralizes certificate handling.

TLS Termination vs End-to-End TLS

TLS termination:

Client
   │ HTTPS
   ▼
Load Balancer
   │ HTTP
   ▼
Backend

Re-encryption:

Client
   │ HTTPS
   ▼
Load Balancer
   │ HTTPS
   ▼
Backend

The second model provides encryption across both network segments.

Whether that is necessary depends on:

Threat Model
Compliance
Network Trust
Cloud Architecture
Data Sensitivity

WebSockets

Long-lived connections create different balancing behavior from short HTTP requests.

Traditional:

Request
   ↓
Response
   ↓
Connection Ends

WebSocket:

Client
   ⇅
Persistent Connection
   ⇅
Server

If thousands of long-lived connections accumulate unevenly:

Server A → 500
Server B → 5,000
Server C → 800

request-count-based balancing alone may not represent actual load accurately.

This is one reason connection-aware strategies can matter for persistent protocols.

APIs and Microservices

Consider:

/api/users
/api/orders
/api/payments
/api/search

A Layer 7 load balancer can route each service independently:

                       Load Balancer
                             │
          ┌──────────────────┼──────────────────┐
          ▼                  ▼                  ▼
      User Service      Order Service      Search Service
          │                  │                  │
       Instances          Instances          Instances

Each target group can then scale independently.

This is one of the foundations of modern microservice architecture.

Docker Load Balancing

Containers make horizontal scaling easy:

docker-app-1
docker-app-2
docker-app-3

But clients still need a stable entry point.

Architecture:

Internet
   │
   ▼
Load Balancer
   │
   ├── Container 1
   ├── Container 2
   └── Container 3

The load balancer separates:

Stable Public Endpoint

from:

Dynamic Backend Instances

This distinction becomes even more important in orchestrated environments.

Kubernetes Load Balancing

Kubernetes adds several networking abstractions.

A simplified request path can be:

Internet
   │
   ▼
External Load Balancer
   │
   ▼
Ingress / Gateway
   │
   ▼
Service
   │
   ▼
Pods

The external load balancer handles entry into the cluster.

Internal Kubernetes mechanisms distribute traffic toward application workloads.

This architecture illustrates an important point:

Large systems may contain multiple load-balancing layers.

Multiple Load Balancers Are Not Automatically Wrong

Consider:

Global Load Balancer
        │
        ▼
Regional Load Balancer
        │
        ▼
Kubernetes Gateway
        │
        ▼
Service
        │
        ▼
Pods

Each layer may have a different responsibility.

The problem arises when several layers perform essentially the same job without a clear reason.

Every infrastructure component should answer:

What failure does this layer handle?

What routing decision does it make?

What visibility does it provide?

What unique capability does it add?

Database Load Balancing

Load balancing is not limited to web servers.

Database architectures can include:

Application
    │
    ▼
Database Proxy
    │
 ┌──┴────────────┐
 ▼               ▼
Primary        Replica

Read-heavy systems may route:

Writes → Primary
Reads  → Replicas

Conceptually:

                    Application
                         │
                         ▼
                   Database Layer
                         │
              ┌──────────┴──────────┐
              ▼                     ▼
            Writes                 Reads
              │                     │
              ▼             ┌───────┼───────┐
           Primary          ▼       ▼       ▼
                         Replica Replica Replica

Database consistency requirements make this considerably more complicated than ordinary HTTP balancing.

For cloud database architecture, see Cloud Databases Explained.

Global Load Balancing

Traditional load balancing may distribute traffic within one region:

Region A
   │
   ▼
Load Balancer
   │
 ┌─┼─┐
 ▼ ▼ ▼
 A B C

Global load balancing selects between regions or data centers:

                       Global Users
                            │
                            ▼
                    Global Traffic Layer
                            │
              ┌─────────────┼─────────────┐
              ▼             ▼             ▼
           Europe          Asia         America
             │              │              │
             ▼              ▼              ▼
         Regional LB    Regional LB    Regional LB

Selection can consider:

Geography
Latency
Health
Policy
Capacity

HAProxy’s GSLB documentation, for example, describes DNS-based routing that can consider geographic location and data-center health.

DNS-Based Global Load Balancing

One method operates through DNS.

Normally:

User
 │
 ▼
DNS
 │
 ▼
IP Address

A global routing system can instead choose among several regional addresses:

User in Europe
      │
      ▼
DNS
      │
      ▼
European Endpoint

while:

User in Asia
      │
      ▼
DNS
      │
      ▼
Asian Endpoint

DNS caching must be considered because clients or resolvers can continue using previously returned records until TTL expiration.

CDN vs Load Balancer

These technologies solve different problems.

A CDN primarily focuses on:

Content Distribution
Edge Caching
Geographic Delivery

A load balancer primarily focuses on:

Backend Distribution
Availability
Traffic Routing

A common architecture uses both:

User
 │
 ▼
CDN
 │
 ▼
Load Balancer
 │
 ├── Server A
 ├── Server B
 └── Server C

Cache HIT:

User → CDN → Response

Cache MISS:

User
 ↓
CDN
 ↓
Load Balancer
 ↓
Application

For detailed cache architecture, see CDN Caching Explained.

Edge + CDN + Load Balancer

A more complete architecture is:

                         Global Users
                              │
                              ▼
                         Edge Network
                              │
                              ▼
                           CDN Cache
                              │
                        Cache MISS
                              │
                              ▼
                         Load Balancer
                              │
                ┌─────────────┼─────────────┐
                ▼             ▼             ▼
             App A          App B          App C

The edge reduces latency.

The cache reduces requests.

The load balancer distributes remaining requests.

The application processes business logic.

These are complementary layers.

For the first layer, see Edge Computing Explained.

Load Balancing and Auto Scaling

Load balancing becomes even more powerful when combined with automatic scaling.

Suppose traffic increases:

2 Servers
   │
   ▼
High CPU

An autoscaling system adds instances:

Server A
Server B
Server C
Server D

The load balancer begins distributing traffic to the new healthy instances.

When demand falls:

4 Servers
   │
   ▼
Low Demand
   │
   ▼
2 Servers

This creates dynamic infrastructure:

Traffic
   │
   ▼
Load Balancer
   │
   ▼
Elastic Backend Pool

Load Balancing and Infrastructure as Code

Production load balancers should ideally be reproducible.

Instead of manually creating:

Listeners
Target Groups
Health Checks
Rules
Security Policies

through dashboards, infrastructure can be defined in code.

For example:

Terraform
   │
   ▼
Load Balancer
   │
   ├── Listener
   ├── Target Group
   ├── Health Check
   └── Routing Rules

This provides:

Version Control
Review
Repeatability
Automation
Disaster Recovery

For the broader workflow, see Infrastructure as Code Explained.

High Availability

A load balancer is supposed to protect applications from backend failures.

But what happens if the load balancer itself fails?

Bad architecture:

Internet
   │
   ▼
Single Load Balancer
   │
   X
   │
Backend Cluster

The application servers remain healthy but unreachable.

A highly available design needs redundancy at the load-balancing layer.

Conceptually:

                    Traffic Entry
                         │
               ┌─────────┴─────────┐
               ▼                   ▼
           LB Node A           LB Node B
               │                   │
               └─────────┬─────────┘
                         ▼
                    Backend Pool

Managed cloud load balancers typically abstract much of this infrastructure from the application operator.

Self-managed systems require explicit HA design.

Multi-AZ Architecture

In cloud infrastructure, servers should often be distributed across independent failure domains.

Example:

                      Load Balancer
                           │
               ┌───────────┴───────────┐
               ▼                       ▼
        Availability Zone A     Availability Zone B
               │                       │
           ┌───┴───┐               ┌───┴───┐
           ▼       ▼               ▼       ▼
         App A   App B           App C   App D

If one availability zone fails:

AZ A ✗

traffic can continue through:

AZ B ✓

assuming the rest of the application—including databases and dependencies—is also designed for that failure.

The Database Can Still Be a Single Point of Failure

Consider:

                    Load Balancer
                         │
             ┌───────────┼───────────┐
             ▼           ▼           ▼
           App A       App B       App C
             │           │           │
             └───────────┼───────────┘
                         ▼
                  Single Database
                         │
                         X

The web tier is highly available.

The application is not.

True availability requires examining the complete dependency graph:

DNS
CDN
Load Balancer
Applications
Database
Cache
Queue
Storage
External APIs

Any critical single dependency can become the actual failure point.

Observability

A load balancer provides an excellent view of application traffic.

Useful metrics include:

Requests per Second
Active Connections
New Connections
HTTP 2xx
HTTP 3xx
HTTP 4xx
HTTP 5xx
Backend Response Time
Healthy Targets
Unhealthy Targets
TLS Errors
Rejected Connections

These metrics can reveal problems before server-level monitoring does.

Backend Response Time

Suppose:

Load Balancer Processing = 2 ms

Backend Response = 3 seconds

The load balancer is not the bottleneck.

Monitoring should distinguish:

Load Balancer Latency

from:

Application Latency

and:

Database Latency

Otherwise engineers may optimize the wrong layer.

Monitor Target Health

A production dashboard should immediately expose:

Healthy Targets: 12
Unhealthy Targets: 0

If that suddenly becomes:

Healthy Targets: 3
Unhealthy Targets: 9

the application may still be online, but its redundancy has largely disappeared.

Availability monitoring should detect degradation before total outage.

Troubleshooting Load Balancing

Use a layered approach.

1. DNS
2. Client → Load Balancer
3. Listener
4. Routing Rule
5. Target Group
6. Health Check
7. Network Path
8. Backend Service
9. Application
10. Database

Do not immediately blame the load balancer.

Step 1: Verify DNS

Use:

dig example.com

or:

dig +short example.com

Verify that the hostname points to the expected traffic endpoint.

For DNS fundamentals, see DNS Explained.

Step 2: Test HTTP

Use:

curl -I https://example.com

For more information:

curl -v https://example.com

This can reveal:

TLS Errors
Redirects
HTTP Status
Headers
Connection Failures

Step 3: Test the Backend Directly

From an authorized internal system:

curl http://10.0.10.11:3000/health

If:

Load Balancer → Backend

fails while:

Direct Test → Backend

works, investigate:

Routing
Firewall
Security Groups
Proxy Configuration
Health Check Configuration

If the direct test also fails, the problem is probably deeper in the backend stack.

Step 4: Check Listening Ports

On Linux:

ss -lntp

Confirm that the application listens on the expected:

Address
Port
Protocol

A common mistake is:

Application listening:
127.0.0.1:3000

while the load balancer attempts to reach:

10.0.10.11:3000

The process exists but is not reachable through the expected interface.

Step 5: Check Firewall Rules

The network path may be:

Load Balancer
     │
     ▼
Firewall
     X
     │
Backend

Only the required load-balancer-to-backend traffic should be permitted.

For firewall concepts, see Firewalls Explained.

Step 6: Check Health Check Path

Suppose the load balancer checks:

/health

but the application exposes:

/healthz

The application can be fully operational while every target appears unhealthy.

Likewise, if /health redirects unexpectedly or requires authentication, the health-check behavior may not match expectations.

Always test the exact:

Protocol
Port
Path
Expected Status

used by the load balancer.

Step 7: Check Application Logs

If traffic reaches the backend but responses fail, inspect:

Application Logs
Web Server Logs
Database Logs

A 5xx response generated by the application is different from a connection failure between the load balancer and the application.

Uneven Traffic Distribution

Suppose:

Server A → 80%
Server B → 10%
Server C → 10%

Potential causes include:

Sticky Sessions
Server Weights
Long-Lived Connections
Hashing
Unequal Health
Different Backend Capacity
Connection Reuse
Traffic Sample Size

Do not assume the balancing algorithm is broken.

First understand its inputs.

Load Balancer Security

A load balancer is part of the application’s public attack surface.

Security considerations include:

TLS Configuration
Listener Exposure
Security Groups / Firewall
Header Handling
Request Limits
Authentication Integration
Logging
Patch Management
DDoS Protection

Backend servers should generally accept traffic only from authorized infrastructure where practical.

Architecture:

Internet
   │
   ▼
Load Balancer
   │
   ▼
Firewall Rule
   │
   ▼
Backend

not:

Internet ─────────────► Backend

unless direct exposure is intentionally required.

Forwarded Client Information

Like reverse proxies, load balancers can change the immediate network source observed by applications.

Applications may therefore receive forwarded metadata containing the original client address and protocol.

This information must be trusted only when supplied by known infrastructure.

Otherwise an attacker could attempt to submit spoofed forwarding headers directly.

Load Balancing Anti-Patterns

One Load Balancer, One Backend

LB
 │
 ▼
Server A

This can still provide useful routing or TLS capabilities, but it does not provide backend redundancy.

Multiple Servers with Local Session State

User
 ↓
A → Session exists

Next request
 ↓
B → Session missing

Use intentional session persistence or externalize state.

Health Check Only Tests the Port

TCP connection works

does not necessarily mean:

Application works

Choose a health check appropriate to the failure you need to detect.

All Servers in One Failure Domain

Load Balancer
      │
      ▼
All servers
      │
      ▼
Same physical host

provides little protection from host failure.

Blindly Using Round Robin

Round Robin is excellent for many workloads, but not every workload.

Long-running or highly variable requests may require another strategy.

Increasing Timeouts Instead of Fixing Latency

If:

Backend takes 90 seconds

changing:

timeout 60 → 120 seconds

may merely hide a performance problem.

Measure the backend first.

Production Load Balancing Architecture

A mature architecture can look like:

                         Global Users
                              │
                              ▼
                            DNS
                              │
                              ▼
                       CDN / Edge Layer
                              │
                              ▼
                       Load Balancer Tier
                              │
                 ┌────────────┴────────────┐
                 ▼                         ▼
          Availability Zone A      Availability Zone B
                 │                         │
            ┌────┴────┐               ┌────┴────┐
            ▼         ▼               ▼         ▼
          App A     App B           App C     App D
            │         │               │         │
            └─────────┴───────┬───────┴─────────┘
                              ▼
                        Shared Services
                              │
               ┌──────────────┼──────────────┐
               ▼              ▼              ▼
             Cache          Database        Queue

Autoscaling can modify the backend pool:

Traffic ↑
   │
   ▼
Add Instances
   │
   ▼
Health Checks Pass
   │
   ▼
Load Balancer Adds Traffic

Deployments can happen gradually:

Old Version
   │
   ├── 90%
   │
New Version
   └── 10%

Observability monitors the complete system:

Metrics
+
Logs
+
Traces
+
Health Checks
+
Alerts

This is no longer merely traffic distribution.

It is an availability architecture.

Load Balancing Design Checklist

Before deploying a production load balancer, answer:

Traffic

• Is the workload HTTP, HTTPS, TCP, UDP, gRPC, or WebSocket?
• Do you need Layer 4 or Layer 7 routing?
• What is expected peak traffic?

Backends

• Are instances interchangeable?
• Are they stateless?
• Do they have equal capacity?
• Are they spread across failure domains?

Algorithm

• Round Robin?
• Weighted?
• Least Connections?
• Hash-based?
• Application-specific routing?

Health

• What does healthy actually mean?
• What path is checked?
• What failure threshold is appropriate?
• What recovery threshold is appropriate?

Sessions

• Is session persistence necessary?
• Can state move to Redis or another shared service?

Security

• Is TLS terminated at the load balancer?
• Are backend connections encrypted?
• Can backends be reached directly from the internet?
• Are forwarded headers trusted safely?

Availability

• Is the load-balancing tier itself redundant?
• What happens when one availability zone fails?
• What happens when the database fails?

Observability

• Are target-health changes monitored?
• Are 5xx responses monitored?
• Is backend latency visible?
• Are connection counts visible?

Automation

• Is configuration version controlled?
• Can the load balancer be recreated through Infrastructure as Code?

Frequently Asked Questions

What is load balancing?

Load balancing distributes incoming network connections or application requests among multiple backend resources to improve scalability, resource utilization, and availability.

What is Layer 4 load balancing?

Layer 4 balancing makes traffic-distribution decisions primarily using transport and network information such as IP addresses, TCP/UDP protocols, and ports.

What is Layer 7 load balancing?

Layer 7 balancing understands application-layer protocols and can make routing decisions using HTTP information such as hostnames, URL paths, headers, and other request properties.

What is Round Robin load balancing?

Round Robin sends requests across available servers sequentially. Nginx uses Round Robin as its default HTTP balancing method when another method is not configured.

What is Least Connections?

Least Connections selects the backend currently handling fewer active connections, accounting for weights where supported.

What is weighted load balancing?

Weighted load balancing assigns different traffic shares to servers according to configured weights, allowing higher-capacity systems to receive more traffic.

What are sticky sessions?

Sticky sessions attempt to keep requests from a particular client associated with the same backend server.

Are sticky sessions required?

No. Stateless applications with shared external session storage often scale more cleanly without strict server affinity.

What is a health check?

A health check determines whether a backend is currently capable of receiving traffic. Checks can operate at different levels, including TCP connectivity or application-aware HTTP requests.

What happens when a server fails?

A health-aware load balancer can remove an unhealthy server from normal traffic rotation and continue using healthy servers.

Is Nginx a load balancer?

Yes. Nginx can distribute HTTP traffic among multiple upstream servers using algorithms including Round Robin, Least Connections, and IP Hash.

Is HAProxy a load balancer?

Yes. HAProxy supports load balancing and proxying across HTTP and TCP workloads, with multiple algorithms and health-check capabilities.

What is AWS Application Load Balancer?

AWS Application Load Balancer is a managed Layer 7 load balancer that evaluates listener rules and routes requests to targets through target groups.

Load balancer or reverse proxy?

The roles overlap. Reverse proxy describes intermediary server behavior, while load balancing specifically describes traffic distribution among multiple destinations. Products such as Nginx and HAProxy can perform both.

CDN or load balancer?

They solve different problems. A CDN primarily improves distributed content delivery and caching, while a load balancer distributes traffic across backend infrastructure. Production applications frequently use both.

Final Thoughts

Load balancing starts with a simple problem:

One Server
+
Too Much Traffic

and introduces:

Multiple Servers
+
Traffic Distribution

But production load balancing quickly becomes much more sophisticated.

The load balancer must understand:

Where should this request go?

Is that server healthy?

Is another server less busy?

Does this user require session affinity?

Does the hostname select another application?

Does the URL path select another microservice?

Should TLS terminate here?

What happens when an availability zone fails?

This transforms the architecture from:

Internet
   │
   ▼
Server

into:

                         Internet
                            │
                            ▼
                           DNS
                            │
                            ▼
                       CDN / Edge
                            │
                            ▼
                      Load Balancer
                            │
               ┌────────────┼────────────┐
               ▼            ▼            ▼
             App A        App B        App C
               │            │            │
               └────────────┼────────────┘
                            ▼
                        Data Layer

The most important principle is not to select the most complicated balancing algorithm.

It is to select an algorithm and architecture that match the workload.

Round Robin can be excellent for homogeneous short-lived requests.

Least Connections can help with uneven or long-lived connections.

Weighted balancing can account for unequal server capacity.

Hash-based routing can provide affinity.

Health checks prevent failed systems from receiving normal traffic.

Layer 7 routing can separate applications and microservices.

Global load balancing can distribute users across entire regions.

And when load balancing is combined with CDN caching, reverse proxies, autoscaling, distributed databases, and Infrastructure as Code, it becomes one of the foundations of resilient cloud architecture.

Continue with Reverse Proxy Explained for the proxy layer immediately surrounding application servers.

Then read CDN Caching Explained to understand how requests can be eliminated before they ever reach the load balancer.

For globally distributed execution, continue with Edge Computing Explained.

For the underlying network, see Cloud Networking Explained.

And for reproducible deployment, continue with Infrastructure as Code Explained.

The CorgiNetwork infrastructure learning path now becomes:

DNS → Edge Computing → CDN Caching → Reverse Proxy → Load Balancing → Auto Scaling → Application Servers → Databases → Cloud Networking → Infrastructure as Code.