Load Balancer
Load balancer
A load balancer sits in front of a group of web servers and evenly distributes incoming traffic among them. The group it distributes to is called the load-balanced set.
How the traffic flows
Clients no longer connect to a web server directly. Instead:
- Clients connect to the public IP of the load balancer.
- The load balancer picks a web server from the pool and forwards the request to it over a private IP.
- The web server responds back through the load balancer to the client.
Because the web servers are now only reached through the load balancer, they no longer need public IPs at all. A private IP is an address that is only routable between servers inside the same network and cannot be reached from the internet. Moving the web servers onto private IPs is better for security: an attacker on the internet cannot address them directly, only the load balancer is exposed.
What this buys the web tier
Adding a load balancer and a second web server fixes the single point of failure in the web tier and improves its availability.
- Failover. If server 1 goes offline, the load balancer notices and routes all traffic to server 2, so the site stays up. A fresh healthy server can then be added back into the pool to carry its share of the load.
- Easy scaling. If traffic grows and two servers are no longer enough, you just add more servers to the pool. The load balancer automatically starts sending requests to the new servers, with no change on the client side.
What is still fragile
The web tier now looks solid, but the data tier does not. The design still has a single database, so it has no failover and no redundancy: if that one database fails, the whole application loses its data layer. The common fix is database replication, which is the next post.
But what if the load balancer itself fails?
A sharp reader will notice we just moved the single point of failure rather than removing it. All traffic now flows through the load balancer, so if it dies, the whole site is unreachable even though every web server is healthy.
In practice a load balancer is not run as one box. You deploy at least two, a primary and a secondary, that watch each other with a heartbeat. If the primary stops responding, the secondary takes over its IP address and keeps serving, usually within a few seconds. Managed load balancers from cloud providers (AWS ELB, GCP Cloud Load Balancing, and similar) do this for you: what looks like a single endpoint is a redundant, self-healing fleet behind the scenes. So the load balancer is a single point in the diagram, but not a single point of failure in a real deployment.