Horizontal vs Vertical Scaling

Overview

Vertical scaling (scale up) gives one machine more CPU, memory or disk. Horizontal scaling (scale out) adds machines and spreads work among them. Most architectures use both, at different tiers.

How it works

Vertical:    [ 4 vCPU ]  ->  [ 32 vCPU ]            one bigger node
Horizontal:  [ node ]    ->  [ node ][ node ][ node ]   behind a load balancer

Horizontal scaling works when requests are independent. Any instance can serve any request if state lives elsewhere: sessions in Redis, files in object storage, data in a database. The load balancer then distributes by round robin or least connections.

Data is the hard part. Reads scale out through replicas, writes through partitioning, and both need a strategy for routing and rebalancing. Until that is necessary, a single large database with good indexes is often the cheapest and most reliable choice. Modern instances offer hundreds of cores and terabytes of RAM, which postpones distribution for many products.

Amdahl's law applies: if 10 percent of work is serial, no number of nodes gives more than a 10x speedup. Find the serial parts first, such as locks, a single queue or one database primary.

Pitfalls

• Scaling out a service whose bottleneck is a shared dependency, only moving the queue.
• Hidden state: in-memory caches, local uploads and sticky sessions.
• Waiting until the vertical ceiling to plan the horizontal design.
• Assuming linear gains. Coordination overhead reduces returns.

Key takeaways

• Scale up for simplicity and stateful cores; scale out for stateless tiers and availability.
• Remove local state before adding instances.
• Locate the true bottleneck before choosing.
• Begin the scale-out design well before the vertical limit.