Introduction

Scaling an application involves maintaining performance as load increases. The long-term solution for increasing capacity is to architect for horizontal scalability. In this section, we’ll explore scaling a simple CRUD web application called Cruder, consisting of a single-page JavaScript application communicating with an application server via a RESTful HTTP API. The server uses local disk for large files and a relational database for state, all hosted on a compute platform like AWS EC2.

Users interact with Cruder through browsers, issuing DNS requests to resolve domain names, opening TLS connections, and sending HTTP GET requests.

While this architecture suffices for a proof of concept, it lacks scalability and fault tolerance. However, as you’re learning about building such applications, we’ll assume eventual need for serving millions of requests per second.

The quickest way to increase capacity is to scale up the hosting machine, e.g., by adding processors, disks, NICs, SSDs, or memory. Alternatively, scaling out distributes the application across multiple nodes, increasing complexity but offering long-term benefits. For instance, moving the database to a dedicated machine can increase both server and database capacity.

Chapter 14 – HTTP Caching

Cruder’s server handles static and dynamic resources. Static resources, like JavaScript or CSS files, can be cached to reduce load and response time. Using server-side HTTP cache with a reverse proxy can further improve caching efficiency.

Chapter 15 – Content Delivery Networks

Chapter 16 – Partitioning

As an application’s data grows, it needs to be split into partitions or shards. This not only accommodates large data volumes but also increases request handling capacity by distributing the load across more nodes. A gateway service, like a reverse proxy, routes requests to the appropriate nodes based on data mapping.

Chapter 17 – File Storage

Managed file stores offer scalability, high availability, and durability. Files uploaded to such stores can be accessed via URLs, allowing CDNs to serve them directly, offloading storage and serving responsibilities.

Chapter 18 – Network Load Balancing

To handle increasing requests, multiple application servers can be deployed behind a load balancer. This scalability pattern, known as scaling out or horizontally, increases overall capacity.

Chapter 19 – Data Storage

With increasing load, the relational database will face capacity limits. Scaling out by running multiple application servers behind a load balancer can help, but as the database load grows, more advanced scaling strategies may be needed.

Chapter 20 – Caching

Introducing a cache for frequently accessed data can improve performance and reduce load on the data store. However, the effectiveness of caching depends on factors like the hit ratio and cache size.

Chapter 21 – Microservices

Functionally decomposing a monolithic application into independently deployable services can mitigate complexity and facilitate scalability. This architectural style, known as microservices, emphasizes small API surface areas encapsulating significant functionality.

Chapter 22 – Control Planes and Data Planes

Chapter 23 – Messaging

Summary

Building scalable applications boils down to exploiting three orthogonal patterns:

• Breaking the application into separate services, each with its own well-defined responsibility (functional decomposition).
• Splitting data into partitions and distributing them across nodes (partitioning).
• Replicating functionality or data across nodes (replication).

There is a subset of managed services that can be used to build a large number of applications. Managed services are attractive because someone else handles their operation.