Scalable Internet Architectures

By Theo Schlossnagle

The language is mostly irrelevant. I can’t count the number of times I’ve heard people say “Java doesn’t scale” or “PHP doesn’t scale”. No language scales; solutions scale.

The dot com era taught us something that no businessperson would have ever believed before. It taught us that it is possible to take a concept at 8 a .m.; translate that into a business initiative; proceed through design, implementation, testing, and launch; and have millions of customers for that idea by the close of business. That fact that this is possible means that if the business can capitalize on such efficiencies, it will.

A push is a controlled and planned action that changes the production architecture. … A successful push plan has four parts: a plan to get from A to B, a plan to get from B to A, a plan to restore A from bare metal, and a successful test of the first two. (Testing a bare metal restore for every push would be suicide, or at least leave you constantly contemplating it.)

Ch 3: Mission-Critical Environments

The only way to increase the performance of a complex system is to reduce the resource consumption of one or more of its individual components. Contrapositively, if an individual component of a complex system performs slowly, it is likely to capsize the entire architecture.

Do not choose the slowest architectural component or piece of code to focus on. Start with the most common execution path and evaluate its impact on the system. The thing to keep in mind is that a 50% speedup of code that executes 0.1% of the time results in an overall speedup of 0.05%, which is small. On the other hand, a 50% speedup of code that executes 5% of the time results in an overall speedup of 2.5%, which is significant.

Ch 5: Load Balancing and the Utter Confusion Surrounding It

It is important to understand the difference between translation and proxying. When translating, the packets are simply manipulated (which can be done on an immense scale). Proxying requires processing all the data from the client and the real servers to which traffic is being sent, which includes reforming and originating all needed packets — that is substantially more expensive.

Serving multiple SSL services over a single IP address is only possible using an application-level load balancer and terminating SSL at the load balancer. Otherwise, a single SSL service requires a single IP address as the client and server participate in a strong cryptographic handshake that occurs before HTTP traffic is sent.

Capacity Planning Rule of Thumb

Expect to achieve an average of 70% utilization on each server in clusters with three or more nodes. Although better utilization is possible, be safe and bet on 70%.

Ch 6: Static Content Serving for Speed and Glory

… we conjectured about how we should determine whether a user is geographically closer to one location or another. Somehow, that “geographical” qualifier slipped in there, likely because people prefer to think spatially. The truth of the matter is that you don’t care where in the world (literally) the user is, just where on the Internet the user is. The proximity we should be attempting to capitalize on is the user’s proximity on the network.

Ch 8: Distributed Databases Are Easy, Just Read the Fine Print

Multi-master replication (master-master replication is a two-node special case)

Two-phase commit… The basic idea is that the node at tempting the transaction will notify its peers that it is about to commit, and they will react by preparing the transaction and notifying the originating node that they are ready to commit. The second phase is noticing any aborts in the process and then possibly following through on the commit (hopefully everywhere),

Master-slave replication

There are two common methods to replicate a master database to a set of slaves based on the preceding transaction flow. The first is to replicate the operation — that is, distribute the statement that resulted in the change to all the slave nodes and allow them to plan and then execute the operation to (hopefully) arrive at the same result. The second technique consists of tracking the actual data changes (called a changeset) and distributing the data that has changed to each slave. We will call the first DML replication, and the second DML log replication because the first replicates the actual data modification language, whereas the second replicates the log of what happened due to the data modification language.