Planning: Scalable Hosting Servers

An Introduction to Scalable Servers
Why do you need to concern yourself with scalability and scalable servers in particular ?

There are many examples of web sites that were established as some form of joke, or for sensible reasons, where the owners did not expect to see a lot of traffic (visitors) initially. http://www.amihotornot.com/ is one such site, at the other end of the spectrum http://www.thedrudgereport.com/ is another. Both of these sites, for different reasons, went from almost no visitors to thousands, hundreds of thousands and millions almost instantly, even in Internet terms.

As a consequence, the resources provisioned for the web sites were not adequate to the task and they needed to be quickly scaled up to meet the demand.

This can be done in a couple of ways. The simplest is to first make certain that the server running your web site is only running your web site, that is, it is a dedicated host. Next step up is to speed up the server itself with a faster server and, the last level up is to distribute the load to multiple servers. In the end, you may have multiple servers serving ages to browsers and multiple database servers servicing queries from the page servers. This is called load balancing.

The Ideal Scalable Website Architecture
Building a scalable web site can be a very tricky proposition. It might involve a lot of redundancy, load balancing, multiple web servers, a separate database server, backup servers and server clustering or blade servers. In the figure below, all connections to the Internet first go through a system whose sole responsibility is to perform load balancing.

The load balancing server simply determines which 'real' server is the least busy and distributes the next request to that server. The rules used to make this determination are normally configurable by the system administrator. Multiple servers are used to help keep the system fault-tolerant, or provide failover where, in the event one server crashes the traffic can be diverted to the another. The same applies to the database servers. Database server redundancy is a little more complex than web server redundancy as data posted to / updated on one server has to be updated on the backup servers immediately so that concurrency is not lost. This is especially complex in a transaction processing environment where it is critical that all aspects of a transaction complete or none do.

Scalable Web Hosting

Scalable Architecture On A Budget
Well, scalable is really all about doing it on a tight budget. It means that at any one time you do not have to invest a large amount of money in servers. This is, in particular, what the new blade server technology from HP, Sun and Cobalt is all about. It is, in fact, having multiple cheap (?) computers in a single rack. Scaling up to meet web traffic requirements then simply means adding a new blade, which usually comes with a disk drive, and telling the management software to start using the new resource.

Blade servers are not that cheap, so lets look at some other structure.

Master and Slave Sites
One way is to establish a master site, your primary location, and slave sites, a secondary location. This is often done by large companies, eBay and Yahoo! come to mind om a geographical basis (ebay.com, ebay.ca, ebay.co.uk). This method separates both traffic and backend resources for each geographical region making the overall system much easier to manage and much less succeptible to a complete system failure. However, it also makes database integration more complex and slower.

Sitepoint.com, which implements a similar strategy, might serve one page of an article from a server in Sydney, Australia and a second from a server in Detroit. You'll never know from whence the page came, nor does it particularly matter.

The easiest way to send users to a distributed server is to use a round robin DNS server. A round robin setup would send each request to a random server. However, because each request goes to a different server, it is hard to track session data (such as the user's login details). To counter this, you can use cookies or hidden HTML fields. This also does not provide for statistical evaluation of traffic demands — it is usually faster to serve a page to a visitor from a server that is geographically closer to the user.

One Server Per Visit
Another method of load balancing is to track a connection from a visitor and, once the visitor is connected to a random server, keep that visitor connected to that server for the duration of their visit.

This is very simple to implement but can easily result in a single server becoming overloaded and, should that server go down, the visitor locked to that server won't be able to receive pages even though there are redundant servers in other locations.

Serve Content By Type
Another way to distribute load is to distribute content. Placing all static pages on one server, dynamic pages on another, graphics, sound and video on another multimedia server and database on yet another server. This works well with cheap, slow servers and high bandwidth, particularly for the graphics server. However, you could run into problems if one of the servers goes down as there is no built in redudancy and you have increased the risk of bringing the site down as you now have 5 links in the page delivery chain instead of one.

Caching
Caching is the process of sending already constructed pages to another server. Inktomi is a caching server company. Their servers go out and look for documents to cache. If the cacheing server is close to the user, the user will not be using your bandwidth / resources directly, but those of the cacheing server. There are a number of firewall and other packages, like squid, that can be setup by companies that will cache your pages within that company.

This is a very practical solution for sites that do not have a lot of dynamic content and are not overly database dependent as both will destroy the usefulness of the cache as the cache would not be current.

The Site Design
From the above you should have developed an understanding that structuring your site will have a significant impact on your ability to scale up, should visitor traffic demand you do so. Here are some hints on structure:

  • Keep your site divided into logical sections. For example, keep static pages in one directory, dynamic pages in another, CGI scripts separate, images separate again. This will faciliate moving different areas of your web site to different servers should that become desirable.
  • Limit the session persistence of pages which will allow you to use DNS round robin server allocation.
  • Don't hardcode links your pages on your site. make them relative and use variables if at all possible. This way, you can easily update your site if you change its location. For example, links can be stored in a file and referenced by inclusion in the html documents as (i.e. Click here )
  • Use caching wherever possible.
  • Try to avoid server side sessions. If you have to implement server side sessions try to set them up so that either the data can be shared with the other servers or the user uses only one server per session.

rsync, a Unix and Linux tool, is one of a number that can be used to synchronize servers. CVS can also be used.

Converting a Site
If you have already developed your site and not taken any of the above into consideration, don't panic. It's easy enough to implement each of the suggestions in turn, over time.

If your site is completely made of static HTML pages (i.e. no scripts, no changing data, no database accesses), then you don't have a lot to change. All you need is to copy the web site to at least two different servers and setup a round robin DNS. If your site is almost completely static you have some dynamic content, you can start by placing all static content on one server and all dynamic content on another.

If, on the other hand, you have a complex session driven, user oriented site then you will have to do a bit more work. Start by looking at all the database access code to assess whether you can separate code that updates the database from code that simply retrieves information from the database. If you can you can easily put pages that only interrogate the database on one server and pages that update the database on another (probably the one running the database).

There are many other options, like server clustering where all the servers look like a single server.

Conclusion
If your web site starts to grew like a weed, be grateful. This is how you make money. Or at least converting those visitors into cash. The cash you generate will help you solve your web server problems.