A Look Behind the Scenes at Social Networking Sites
One thing that stands out with the majority of the social networking sites is that they use open source software--Twitter, LinkedIn and MySpace use Linux, while Facebook uses F5 Big-IP, a family of Linux-based network management appliances. Most of them also use the Apache Web server, by itself or along with other web servers, (LinkedIn also uses Sun ONE Web servers). Another similarity is the use of Sun's MySQL database management system for storing and organizing user messages, status and profile updates.
The social networks only allow us to know so much about the inner workings of their infrastructure. We know they run on LAMP stacks (Linux, Apache, MySQL, PHP/Python/Perl), and we know what their APIs tell us, because, after all, they encourage developers to write applications which interact with their networks and add functionality to the social networking sites.
Twitter has been open about its use of Ruby on Rails as its foundation and Jabber/XMPP (Extensible Messaging and Presence Protocol) instant messaging. These are both open source technologies. This means that open source products keep the social networks up and running. Facebook, which serves "260 billion page views per month" according to Royal Pingdom, a site that tracks Internet sites uptime, with only only 7.2 hours of downtime in 2008, and that's with more than 35 million users supposedly updating their status at least once each day. Twitter was down for 84 hours that same year, with LinkedIn down for 45.8 hours.
Back to open source, another similarity of most social networks is their use of Memcached, an open source, high-performance, distributed memory object caching system. Memcached is used to handle the millions of database reads and writes that occur on a daily basis throughout the networks. Twitter created their own add-on to Memcached called Starling, light-weight persistent queue server which deal with its users' Memcached requests. Facebook dealt with it another way by reworking Linux and Memcached's networking restrictions so it is now able to handle 200,000 UDP requests per second, whereas the stock Linux kernel can only handle 50,000 UDP requests per second.
The hardware side of social networking sites is just as interesting and just as hardcore. A Facebook server typically has two or more high-speed 64-bit Intel Nehalem processors with 4 or 8 cores and as much high-speed memory as possible. These servers are connected using equipment that can handle up to 5 terabytes per second with multiple gigabit and 10-gigabit Ethernet connections. All the data itself is contained on many types of storage servers, which themselves are located within server farms in data centers that are co-located.
The backend of the social networking sites is a complex maze of hardware, software, services, data and blood, sweat and tears. If you are interested in running your own social networking site, there are many options available to you, from free open source software to large scale commercial applications to SaaS (Software as a Service) enterprise applications. And this is just the tip of the iceberg...