Sometimes it’s easy to forget that “this” – the Internet – does exist in the physical world too. Millions of servers around the world.
Here’s a picture from one of the Stardock data centers. This rack contains one of the Impulse cluster manager servers and just inches below is Neowin.net’s servers.
The Neowin requires 5 very high end servers to run the site that is almost purely database/text is a testament to how popular that site has become over the years.
I’m not much of an IT expert but my understanding is that cluster managers are basically a series of virtual machines that help manage a raid drive unit for storing a piece. You typically have several of these cluster managers (they’d kill me if I said how many there were total) to help distribute resources between each other and then mirror onto data clusters that are stored around the world to increase throughput.
The Neowin setup is very different since Neowin is a tech news site with very little downloading. So it’s mostly a series of database servers and VM’d web servers to help distribute its load to keep the site lightning fast.
Years ago, when we were a lot smaller and I was more involved in setting up servers, the traffic was so much lower that you could have a single box for handling all the databases and another box for handling the web server. Such a setup, back in 2001, was enough to handle millions of users.
Our friends at deviantART were much the same way. A couple of boxes were enough to handle the whole thing. I remember comparing notes with their guys back in the day.
To give you an idea of how small a world things used to be, Betanews, Stardock, deviantART, and Neowin all shared either IT, bandwidth, or sever space. Long ago, if Stardock.com was down or if there was a problem over at Neowin I’d call up Nate Mook (the founder of Betanews/FileForum) and see if he could “kick” the appropriate server.
Heck, for all I know, there’s still some Stardock file servers in the Betanews rack somewhere. Things are a lot more sophisticated these days.
Now, the machines are unimaginably faster and more powerful and you need more of them because the traffic and also because of the sophistication of data mining people expect in real time.
Though still, even today, part of me almost expects to walk in and see some sort of huge machine with blinking lights filling a room.