In my experience if you are really doing serious traffic, you have to do a lot of custom stuff. The platform-as-service vendors are not prepared to really help you. Or if they say they are, they will charge you so much money that it is cheaper to do it yourself.
In fact... I actually worked at one of those platform-as-service places, back in the day before they were called such. This is how it works:
You are a big whale of a client, like twitter. Nobody at twitter likes systems engineering or sysadmins, so they call up the platform-as-service place. The platform-as-service place doesn't really know how to do it either... but they are thinking "fuck... this is TWITTER... we NEED this account." So they say they can do it. Then when they get twitter, they are totally in over their head and are scrambling to hire people who can fix twitter's shit. So essentially you are not just outsourcing your infrastructure, you are outsourcing the hiring of the sysadmins. You could just bypass this altogether and hire your own systems engineers in the first place. However lately it seems like the story has to include 7 months of half-assed "platform as service" with ultimate failure before realizing this on your own.
I almost replied to your terse OC to disagree [1], but I think this explanation is extremely insightful and worthy of attention and upvotes[2].
I call it the myth of the commodity server, and it's particularly prevalent among software engineers, since the natural inclination is to approach every problem as though it can be solved in software. Perhaps ironically, this often excludes the kind of software sysadmins write, which is deployment and configuration.
Far more importantly (from my POV), such an approach ignores the importance to cheap, reliable scaling of being able to customize the arrangement of the hardware [3].
Pretend that all servers are the lowest common denominator, and scaling gets expensive fast. Often the end result is more/earlier customization of software (e.g. writing a distributed DBMS from scratch, which takes on a life of its own, not getting any cheaper), and I can't imagine that's cheaper than hiring a sysadmin to customize configuration and deployment cheap hardware (which does get cheaper) to get mind-blowing I/O.
[1] I still do, in the sense that I feel it implies that the server hardware or its operation is anywhere near as expensive as employees. I'd be quite surprised if any startup spends more than one tenth on servers as it does on people. However, your point that some people are required when one has enough serious traffic is one with which I do agree and one that is often missed.
[2] Bias disclosure: I'm a sysadmin
[3] Which effective customization requires knowledge of and experience with
If you are 5 to 10 years away from profitability. Amazon incorporated in 94 didn't show its first profit until 2002, 8 years later. If you pay your sysadmin an average of 100K that is almost a million just for the one sysadmin(Some how I doubt amazon had just one sysadmin in 02). Sure they weren't purely a web business, what with the need to warehouse and ship all of that stuff, but as they say it takes a long time to be an overnight success.
It's probably also important to note that Amazon was a pioneer and that it's 16 years later, so perhaps their model isn't as relevant today.
More importantly, I'm confident they showed their first revenue well before '02, so it wasn't just VC money paying for sysadmins (and other employees) the whole 8 years.
