I'm actually quite amazed that a more fleshed out solution to AWS logging doesn't exist. I've worked on a small side project to pipe AWS logs to arbitrary locations for work purposes so that they could be visualized in whatever tool of your choosing
Mind if you could cover some broad details about how things are handled in the back end?
I've been meaning to write that blog post for a long time.
In broad terms, EC2 is the perfect fit for a service like this, that needs to run something like 100 hours of computing each day, but needs that all to happen during a 3 hour window before Europe wakes up in the morning. It's even more fun when something breaks and I get to spin up 200 machines in one go. For like ten dollars.
I use something like 8 different AWS services for various bits of the thing. Everything from computing to storage to queueing & mail. I even used them for payment at one point.
Mind if you could cover some broad details about how things are handled in the back end?