What I loved about this article was the ending. By posting the sample job application it removes this article from a 'hey you should do it' to actionable direction on how you can do this.
So essentially you allow the new guy to work on the marketing side of the site without fear of "breaking" any of any critical backend functionality? I assume he was able to get started fairly quickly (not much to learn besides some html and css).
I love the thought of A/B testing. But like you said, until I hit that 100/$1000 threshold, I still need to work on a lot of other things besides conversion rate (i.e. marketing.)
Interesting way of approaching a/b testing. The core offering is in place so having a college intern just test as many things as humanly possible seems like palatable to any founder/exec. Might have to try this...
I used to a/b test titles and all this micro stuff. Total waste of time but great mental porn. The global improvements from those tests are so miniscule it's not worth it.
Having my co-founder available for only limited time a week and tons of yelling from him have greatly improved my prioritizing ability.
We were using kissmetrics, google site optimizer and google analytics. All great tools but as we are growing we didn't like the delay in the data and didn't trust all the data integrity.
He is thinking of open sourcing our optimization engine at some point in the future.
1- About 80% of tests are producing results that don't show increase / decrease
2- Assuming things will work better is completely off. I was convinced my lightbox idea from the article was easier for the user but turned out it was annoying.
One thing is a balance of making things too annoying vs getting features that help us grow. An on going challenge of always trying to do right by our visitors and customers.
It's interesting where about 80% of our test don't produce any significant results. One test nearly doubled our conversion of email and another one that is on-going now has done about a 30% increase.
Yeah, back at my last company we added 1 outside the box feature to the site and it increased per page conversion 50%, this was after years of rigorous optimization. It's always interesting what lateral thinking can do.
