Why is everyone obsessed with AB testing sign up button color? Sure, it's easy to test, but it's such a tiny factor, in amongst huge things like copywriting, page layout and the inter-page structuring of user journeys...
That's because that is the most "oh-wow" example you can give to a person who has never tried AB testing before. Lot of people still don't do A/B testing because they either don't know about it or they don't think it works. Sign up button color examples works well for educating both types of people.
Just as important, skeptics can be convinced that the button mattered. People think with their eyes, so visually obvious changes feel real. If your first exposure to AB testing was me telling you that a one word tweak to the sidebar of my landing page increased signups by ten percent you might conclude that I am as barking mad as someone with a perpetual motion machine.
Still, even if the reaction is an "oh-wow" ... it still takes work to do, even for simple things like changing the color of a button.
And since we are talking about statistical confidence, you may end up waisting time instead of doing more meaningful work ... so I don't think A/B testing helps when you're small, unless you have the resources to spare.
I am happy to read about tools/advices that I can use. Nice article.
