Buzz words in job requirement - A/B testing

Here are some other buzz words, A/B testing and multi variate testing.

This blog focuses on A/B testing because it's simple.

First of all, let's see the definition in Wikipedia.

A/B testing, split testing or bucket testing is a method of marketing testing by which a baseline control sample is compared to a variety of single-variable test samples in order to improve response rates. A classic direct mail tactic, this method has been recently adopted within the interactive space to test tactics such as banner ads, emails and landing pages.

If we can put the buzz word in the context of experiment instead of marketing or business, things are quite easy actually. The situation is that we need to find the optimal parameter for an experiment. The most simple thing we know is to change the parameter as many as time as we like, and compare the performance understand different value of the parameter. Of course, we need enough samples under each value of the parameter to see difference in a statistical significance way.

This is the dumbest way to optimize an experiment. But there are a few companies well know for A/B testing, listed by Wikipedia,

• Amazon.com pioneered its use within the web ecommerce space. 
• BBC.
• Google. One of their top designers, Douglas Bowman, left and spoke out against excessive use of the practice.
• Microsoft 
• Playdom (Disney Interactive) 
• Zynga 
• eBay

It's a really dumb method. That is my conclusion for this blog.

P.S. Here is product tour of a company who help to optimize visual website through A/B testing. It might be helpful to understand A/B testing.
http://visualwebsiteoptimizer.com/features.php