A/B Testing Amazon book sales

A/B Testing is when you try two almost identical approaches out with real people to see which option gets the best response. The only difference between A & B is the thing that you are trying to test.

In my trial I had the same science fiction novella available through two Amazon ASIN. Both were enrolled in KDP (there’s nothing forbidding this but I’m sure Amazon don’t encourage that behaviour).

What was different between these two was that one of the versions has two reviews, both five stars. There are also changes to the wording in the blurb.

The test

So this wasn’t quite true A/B Testing as I didn’t perform the free days simultaneously, one was over the Christmas period and the other was last weekend. However both versions got the same twitter announcements from me. I scheduled a number of tweets giving a mixture of links to the UK, US, Canadian and Australian amazon links. The key difference on these was that for the reviewed version I mentioned it as having five stars.

One thing that the reviewed version hasn’t had, but the unreviewed version does, is some paid twitter advertising. This gets it five tweets a day from different accounts with a seven day cycle of tweet content. So it has had more advertising for a period of a month before I started the free days for the version with reviews.

Results

Despite lots of pushing the unreviewed version only got 18 downloads when it was free. Given that this was 24-29 Dec when I figured lots of people would be looking for stuff to put on their new kindle it seems pretty poor to me.

The version with two five star reviews and slightly less publicity beat the other one for free downloads within six hours of being free. There was also a sold version in the period between the first tweet and it becoming free.