What Matters more: Conversion Volume or Conversion Rate – Case Study


 What matters the most to a business: Conversion Volume or Conversion Rate? 

There are three camps involved in this debate:

1st Camp – is in the favor of giving more weight to the conversion volumes. I am from this camp.

2nd Camp – is in the favor of giving more weight to the conversion rate.

3rd Camp – is in the favor of giving equal weight to both conversion volume and conversion rate.

Let us first start with the 2nd camp - I think people from this camp are those who have either not read the post on ‘conversion volume optimization’ or they just averse any change to traditional systems and beliefs.  If they are from the latter category,  I put them in the ‘conservatives’ bucket.  

There is nothing much you can do about changing the mindset of these people.  They will continue to talk about CRO and how great it is.  So let us leave them in their state of virtual bliss and move on.

People from the 1st camp are ‘liberal democrats’ (not to be confused with the UK political party, any similarity is purely coincidental ;) ).  

People from the 3rd camp are coalition of conservatives and liberal democrats.

Before we move further:

at no point I am saying to completely discard conversion rates.

But I am not in the favor of giving equal weight to both conversion volume and conversion rates either.  This is because:

Conversion rate calculations are horribly prone to errors

Conversion rate calculations are prone to errors (observational error, computation error, statistical errors, interpretation errors, reporting errors etc) and there will always be some sort of inaccuracy no matter how much you segment the data as it is a ratio metric.

This is clearly not the case with conversion volumes as it is a number metrics.

 

Marketing decisions based on Erroneous Data can’t produce Optimal results

How accurate your marketing decisions can be if they are based on the erroneous conversion rate data? Consider the following hypothetical scenario:

 

Visits

Transactions

E-Commerce Conversion Rate

Campaign A

1820

150

8.25%

Campaign B

20

4

19.25%

Campaign C

780

41

5.24%

Do you think you should be investing more in campaign ‘B’ because its conversion rate is highest?

I would suggest, not. The sample size in case of campaign ‘B’ (4 transactions out of 20 visits) is too small to be statistically significant. Had campaign B got 1 transaction out of 1 visit, it conversion rate would be 100%. Will that make its performance even better? No. 

Do you think you should now be investing in campaign ‘A’ because it has higher conversion rate?

Hold your horses right here.

Are you really sure that the difference between the conversion rates of campaign ‘A’ and Campaign ‘C’ is statistically significant.?

Let us assume that after conducting a statistical test we came to the conclusion that the difference in the conversion rates of the two campaigns can’t be proved to be statistically significant.

Under these circumstances we cannot draw the conclusion that campaign ‘C’ is not performing better.  So what we can do then?

Well we need to collect more data to compute statistical significance of the difference in the conversion rates of the two campaigns.

At this stage investing more money in campaign ‘A’ may not produce optimal results as you may think it will.  If you don’t understand what I meant, check out this post: Is your conversion Rate Statistically Significant?.

Now let me ask you one more question:

How many times do you conduct a statistical test (like Z test) to calculate the ‘confidence’  that difference in the conversion rates of two or more campaigns is statistically significant before you declare one campaign as ‘winner’ and decide to invest more?

I am the confidence you need, to play with conversion rates

I can bet only handful of marketers/analyst go through this hassle. Can you see yourself conducting such statistical tests every time you look at your conversion rate reports, day in, day out?

 

Conversion Volumes reflect ‘effect size’ (Signal) much more accurately than conversion rate

This is one of the biggest reason of using conversion volumes while taking marketing decisions.

It is possible and quite common for a result to be statistically significant and trivial or statistically insignificant but still important.

From the table above we can see that the ecommerce conversion rate of Google CPC is higher than that of Google Organic.

Does that mean Google CPC campaigns are performing better than organic?

Before we jump into any conclusion and invest more in PPC, let us calculate the statistical significance of the difference in conversion rates of Google organic and PPC campaigns.

So according to my statistical test (Z-test), I have only 65% confidence that the difference in the conversion rates of Google organic and Google PPC is not by chance.  

As confidence is less than 95% the difference is not statistically significant and we need to collect more data before drawing any conclusions.  

To calculate the confidence in my analytics report I used the ‘z-test bookmarklet’ developed by Michael Whitaker.

You can find details regarding installing and using this bookmarket from here.


Even if the difference in the conversion rates of Google organic and Google PPC turned out to be statistically significant we should still be investing more in Google organic (in this particular case) as the effect size (here revenue) of Google organic is much larger than that of Google PPC.

Just because a result is statistically significant, it doesn’t always mean that it is practically meaningful.

That is why we should interpret both the statistical significance and effect size of our results.  This makes ‘conversion volumes’ such a powerful metric.

 

Conversions Volumes go well with regular people

You don’t need to be a geek to understand conversion volumes.  Conversions volumes reflect marketing efforts much better than conversion rate and are easy to communicate with people from all walks of life.

Nobody gives a crap what your conversion rate is if the sales are going down.  Your site conversion rate may be increasing but sales may still be go down.  This is possible if traffic to your website is declining.

 

Conversions Volumes can be optimized

You can use conversion volume in any analytics framework you create for your business. This is because you can set achievable targets for them. You can’t set achievable numerical targets for conversion rates with any ease.

 

Conversion Volumes are error free

Conversion volumes are not prone to errors (observational error, computation error, statistical errors, interpretation errors, reporting errors etc), are independent of conversion rate and other factors like website traffic.

Remember conversion rate = conversion volume/total visits.  Without conversion volume there will be no conversion rate.

So whenever there is a trade off between ‘conversion volume’ and ‘conversion rate’,  I support ‘conversion volumes’ any time of the day.

 

So do i still recommend using ‘conversion rate’?

Absolutely.

But just remember what you are playing with. This is not just any other metric. It is a powerful ratio metric. If you are not segmenting the data, the way it should be segmented, if you are not taking conversion volumes,  statistical significance and ‘confidence’ into account then your ‘conversion rate’  focus strategy won’t produce optimal results and you may even incur huge losses.

There is one more thing. Some people confuse CVO with a metric.

CVO is not a metric, its a process just like CRO.

But ‘conversion volume’ is a metric just like ‘conversion rate’ and conversions volumes not only include macro conversions like ‘orders’ but also micro conversions like avg. time spent on the site, newsletters signups etc.

In the end conversion volume is what matters the most to a business because it speaks the universal language (the language of money) loud and clear.

Other Posts you may find useful:

 

 

Himanshu Sharma About the Author: is the founder of seotakeaways.com which provides SEO Consulting, PPC Management and Analytics Consulting services to medium and large size businesses. He holds a bachelors degree in ‘Internet Science’, is a member of 'Digital Analytics Association', a Google Analytics Certified Individual and a Certified Web Analyst. He is also the founder of EventEducation.com and EventPlanningForum.net.

My business thrives on referrals, so I really appreciate recommendations to people who would benefit from my help. Please feel free to endorse/forward my LinkedIn Profile to your clients, colleagues, friends and others you feel would benefit from SEO, PPC or Web Analytics.

 

 

  • http://www.timpeter.com/blog Tim Peter

    Hi Himanshu,

    Just a quick question about the above: Is the data in the GA screenshot correct? Your CPC numbers displayed should be a 0.73% conversion rate, not 1.73%. In this case, the organic traffic/conversion rate would likely be better than CPC (statistically significant at a 99% confidence interval).

    • http://www.seotakeaways.com/ Himanshu

      Good catch Tim. I don’t know how it happened. I just re-used the screenshot from the ‘statistical significant’ post. Anyways I hope i got my point acrossed.

  • http://www.paulmadden.co.uk Paul Madden

    This is not really a geek study, every seo should be doing CRO…good post though Hamish

  • guillaume

    I disagree with your approach, marketers should be using the all mix, there is no better option, you have to use all of them together. There is enough external elements to skew your data, using CRO and volume conversion is the way to have insight on the all picture. As mentioned by Tim, the CR should be 0.73%, regardless of this results, I wouldn’t be comfortable comparing SEO Vs PPC as I consider them as being 2 different segment. Looking at the data above there is no doubt that you should concentrate on SEO rather than PPC (or improve your PPC campaigns to bring in more qualified traffic). Last point, you seem to work only ith GA , I have been working with other vendors and in my experience, I never had most of the issue you have expressed here, alos I used to be an SEO as well, and I remember how volume was more important than CR – so I would definitely recommend to consider CR + volume AND AOV without overlooking any them

    Regards

    • seohimanshu

      This post is about choosing between Conversion Volume and Conversion Rate whenever there is a trade off between the two. It is not about any methodology or approach. Regardless of that conversion volumes is less prone to errors than conversion rate no matter which Analytics tool you use as CR is a ratio metric.