The Great Open Rate Debate
Are you reporting email campaigns based on the metrics provided by your email software? If so, is that because they work, or because they are easy to find? And do you know what those numbers really represent?
Emails opened per campaign and their open rates (ORs), along with click-through-rates (CTRs) and click-to-open-rates (CTORs) are the basic metrics of email marketing. They measure the effectiveness of email campaigns, are used as KPIs, brought up in meetings and fairly important decisions are made based on the info they provide. One might say that some base their company’s good fortune on that data.
This problem is compounded by the prevalence of these metrics within most common email marketing platforms. But just because they exist and most people use them, does this mean they are any good at what they are intended for? Marketers should ask themselves if OR, CTR and CTOR metrics are fit for their intended purpose.
Let’s try to answer the question – so how do we measure OR’s? How does your email marketing platform know which emails are opened and which are not? In most cases,this is tracked by the viewer loading images from your email. “But what happens when someone doesn't load the images or just views a stub of the message in Outlook without even opening it” I can hear you say? Well, they might have read your whole email and you will never know about it, as a result of their company’s security policy blocking image downloads. Indeed, from Outlook 2007 on, images are automatically blocked by default in these clients. Consider this as well – many other email clients do load images in the preview pane as a default and will record an ‘Open’ even though no one spent time reading the email. Furthermore, if the recipient clicks on the message with the intention to delete it straight away, the images might load and show up on your report as opened, even though they haven’t read a single word of your email copy.
Let that sink for a second…
It’s not all bad though and certainly doesn't mean that Open Rates are useless. It just makes them less important and changes their focus. They still work for comparative analysis, testing which day of the week, time of the day, “from” address, “from” name and subject line are performing best. It’s when you try to use them for other analysis you need to be careful about it – remember that open rates measurements are not exact science and be careful when basing your decisions around them. Look at it this way – treat ORs as a relative measure rather than absolute one – for example when the March Newsletter campaign had 10% higher open rates than the February one. All in all don’t read too much into the numbers – since 50 opens doesn't mean 50 people saw your email, 20% OR doesn't mean ⅕ of your audience saw it.
Now, there are some companies that will send an email without a call to action and have their sales call people who just open the email. Fortunately, it’s a rare practice now but at this point, knowing that you can’t rely on the Opens measurement, it is arguably only slightly better than cold calling.
Another implication is the CTOR. Since it’s calculated by dividing the number of clicks by the number of opens, this metric again is only good for comparing results of A/B test or seeing anomalies, like massive drops or rises of performance, but as with open rates is an indication and not science. Again, as a relative measure, CTOR is the metric that you should be using A/B testing email design and copy. Put it this way – the “outside factors” of the email are measured with OR, the “inside factors” (copy and broadly understood design) with CTOR. Use this to compare different campaigns to know what works, rather than have this as your KPI and you’re all set.
This leaves one last metric – the CTR. It is often considered the most accurate, by using redirect links to measure clicks from email campaigns. Clicks are only counted when a click is made and this provides hard, scientific facts on who clicked on your email and on which link. In my opinion, CTR should be the primary way of measuring effectiveness of campaigns on the operational level. As a campaign KPI, measuring Opportunities created per campaign would work better (in B2B) but measuring this effectively is a topic for a whole different discussion.
In general, open metrics can definitely be a useful measure of performance. They allow for monitoring variables like subject line, time and day of the week of send, problems with spam filters and “from” names.
(OR) Open Rate: Great for answering how well this email performed against the last month’s version to the same list. Also great for A/B tests of the “outside factors” like subject line, time and day of the week, from name and address as well as making sure you don’t have problems with spam filters. not so great for measuring how many people actually saw your email.
(CTOR) Click-to-Open Rate: This one is valuable tool for comparative measurement of the “inside factors” of emails – shows which design, email text, CTA placement, etc works better but has inherited the inaccuracies of Opens measurements.
(CTR) Click-through Rate: The one totally accurate measure in the set discussed in this post. In the emails that have a CTA (are you still sending ones that don’t? sheesh…), they should be used to measure the success of an email. On the other hand, if you want use A/B tests to improve the subject lines used in your emails or see whether your CTA should be at the top or the bottom of the email, use one of the above metrics.
So how do you use Open Rates? Do you find them useful or prefer other metrics? Will be happy to hear your opinion.