How to Actually Use Key Metrics and Data Analysis to Make a DifferenceOn any given day, I spend a few hours discussing web analytics with a bevy of businesses, small to large. The one thing I consistently notice in companies without a staff analyst, is randomness. Seldom is there a coherent strategy for analysis and measurement.
By basic logic, random actions lead to random results ~ Neil PatelWithout strategy, analysis usually consists of the following steps: 1. First, look at some collection of data (any data you can find will suffice). 2. Step two, make wild assumptions. 3. Step three, do something. Weeks later, someone in a meeting will ask if “something” worked. No one will be able to say for sure. Usually, no one remembers to measure the results properly anyway. Luckily, this is easily solved. With a smidgen of focus, you’ll be jostling answers loose from your web analytics questions in no time!
Measurements inform uncertain decisions ~ Douglas W. HubbardIt doesn’t matter which web analytics tool you use. Almost any half-decent tool will give you enough data to work with. Analytics is not about all-the-data; analytics is about reducing uncertainty. I use Google Analytics and it does a fine job of uncovering the insights I usually look for. Then, throw in some Excel or Google Docs for the occasional sum, and I’m good to go. As the founder of Teacup Analytics, it is tempting to recommend my own tool and, while I believe it can make your life easier, I’ll keep things platform agnostic. As you wade through the rest of this post, please remember the following two points:
One of the harder things to do is correlate measured results back to specific and repeatable actions ~ Ash Maura1. As you test ideas, just be aware that you don’t muddy the waters too much. Try things out, but please avoid trying too many things at once. Oh, and avoid multiple concurrent tests aimed at improving the same metric.
An unrepeatable result is no result at all ~ Johannes Cabal, necromancer2.Once you’ve taken an action, you’ll need to monitor the results over time. Often, you’ll notice a sharp impact right away, but it is the ongoing impact that matters. We’ll discuss how to measure this in more detail later.
Understanding Quality With “Good” MetricsTake a look at a metric. Any metric. Is it good?
A good metric changes the way you behave ~ Ben Yoskovitz, author of Lean AnalyticsAs weird as that question might sound to you, it’s a common one. Some people want ponderable numbers and data with significance. Others simply prefer the reassurance that they’re doing alright. That, to them, is a good metric. To me, a metric should point you in the right direction and inspire you to act. That is a good metric. And to get inspired by a metric, I first calculate its “quality.” I’ve used Excel, Google Analytics and Teacup Analytics to handle the following calculations in varying degrees of complexity but I’ll keep it somewhat simple for now. We’re not necessarily aiming for precision, merely for reducing uncertainty.
Calculate Your BaselineThe baseline is your point of comparison. When looking at your performance today, and when looking at your performance after you take action, you’ll need to be able state, unequivocally, that your change resulted in an increase, decrease, or had no tangible impact. The baseline will help contextualize your results. This baseline allows you to classify your results as successful or as failure.
Step 1: Choose Your MetricsDepending on your business, you might have one, or two metrics that matter the most to you. The easiest metric to hang your hopes on is conversion rate. Revenue per transaction, bounce rate, site depth are all great metrics too. You can roll everything up into a larger KPI so you don’t need to limit yourself to one metric. I usually use three or four metrics depending on the website and product range. For our example, let’s calculate quality using the engagement metrics of conversion rate, bounce rate and site depth (pageviews per session).
Step 2: Calculate the AverageSelect a time period. I like to use three months as my period. In my experience, this is a long enough timeframe for ideas to manifest results as well as smooth out the impact of outliers like holidays and sales. Depending on your customer behavior and sales cycle you might increase this time period, even up to a year. Next, find your average bounce rate, conversion rate and site depth for that time period. For three months, I prefer to take the weekly averages as my unit of measurement and then average that out for the twelve weeks. For a shortcut, you can just grab the overall average in Google Analytics. Set your time period in the upper right corner of any Google Analytics page. Then, navigate to Acquisition > Overview. You’ll see your averages for each metric here: Write these averages down somewhere. We’ll be coming back to them shortly.
Calculating Your QualityNext, we’ll be unearthing the quality of various traffic channels. It’s actually quite simple. I use this technique to “troubleshoot” channels by repeating this process for segments of channels to find where the weakest links are. Let’s look at current performance. Simply stay on the same page in Google Analytics we were last on, and change the time period to the last week. Now, type in each channel’s weekly avg. into Excel and follow these steps: 1. Calculate the difference between Current Performance and your Baseline 2. Add the results of the differences up into an overall KPI It might look a little something like this: We’ve just calculated which traffic channels are currently the highest quality (green), average quality (yellow) and lowest quality (red). We’ve calculated this relative to our recent history. At Teacup Analytics, we do something quite similar to this. After assessing your unique variability, Teacup grades your traffic on a curve, offering more nuance to these concepts. However, these ideas apply all the same, despite a somewhat simplistic approach in this example.
Analysis Is Easy!OK, we’ve visually ranked our traffic channels by “quality.” So what? Well, here is where the insights lie. And after insights, comes inspiration and action! Let’s take a look at our data on a graph: What we’re looking at here is our channels, ascending by volume along the X-axis. The Y-axis is our quality. What we’re looking for is which channels offer us the best opportunities to act! Let’s pause a second and think about the following statement: All analysis boils down to three possible actions. 1. Do Nothing 2. Grow 3. Optimize
Do Nothing:Channels that are both low quality, and low volume aren’t worth your time and effort. There is too little payoff. These are the channels that will offer little return for your investment.
Growth Opportunities:When a channel has significant volume and is high quality, it indicates that they’re the right type of audience and they are engaging nicely with your site (i.e. they’re converting). You want more! Thus, you’d grow the channel.
Optimization Needed:When you have a low quality channel with significant volume, the returns on growth efforts are minimized. What the smart folks do is optimize these channels first. Once the channel has improved, and has become high quality, it will be ripe for growth. Let’s take a look at our graph again through this analysis lens: Insights abound! Social and Email channels are simply too low volume to offer much return so they don’t warrant any effort. Both Paid Search and Organic Search have an engaged audience and are absolutely ripe for further investment to grow those channels. Referral is doing alright, but could be performing better. Here, the best thing to do is improve the experience for the referral traffic we’re already getting. Like a leaky bucket, you want to plug the holes first, before pouring more water in. Direct is a special case. Direct traffic is notoriously challenging to optimize, due to the difficulty in understanding the audience intent. I’ve left that off of the actionable items here but if you’re interested in “cleaning up” your Direct traffic, I heartily recommend this awesome article by Avinash Kaushik: Excellent Analytics Tip #18: Make Love To Your Direct Traffic. Note: Check your trend! Please don’t make big decisions if your current performance is an outlier. Rather wait a week or two until things level out. Most outliers level out around four weeks anyway. Another note: When I say “significant” traffic, I simply mean that there should be enough traffic to consider the channel impactful.
Actions Are Easy TooWe’ve analyzed our web traffic and come up with plausible opportunities for growth and optimization. Now that we know where the opportunities lie, deciding on the actions to take is not difficult. Sure, the details will need to be threshed out, but at the high level, you can now decide on where to focus. Want to grow Organic Search traffic? Google has a ton of ideas for you: Need to optimize your referral traffic? Guess where to find some tips! Finding the actionable insights is quite feasible, isn’t it? But before you charge ahead, weigh your options.
Some Things To Consider Before You ActWhen deciding on a plan of attack, inspired by your analytics, you want to consider where you can get the most return for the least effort. While discussing these ideas in detail is beyond the scope of this article, I encourage you to investigate these ideas further one day.
Calculate Traffic Value Per Visitor:Before deciding which channel to grow, try calculating the value of each visitor. The formula is: Total_value_of_conversions_per_channel / Total_unique_visits = Value_per_visitor Using this formula, you can decide between two growth opportunities and choose the one that offers the most return.
Cost To Benefit:I shouldn’t need to mention cost to benefit but, well, too many people ignore this. Either it’s too disheartening, or it’s not fun enough when you’re inspired to act. If your planned action requires any investment in time and money, please calculate the costs relative to likely benefit.
The Cost Of Inaction:But what about not doing anything at all? If you’re not sure whether optimizing a channel is worth your time, try figuring out what inaction is costing you using Monetization Models. Monetization models, which I first read about in Actionable Web Analytics assign value to specific actions on your site – the types of actions you’d optimize for. With these models, you can learn which optimizations offer the most ROI as well as what you’re losing by not doing anything.
The Long Term Benefits Of Optimization:Don’t forget this simple idea! Even the smallest real increase to your conversion rate continues to pay dividends in the long run. As your traffic grows, you continue to reap the benefits of that extra percentage point!
Assessing Your Results
Ignorance is never better than knowledge ~ Enrico FermiAssessment takes time. However, because monitoring results lacks sustained dramatic interest, many take action and then wander off to find new, excited things to do. My guess is that many ignore the results because not knowing is easier than admitting you took the wrong actions. Not you, though, right? If you’re not measuring the results of your actions, you’re wallowing in ignorance! And that is silly, considering how hard you’ve worked to take action! Sometimes the steps you took worked out well. Other times, they cause adverse reactions in customer behavior. Being aware, and able to react is important. Here is the process to properly assess whether your actions are positively impactful or not.
Step 1: Check Current Performance Against A Segmented BaselineNow that you’ve taken an action, it’s time to see if there was any immediate impact. To do this, you’ll need to create a baseline again. Previously, we’d created a baseline for all website traffic to uncover which channels are over- and under-performing. This time, you’re following those exact same steps but focusing only on the metrics you wanted to impact. Let’s call this the segmented or granular baseline. For example, let’s say we wanted to impact the bounce rate of referral traffic. To do this, you’d first find the average bounce rate on referral traffic for the last three months. This is the granular baseline. Give it about a week and then compare your latest week’s referral traffic bounce rate to your granular baseline. This is the early warning system! Depending on your changes, some dips are expected but if something drastic occurs, at least you’ll be aware and ready to change course if needed. However, like The Hitchhikers Guide To The Galaxy says, don’t panic. Give things time to level off.
Step 2: Average Out ResultsAs each week progresses, start averaging out the results. Just like you created your granular baseline, you’re creating a new post-action baseline as time goes by. After a month, you’ll be able to compare your new baseline average to the granular baseline. Ideally, you should see an improvement by now.
Step 3: Compare Time PeriodsOur baselines have been three month periods. We want to give our actions that same amount of time to make their impact. By comparing averages over longer time periods, you’ll be able to smooth out the variability. A significant overall improvement over three months indicates that the improvements are likely to be “real” and ongoing. You’ll be able to say, with certainty, that your actions resulted in genuine improvements.
Web Analytics Wonderland
Anything you need to quantify can be measured in some way that is superior to not measuring it at all ~ Gilb’s LawCertainty is hard to come by, but in a world gone mad, you don’t need certainty. You just need to reduce uncertainty. Look at your analytics and you’ll find insights, if you’re open to it. By assessing the results, you’ll be able understand the relationship between your actions and your audience. Be wary of ignorance! You should have a good idea as to why your actions worked, not just that they do. Interview customers, use surveys and listen to feedback. Armed with this understanding, you’ll be able to replicate your successes and avoid your mistakes. The best part of analysis? It never ends! Each success offers a further opportunity to learn, act and measure.