Listen on Apple Podcasts
JS: What’s one unconventional growth tactic that works surprisingly well?
Francis Brero: Yeah, I think for us it was something about going against, not just conventional wisdom, but going against the opinion of the CEO. So, this was in a case where we’re looking at like a high volume, really high volume of inbound sign ups for a freemium product. And we had really, really low day-2 retention rates. So, there was a really big drop-off on day-2. And so the whole team was focused on how we could increase that day-2 retention.
Because the idea was that, the leading indicator to revenue and conversions was going to be fixing that day-2 retention. And every chart seemed to be indicating that was the problem. And what happens is that when we actually started diving into the data, in a straightforward way where we said, okay, let’s stop looking at everything at a macro level and actually pick examples, one by one, and look at the journey. we actually found that the problem was not the day-2 retention. It was actually conversion. And so we ended up running a test of saying, okay, let’s run a campaign that tries to increase day two retention versus a campaign that increases conversion for the right people.
And we saw a huge difference in results. So, I guess what was unconventional here was even though all of the indicators seemed to push towards optimizing for activation day-2 retention and leadership was pushing towards that, I was brought into this conversation to solve that problem. We decided to tackle a completely different problem. And that was a much better solution.
JS: Awesome, how did you deal with leadership pushing back? I know that’s a challenge. What tips would you give folks around doing that?
Francis Brero: Yeah, I think that’s where it’s important to have every hypothesis typically backed with some kind of data. And I think what’s important is to be able to bring enough data to justify the possibility that there’s something else happening. So, maybe like very concretely, what we saw in this example was that out of 30K sign-ups that were coming in every month. Probably around 80% of those were either prosumers or spam emails. And so, there was no chance that those would ever convert or they might activate the product.
But again, you’re selling a B2B product. Like, if you get all those signups, you shouldn’t expect them to convert. And so what really helped was when we said, hey, here’s a random sample of a hundred signups that came in yesterday out of those, how many do you think should be able to convert? And we found that it was like maybe three or four something like that. And so that helped us agree that there is a problem, that if we look at the overall cohort, it’s going to be skewed towards the behavior of these poorly qualified or low value leads.
And so, therefore, what we want to do is look at our onboarding funnel through the lens of the high quality leads and actually start thinking about the journey for these high potential leads, rather than just everyone who’s signing up. And therefore breaking down what the funnel actually looks like for these different categories. Then what we found was, the ones that were companies that have the potential to use the product day-2 retention was really good. And what we had really had was a conversion problem.
So what was happening was that the sales team would end up talking to these folks, but wouldn’t know how to get them to pay. So there was actually a product issue where it was not clear what the value was to move from the free plan to the paid plan for these people that were a great fit. And one of the reasons there was because all the effort was put into making the free plan really awesome to try and boost that day-2 retention, say, Hey, look, our product is so cool. And we’re offering all of this value for free. You should keep on using it after the sign up.
And so that actually generated a pretty big problem when it came to conversion, because there was a very minimal differentiation between the paid product and the free product. And so that, I think, is a really interesting example of where the overall data and the averages tell an incorrect story that could lead you to actually damage your top-line in the long run, if you don’t address it soon enough.
JS: Awesome. Are there any sort of quantitative or qualitative results that you can share from making this decision?
Francis Brero: Yeah, absolutely. So, this was the initial analysis of saying, okay, like we have a conversion problem and now we need to figure out how we actually solve it. And then we can talk a bit about the tools we use there and what we did. But essentially we said, okay, let’s stop focusing on this top of funnel within the trial funnel, and focus on the bottom of the funnel. And we started running campaigns specific to that. And we started seeing very quickly a 5x uptick in meetings booked with the sales team. And after that, we started seeing that translate into revenue.
So that was huge, right! We went from a little under 10 meetings booked by month by one SDR. To over 40, by focusing on targeting the right people and changing the way that the sequences were built, and being less about a time drip saying, hey, you’re three days’ into your trial and this is what you should be doing. Your five days into your trial. This is what you should be doing, let’s try and book a meeting. And sending that to everyone and really focusing on the right people. We also started to think about how the role of sales is less about helping people swipe their credit card and more about being folks that are the right kind of quality, but are blocked in the funnel and how we actually help them overcome that.
So, it was like multiple shifts, but really it was all about, how do we make sure we maximize the value of our reps within this kind of PLG, a product led growth in motion and how sales could really have a huge, incremental impact. And that was not just having sales touch everyone through these automated campaigns, but rather focusing on the right people, so that every conversation they would have had a chance of yielding revenue and that the sales team was able to spend a little bit more time researching who they were contacting. Especially connecting on LinkedIn and things like that, where we saw that people actually respond more on LinkedIn than they do to email.
JS: Awesome. Any bottlenecks or challenges that if folks are wanting to implement this kind of strategy, you can give them a heads up around?
Francis Brero: Yeah, I think there are a couple of the requirements to get something like this up and running. One of the first challenges that I see a lot of people have is how do you do A/B testing in a way that’s easy to monitor afterwards. There’s some tools out there, like the stack was segmented for all the analytics tracking and kind of connection between tools, Customer.io was the email engine and Customer.io was really awesome because of their liquid templating feature. So, you can build these programmatically dynamic emails, and insert things dynamically, have if then conditions, which helps make fairly personalized emails and Clearbit was used for some enrichment to make sure that you talk about what the use cases are based on what job title they have. MadKudu is a core part of it and then Salesforce for tracking the opportunities.
But yeah, so on the A/B testing side, one of the things that we did was very basic, but essentially, if you take the lens of the email, a modulo too, if it’s like an event or an odd number that would go into group A versus B. We did a quick analysis to make sure that was a fair split. But I know that, that’s one thing that I’ve struggled with in the past. Like how do you build an A/B split that is going to be consistent across all your execution tools, right? Because if you do an A/B test, if you think of an optimizely where it drops a cookie, I mean, that works well.
But then if that same email comes through a different browser, you might not get the same allocation. You can do cross-device, all those kinds of things. But what we found was doing it based on the lens of the email is actually pretty straight forward. And because we were doing this kind of A/B test post email capture. It was really easy to know which segments someone fell into and it made it really simple to look at that segmentation in Salesforce and say, okay, if we look at these two cohorts, which one created more revenue, because it’s like a very easy computation to generate.
So, that was a big learning of how easy it was to create that split. Even though, again, it’s not perfect, but it really gets the job done. And it’s easy to implement. I think in terms of the other roadblock, which would probably be ahead of this, was focusing on the wrong thing. I think that one was the biggest problem for the company where every resource was focused on optimizing this day two retention when the problem was actually conversion. And that one, I think the learning and the way to avoid that roadblock is, for some reason we have shifted into a world where statistics rule everything, and everyone wants to look at charts and things like that.
I find that there is an incredible amount of value in going very granular and looking at specific examples and taking 10 random users and looking at their behaviors step-by-step and combing through that data, just to see if there’s any kind of a pattern that you can identify that is not being caught by the dashboard that you have, because the dashboard might have major blind spots. And that’s where looking at the data manually can help you identify some potential hidden patterns. I think that is incredibly valuable. And this is a good example of that.
I’ve seen countless cases where people are looking at these high-level numbers and it becomes a huge roadblock because there’s an analysis paralysis where we were trying to figure out, Oh, how do we change this little metric? And how do we change that? But there was a huge, unknown that people are not realizing. There is something they haven’t seen. And once they see it, they realize that essentially the experiment is completely bogus and you’ll have to go back to the drawing. Right.
JS: Francis, this was awesome. Thank you. So, so much for bestowing your wisdom today. If folks want to follow you on social media or get in contact with you, what’s the best way to do that?
Francis Brero: Yeah. So I would say for anything MadKudu related, so, we actually just launched some big new features. So there’s, the announcement has been held out. So, I would say either come to madkudu.com or subscribe to our newsletter. We’re constantly sharing some more insights around different data analysis that have been relevant. And then I’m fairly active on Twitter. My handle is Francis Brero, I share a lot of insights around data and marketing and data science. So feel free to hit me up there.
JS: Awesome, have a wonderful day.
Francis Brero: You too. Thanks for having me.
AO: Boom. That’s it. Another great episode of The One Growth Show , the official podcast of growth marketing conference to learn more about upcoming events, visit www.growthmarketingconf.com and subscribe to the newsletter. If you enjoy this episode, let us know. We’d really appreciate it if you’d give us a five star rating, super easy, just click the last star on iTunes, and also share this episode on social media. After all you want your network to know you’re the person they can always turn to for the best growth and marketing content, don’t you?
Follow GMC everywhere: https://linktr.ee/GrowthMarketingConference