In some form or another, I’ve spent my entire career generating insights from data. I recently spoke at this meetup about growth, social analytics, and data-driven culture. I’m not fond of presentations where the entire content is on the slide so that I have to wait for the speaker to tell me what I’ve already finished reading. The flipside is that my slide deck below is probably not that useful on its own. So I wrote down roughly what I spoke about.
Everybody wants the hockeystick curve of exponential growth. There is a saying:
“Growth solves all problems.” The deck shows Edmodo’s curve of registered users.
Don’t be fooled by the most recent data point, which is from before the 2013 back to
school time. Once the December data is in, there should be no sign of slowing down here yet.
If you are wondering why I’m only showing registered users, there are 2 easy answers.
First, it’s the larger number. Second, if I give a more meaningful number of active
users, the press will find a way to mix it up, and compare that number apples to oranges.
Even with exponential growth in absolute terms, your relative growth year over year will be slowing. How long you can keep your growth rate high is what really matters.
You won’t create lasting value without a product that provides value to your users and customers. Growth may solve all problems. But while “a rising tide lifts all boats”, “you only find out who is swimming naked when the tide goes out.” Swimming naked is the equivalent of having a product without real value. You have to be very clear what your value proposition is. And unless you are able to communicate the value to your users and customers, even ingenious ideas will not get you very far in creating value. Dogfooding your own product can help a lot in understanding its value or lack thereof.
At some point when discussing growth, viral growth and the viral coefficient K usually come up. K is the number of additional users every new user will bring to your product on average. If it is greater than 1, you have exponential growth. I’m showing a very simple example of K=2. The problem with viral growth is that it models viral infection. People fight off their colds just some users will stop using your product. K means little, if you have a churn problem. If your product offers no real value, you will probably have a churn problem. Spammy behavior can sometimes achieve a high K, but it’s not the way to build lasting value.
For most products, active user count is a far better metric than registered user count. There are some exceptions, for example LinkedIn. For its function as a directory, all profiles add value, even if their owners don’t visit regularly. This has its limits, and even to LinkedIn active users will be more valuable than inactive ones.
Organic contact is always better than marketing. Organic means resulting from a directed,
meaningful action by someone the user knows. Meaningful means few recipients, ideally
includes a personal message composed by the originating user, and is not only and obviously
self-serving to the originating user. Companies can help this last aspect, for example in
the way Dropbox has done, where both users get some free storage for a completed referral,
or in the way of ING Direct, where both users received $25.
Organic interaction is better than pushing users to do something out of context. Put a trigger in their path when it makes the most sense. For example, when I search for another user by email address, and there is no match, it’s a natural moment to prompt me to invite the email address I searched for. Make that very seamless and easy. That’s much better than a popup at a random time, which interferes with whatever the user actually wants to do. You may call it a lightbox now instead, but it’s still disruptive.
If you have growth covered, that’s the quantity part, but you also need to care about quality. Statistics such as “Users now send 5M messages per month” are great for the press, but not for you. Often, the right metric is on a per user basis. How many messages does each user send per month? How has that changed over time?
Carefully consider which metrics will get a boost from the network effect. More users in a social product mean more potential recipients for messages, and potentially more messages sent as a result. But this is a success of growth, not necessarily of the messaging product. Digging deeper, looking at the average number of messages sent between each unique sender-recipient pair should be fairly immune to growth effects. Similarly, looking at the number of messages sent among a fixed set of users can be helpful, though you have account for churn and survivorship bias.
For some products time spent can be a very good measure, especially if the product is focused on consumption and discovery. This doesn’t work for utilities. If you are building a tool, you aren’t doing a good job if you are increasing the time it takes to use it.
Data-driven culture is a popular aspiration today. Everybody likes to think they are
making only the best objective decisions based on facts. It’s very easy to agree with at
a high level, where data-driven means knowing important numbers, such as the average
revenue per customer, or knowing which factors are most strongly tied to user
satisfaction. In reality, this is hard to achieve.
One of my colleagues told me about a new hosted analytics service provider. I couldn’t figure out on their website how exactly their product worked, there just wasn’t enough information for me to be comfortable. Roughly speaking the product works like this:
After visiting their site I got these ads on the internet:
We have a nerdy looking guy in front of a whiteboard full of theoretical scribbles. He’s well dressed, but with a bow tie and suspenders, perhaps suggestive of a professor? At least there is no gender discrimination here. But what are these ads suggesting? Are data scientists an expensive, hard to manage, and hard understand bunch of theoretical brainiacs with a lack of common sense? It doesn’t matter whether your title is data scientist, analyst, or something else. This company’s business model is built at least in part on the premise that executives and decision makers are willing to pay to avoid their fears and expected struggles in building and managing data literacy within their own organization. It is somebody’s projection of what is wrong with us. You should view it as an opportunity. In economic terms, you should claim the dollars they are trying to extract, by doing an awesome job yourself.
If everybody is asking for this number or that, but only a handful of people can actually access, manipulate, and understand your data, data-driven culture will remain an aspiration. You need to make data as accessible as possible. Members of your organization who can write SQL should have a way to submit their own queries. Those who can build visualizations from aggregate data should have a tool to do so. The ones who can interpret charts should have a way to view them. Your organization is not just your team or department. It’s as many people as possible. Don’t think that making yourself irreplaceable is good for you. If you make yourself a bottleneck, you are the person in the ads. Lastly, if many members of your organization start accessing and processing their own data, help them with systems that prevent chaos. Facebook had very broad access to Hive, but it led to a lot of poorly named tables, essentially duplicate pipelines, and overlapping reports and charts. Create good visibility, which leads to accountability for keeping order.
Some words are often used with little consideration for their accurate meaning, for
example “exponential”. Don’t be a know-it-all, but when appropriate you should resist it.
“A/B testing” is also a popular theme nowadays, and a great tool when used correctly. But
doing the test you want, the right way, often isn’t that easy.
Magic numbers are desirable, since they often provide a clear goal or path forward. For example, when looking at a chart of the probability a user will churn as a function of the number of their connections, it would be great to see a steep and sudden drop-off. Then all you’d have to do is get every user that many connections. But in reality you will probably have continuous curve, without big jumps: the more connections, the better, with no magic cut-off. This doesn’t stop companies from creating soundbites, such as “Increasing the number of connections from 10 to 15 will decrease the risk of churn by 100%.” There is nothing wrong with that statement, but more often than not there is no magic number involved either. It’s simply the result of picking 2 points on a continuous curve.
You should strive for a partnership of equals. Data complements the vision and inspiration of
product managers. To be able to provide the best answers and work on the right tasks, make
sure you get the context, importance, and urgency of anything you are asked for. This is hard,
because as a service provider you don’t want to slow things down or make them complicated.
Don't hesitate to circle back and get the information you need, no matter who is asking.
When your CEO has a question, you want to make sure she gets the best answer possible,
not the fastest version of what she may or may not have needed or meant.
Be good at explaining and visualizing insights, and don’t hide behind stats speak and jargon. Make your results look nice. When you’ve spent a lot of time getting to the result, spending an extra 15–30 minutes on the presentation is high leverage. Personally, PowerPoint is not my strong suit. It’s also often abused. I’m not a designer, but I tried with my set of slides. Unfortunately, there is a lot of tragically bad design out there, as I found when I went looking for templates online. You either need to develop an eye for good data visualization and design, or have someone who can help you with it. Be deliberate; with data visualization there is a thin line between making it look good and creating unnecessary eye candy.