(This article originally appeared on LinkedIn Pulse and can be found here.)
Product managers, also known as product owners, are a relatively new and increasingly important role within organizations. We at Emperitas wanted to find out what makes the individuals placed in this unique role “tick”. After conducting a round of in-depth interviews with various product managers, the responses we got were fascinating, to say the least:
- There was no college program that trained them for what they do in their current job and they see themselves as a “mini CEO” without an existing playbook.
- They’re more user/customer focused than anyone else in their organization.
- They trust their gut, but their job is trending toward data and some are evaluated by HR based on their data proficiency.
- They feel empowered by using data in their job, but are largely self-taught and often nervous about data interpretation.
This is an especially pressing issue for the exploding Software as a Service (SaaS) companies because the very thing these owners manage generates a massive amount of data and has the potential to generate even more. If you count yourself among these brave new product leaders, here are our tips for becoming a Data-Driven Product Manager:
Accept That You’re Solving A New (Data) Problem
As a product manager, you’re at the front line of a transition caused by the digital transformation. People are still determining what the product problem is, and this is a prerequisite to identifying any viable solution. This leads to advice being general and often uninformative and since your product and organization is unique, some people don’t understand your problem to begin with. The key in these situations is to focus on core principles instead of case study advice. As a product manager, these principles should revolve around product usage, customer-centric design (especially user experience), competitive offerings, pricing, market share and industry trends. If the advice you receive doesn’t help you use your data to successfully find answers for those pieces of your job function, ignore it.
What We Mean by “Product Data”
As a product manager it’s likely you already have access to a lot of data, along with the means to get even more. But, this high volume can cause difficulties in trying to manage and use all your data. This typically indicates your organization lacks clear data governance or is the result of uncertainty about the data you have and the pros and cons of each.
A clear distinction in data types that was understood by every product manager we spoke with was observational data coming from the product itself, like event data, versus direct user/customer feedback data like app store reviews or surveys. These are not mutually exclusive data types and you can get better insight by blending one with the other. In addition, while one is not necessarily better than the other, they do require vastly different methods of analysis. While structured data is often easier to deal with, unstructured data is usually more informative. There’s a core set of metrics that most product managers currently use to guide their roadmap. They all credited success to having a common definition of the metric and how it’s calculated across the entire organization:
- Customer Acquisition Cost (CAC)
- Customer Conversion Rate (CCR)
- Repurchase Rate (RR)
- Daily Active Users (DAU)
- Feature Usage (FU, yes that’s really the abbreviation)
- User Churn (UC)
- Net Promoter Score (NPS)
- Customer Satisfaction (CSAT)
- Customer Lifetime Value (CLV)
- Use Cases For Data in Product Management
The first priority for your data should be finding the overall “product-market fit.” Usage data (like events) is a good place to start for this, but direct customer feedback is mandatory. One manager we spoke to said that no observational usage data has ever been as effective as picking up the phone and talking to 10 customers directly. As a general rule, we agree. But it’s not just because conversations have magical power, it’s because you probably haven’t setup formal tests for evaluating the usage data.
This leads to the next use case, hypotheses testing. If the consequences of a test outcome are big, and require large development costs or scrapping a part of your product roadmap, it’s a necessity to ensure your tests are valid and to run the test multiple times to prove consistent results. Your data is a great way to validate what you’re doing is positively impacting revenue and not just making users/customers happy. For example, is that expensive new Machine-Learning project really driving a measurable ROI? Did adding an AI component for recommended upsells actually increase purchases? By pairing the right test with the right data, these are things you can readily learn.
Bridge The Departmental Divide
The product managers who seemed the happiest all came from organizations using data as a key component of the decision-making at every level. As a mini CEO obsessed over what customers want from your product, you’re in the best position to continually drive this data culture change.
Data-driven decision making reaches new heights when combining data from across the entire organization. Our list of metrics above is organized into departmental silos, and while daily active users and conversion rates may be the metrics you constantly observe as a product manager, success is often pre-determined by the marketing and sales process before the customer ever touches the product. The use of conceptual frameworks to guide collaboration with the other departments in your organization, especially marketing, sales, and customer experience is a great help in this process. The best frameworks include the customer journey map, user/customer personas, and customer lifetime value cohorts (with a special focus on finding the 20% of all user/customers who predictably create 80% of the total value, known as the Pareto Persona).
How Data Can Fail You
Interpreting data was a major pain point for all the product managers we spoke with, not only because of a (possible) lack of formal data training; but also dirty or incomplete data making things worse. Even with perfect data, misinterpretation is still possible. The only solution we’ve found is to involve multiple people in the interpretation and encourage radical candor in the process. For additional support, you can also hire a data scientist. Your unique situation should determine your decision on whether to embed these data scientists with dev teams or keep them as part of the product management team, but it’s on you to ensure that the right type of data scientist is hired for the job.
Finally, simply having a lot of data doesn’t mean it’s the right data. You’ll learn this through exploratory data analysis, hypothesis testing, and interpreting output. This can also be addressed with qualitative interviews (the topic of our next article and an upcoming Emperitas lecture) because in a big data world there’s still a very important role for humans (you and your users/customers). Don’t ignore your gut, but inform it with more evidence and trust yourself to make the right decision based on all the information available to you.
Want more on how to optimize your data analysis? See our post on how to get from Analytics To Actionable Insights.