Behavioral Segment Analysis
One of the most powerful actions you can take as a product manager is to understand the behavior of your users. Knowing user behavior is key to driving additional value within your product on their behalf.
You can gain that user empathy through various approaches, such as user interviews, shadowing, surveys, and more. But, one method that I’ve found particularly valuable is conducting quantitative behavioral analytics on user segments.
First, I’ll discuss why behavioral segment analysis is powerful. Then, I’ll share the prerequisites for conducting quantitative behavioral analytics on user segments. Finally, I’ll provide a couple of examples of analyses you can run, and the kinds of value that you might derive from each.
Why Use Behavioral Analytics?
What is behavioral analytics? Behavioral analytics focuses on the actions that users take.
For example, how many times do they leverage some functionality? When do they touch that feature? What actions came before, and what actions came after? Which features do they never touch?
In general, analytics are valuable for any product manager. After all, quantitative metrics enable us to observe our product’s performance at scale. But, behavioral analytics are particularly valuable for product managers. Why is that?
Much of the time, “standard” product analytics focus on high-level KPIs, such as overall traffic, overall conversion, overall retention, or overall transaction size. While these KPIs are valuable for the business, they don’t reveal the behavior of your users.
Behavioral analytics is a fantastic way to start understanding how exactly your users are using your product in their day-to-day lives. When you understand user behavior at scale, you can then generate new hypotheses for A/B testing, and you can kick off new threads of qualitative research.
Why Analyze User Segments by Behavior?
Different kinds of user segments will naturally have different behaviors, since behavior is driven by context.
If you have a freemium product, your free users will behave very differently from your paid users.
If you have an enterprise product, user roles drive dramatic differences in behavior. For example, consider the different needs of an administrator, a marketer, a business operations analyst, and a sales representative. These different needs will then drive different behaviors on a CRM (customer relationship management platform) product.
If you have an international product, different geographies will have different behaviors as well.
But, rather than starting from a known user attribute like geography or subscription status, we can segment by user behavior patterns instead.
If you have a strong grasp of the behavioral differences between highly engaged users and highly disengaged users, you can coax disengaged users to be more engaged by nudging them towards the behaviors of the highly engaged users.
And, if you split your users by “who used this feature” vs. “who’s never used this feature”, you’ll find potentially interesting insights that you would have never been able to reveal by looking just at known user attributes instead.
Prerequisites for Segmented Behavioral Analytics
Of course, while analytics can be powerful, you need to ensure that your infrastructure is set up so that you can conduct the appropriate analysis.
What prerequisites do you need to satisfy before you can conduct segmented behavioral analytics? Well, you need to have a way to track the behaviors that you care about.
This requirement regularly trips up product teams, especially for newer startups. Many times, events and actions haven’t been tracked yet in their product, which makes it impossible to identify behavioral differences.
If you haven’t yet implemented behavioral tracking yet, now is the time to do so! After all, how will you know that your new functionality is being used unless you can track that users are interacting with it?
You can use out-of-the-box solutions like Mixpanel, Amplitude, and Heap, or you can build out your own behavioral events pipeline. Here’s an implementation guide you can share with your engineering team on how to build out user events for behavior analysis.
If you have behavioral tracking ready to go, then you can immediately dive into a segmented behavior analysis.
Examples of Behavioral Segment Analyses
Let’s look at some examples of behavioral segment analyses that you can run on your product. To bring the conversation to life, I’ll use Google Maps as an example (though unfortunately I don’t have the actual behavioral data to analyze myself).
One kind of analysis we can use is “how a specific feature changes behavior.” Here, let’s look at the feature of “view COVID-19 info,” which appears as an option under “Layers” in the bottom left-hand corner.
If we’ve set up our behavior events pipeline correctly, we should be able to identify “users who have never used this feature” vs. “users who’ve used it at least once.” These are now two separate segments that we can compare against each other.
We might hypothesize that “people who look at COVID case data will want to stay up-to-date on a regular basis”, so we can compare these two segments in terms of “how frequently they access Google Maps.”
Does the segment who never used it only use Maps sporadically? Conversely, is the segment who uses “see case data” consistently coming back into Google Maps to look at changes in COVID cases in their area?
As you can see, we didn’t segment these users based on static user attributes, like their age or their geographic location. Rather, we segmented them based on whether they’ve ever interacted with the COVID visualization feature.
Another kind of behavioral segmentation analysis we can do is to look at the distribution of usage for a given feature. As an example, let’s look at the feature for “ordering takeout online through Google Maps.”
We can spin up 3 different segments here based on user behavior: low usage, medium usage, and high usage. Since I don’t have the data on hand, I couldn’t actually tell you where the cutoffs for each are, but you can generally use something like “thirds” or “quartiles.”
As an example, say that you have usage ranging from “order online 1x per month” to “order online 60x per month.” Pretend that you have the following distribution:
Your “low usage” segment is the people who use Google Maps online ordering 1-2 times per month (20 + 13 = 33%). Your “medium usage” segment is the people who order 3-4 times per month (16 + 18 = 34%). Your “high usage” segment is the remainder.
We can now compare these 3 segments against one another to yield valuable insights. For example, we can look at the most commonly used features within the high usage segment, and determine which of these aren’t being used by the low usage segment.
From there, we can then nudge people in the low-usage segment towards taking similar actions that our high-usage segment takes.
Or conversely, we can look at the features that our low-usage segment is using that our high-usage segment doesn’t use, and determine whether these features are causing them to become disengaged with our product.
Finally, we can look at how a given feature fits into a user’s flow. Let’s consider Google Maps Reviews. You can access it the following ways:
Through the “order online” search panel on the left hand side
By selecting a point of interest within Google Maps
By searching for a point of interest in Google Search and navigating to Reviews
We can have each of these three patterns define our user segments, and then we can analyze differences between them such as:
Which of these segments generates the most ad revenue?
Which of these segments abandons “order online” most frequently?
Which of these segments are most likely to leave a review?
These insights can then help us spin up new threads of inquiry, new A/B tests, and targeted user interviews to confirm or disprove various hypotheses that we have.
Hopefully, these examples give you a good sense of the kinds of cuts you can make within your own product when leveraging behavioral analytics!
Looking to further level up your skills on the job? Consider joining the Product Teacher PM Masterclass to gain mastery over both hard skills and soft skills.
Summary
Segmented behavioral analysis is a powerful way to gain user empathy, and to take that user empathy to drive business value.
First, identify the segments that you care about. Then, look across behaviors of interest, and analyze the differences in behaviors across segments. Finally, use those insights to drive actionable next steps, whether those next steps are A/B tests or additional user research studies.
Thank you to Pauli Bielewicz, Siamak Khorrami, Goutham Budati, Markus Seebauer, Alina Ha, Juliet Chuang, and Shanthan Gangupantula for making this guide possible.
