3 questions to answer when analyzing customer feedback
Updated: Jul 24, 2019
Brands choose to use open-ended questions more often in customer satisfaction surveys. Compared with closed questions, open questions help brands capture unexpected learnings, gather richer details and even establish a connection with their customers.
So we got open-ended customer feedback, now what? Most of us scan through the feedback and try to get an overall idea. However, we often lose track after reading just tens of them. And what’s worse, emotional feedback catch more of our attention and we ended up having a biased view of the customer feedback.
Analyzing open-ended unstructured customer feedback can be challenging. But approaching in the right way, it can be done very effectively. In this blog post, we will go through the 3 key questions we should answer to extract the valuable insights from open-ended customer feedback. And with an example, we will see how the answers to these questions help correct our misunderstandings and guide us on our future efforts.
3 Questions to answer
Here we have three key questions that maps to three levels of insights to unveil from the open-ended customer feedback.
Question #1: What are the customers talking about?
The first level of insights is about what catches the attention of our customers. Is that the features of the products, or sales process and pricing, or service and experience? With the answer to this question, we will get a quantitative overview of what aspects of the products or services do our customers care about.
Question #2: How do they feel about the products or services?
The next question helps us figure out is how do customers feel. It could be their sentiment of the product experience, or suggestions on a missing product features. The answer helps unveil the second level of insights, which you can use to monitor the performance on different aspects of your product overtime, and verify your efforts on product improvements.
Question #3: What are the most important areas to improve for better NPS (or CSAT or other KPIs)?
After we get the answers to the first two questions, with key-driver analysis, we could get insights on the importance of each aspect against NPS or other KPIs we care about. This third level of insights help us guide the investments going forward, so that we can maximize the output of our efforts on improving our products and services.
Answering the questions above help us connect the dots between the raw customer feedback and trends on customer satisfaction.
And the answers can be surprising - here is an example
It’s important to get quantitative answers for all three questions, or we will likely reach an incorrect conclusion. As an example, let’s take a look at some customer reviews of an electronics product on Amazon.
The product is a popular Virtual Reality headset. It’s a stand-alone battery powered version that doesn’t require a desktop or cellphone to run. We analyzed 349 customer reviews for this product, and here is what we learned.
The chart above contains information that answers the three key questions we mentioned above. Each bubble in the chart represent an aspect that customers mentioned in their reviews. Size of the bubble indicates the number of reviews mentioned this aspect. The x-axis measures the average sentiment of the aspect from customer feedback. And the y-axis measures the importance of each aspect to the overall product rating (customer satisfaction).
Let’s take a look at the answers to the three key questions.
What are the customers talking about? Three top mentioned aspects are App, Game and Experience, with 67%, 49% and 41% of the reviews mentioning these aspects. It shows customers care about the diversity of apps in app store, the overall experience of the device and customers use it often for gaming.
How do they feel about the product and services? Price got the highest sentiment in all the aspects, while Battery got the lowest. It shows that people are really happy with the pricing of this cheaper/standalone version of VR device, but VR devices are power hungry and it’s not doing good on battery life.
What are the most important areas to address to improve NPS (or CSAT or other KPIs)? This is where it gets interesting. Normally we would reach the conclusion that battery life is the most important aspect to improve for the next iteration of the product, as it’s mentioned a lot (33% of review) and the sentiment is the lowest. But with key-driver analysis, we realized where we should focus most to improve customer satisfaction is actually NOT the aspects that are complained most. To maximize the effort on improving customer satisfaction, the product owner should focus the budget the two aspects that are most important to overall rating - better controller experience and video quality, instead of spending lots of resources on the worst performing aspect - battery life (which is indeed expensive).
Why the difference?
This could be caused by the seriousness of different customers. Try comparing these two cases: an already happy customer suggesting battery life could be better, verses a customer on the fence who won’t buy again if the controller is not improved. You get the idea.
Of course the importance of each aspect changes overtime, and it’s important to have continuous evaluations to guide your decisions. This is a process adopted by many enterprises. It produces a concise representation of concrete insights from customer feedback, which help us better communicate with leadership and guide roadmap.
Interested in learning about how do we get these results? Here is the step-to-step guidance on analyzing open-ended customer feedback.
Interested in learning more? Subscribe to our blog here for future posts and case studies.