This week in our poll we asked a very specific question: which is a better way to collect user feedback? There were only two options: over email, or in-app. We had an opinion on the issue, but we didn’t really think that the response would be quite so unequivocal.
100% of respondents, both here and on Twitter, thought that in-app is the better way to capture user feedback.
Our debaters both resisted the question altogether. As their responses suggest, feedback collection is an imperfect science. You can bias respondents in how you ask questions, and you can inadvertently skew results based on when you ask them. Email surveys, often taken well after the behavior you’re trying to understand, are prone to unreliable narratives. But whether you are collecting user feedback using NPS or running a more involved survey with open questions, you’re more likely to reach critical mass with an in-app collection.
Both are suspect
I believe conversion by email will be relatively low. This means lower chances of reaching significant data volume, and the users who do respond are not a good representation. The fact that they were motivated enough to open the email and reply makes them unusual. In-app collection, especially if placed in high intent spots, will have better conversion rate and more chances to reach significance. The problem is, of course, that the specific intent makes it more biased. So I'd say that both systems are not super valid.
VP Product, Wix
I worry that product leaders over-focus on how to collect user feedback, and miss the point that regular qualitative and quantitative data from customers and users is critical and easier than ever to get. Sure, user feedback is likely to be biased, incomplete, statistically skewed, but never let that get in the way of collecting it through multiple methods.
VP Product Strategy, Zendesk