We asked 4 very simple questions, plus 1 free-form text comment field. The email was short, personal, and invited them to email me directly if they wanted to.

Response was fantastic! 27% filled out the survey, about one-third of those filled in text comments. A few emailed me.

I followed up each one with an email myself, whether or not they asked, and that led to many great email and telephone conversations.

Communicating with customers and “near customers” is really enjoyable and educational, not to mention something they really dig, too!

Don't over-survey and keep your surveys short and sweet (and useful, of course). Make the email that leads to the survery personal and also short & sweet, you much assume the recipients have a busy day!

As for who to send and when, we actually built up an automated system that is triggered based on certain activity signatures. Any user who hits that pattern will get the notification. (I'm about to add an improvement to only send each notification once per user, in case you hit the same pattern multiple times.)

It's actually taken a lot of trial and error to refine, but the results are well worth the effort.

I agree, I find direct email communication with people to be a fantastic way to do in-depth development. I wish I'd done more of my early development surveys at Involver in this format (I also wish I'd done more of them in general!). Do you send these types of emails to all users? How do you decide who to send these to?

I still use surveys often when making a specific choice (i.e. how do we prioritize building feature a against building feature b) and dealing with a sizable userbase. I love surveys here because compiling email responses into a sorted data set gets VERY time consuming at scale; it can often add days to your decision-making time.

Food for thought: embedding survey-style questions directly into your product could be a great way to get well sorted and actionable data without wasting your “click budget.”

- Make it easy for them, even if it's hard on you.
- Don't gather data, start conversations.
- Let them set the topic.

In particular, though this seems counterintuitive and against pretty much everything you'd read: I'd strongly, strongly recommend *against* sending out surveys. They have so many problems, some of which include:

- They typically require you to click a link. As any web developer knows, people are loathe to click links; assume at all times that no matter how valuable you think a link should be, 90% of people just won't click it. Assume you have a very tiny “click budget”; I find getting two clicks out of someone (“Reply”, and “Send”) is hard enough.

- They're impersonal. The very notion of being surveyed is de-humanizing. It conveys “you're just a datapoint to me”. Instead, send your questions from a real person, with a real name attached, in the actual language you'd use as if you had just naturally written them a personal message and are genuinely interested in their response. Because if you're not genuinely interested, that will be apparent and people won't bother.

- They're optimized for the receiver, not the giver. The whole point of a survey is to normalize data into a way that is easy to read. It makes the giver do all the work in parsing messy real-world concerns and confusions into a multiple choice bucket. The real world is too complex for that; it's actually *harder* to fill out a survey than to write a multi-page email describing your problem, because you're never forced to choose between options that don't quite fit.

- They only tell you what you ask, but that's rarely what you need to hear. The only filed in the survey that really matters is the open-ended one; everything else is a distraction.

- They get out of date. Surveys take time to create; the questions you want to ask change over time. Even if the survey was perfect for you yesterday, it probably won't be today. Just leave it open ended and people will always tell you relevant information, without needing to update the template.

Anyway, I swear by the “just send an email and let them do the talking” approach. It's worked incredibly well for me. And it's a lot more fun to talk with users directly than to just stare at charts and wonder “is this data even statistically relevant?”

In our surveys of our beta users, we kept most of the questions short and to the point (most multiple choice but we allowed for optional input). We were excited to see such a high response rate- and the input helped us smooth a rough spot out in our design flow.

Good luck out there, and good testing! :)