Photo by Seattle Municipal Archives
I received the following question from a reader a few weeks back:
I’m considering creating a mobile app and I want to know quick/effective ways to validate some of my assumptions. Is it more effective to put out small experiments that test your assumptions, or are surveys of the possible users a better approach?
My answer: it depends on what you’re trying to test.
In general a survey is going to give you decent results for little effort. A survey takes 20 minutes to put together (SurveyGizmo or SurveyMonkey), email it, and you get some info. It’s a simple approach that doesn’t take a lot of time.
However, it’s nowhere near as accurate as putting out experiments. Experiments that ask someone to buy something, sign up for a list, or perform some kind of behavior are the only real way to know if something works. But these kinds of things take so much longer to put together that you have to balance this level of effort with the value they provide.
If you had unlimited time I would always recommend experiments. But surveys save you time, so you’ll inevitably have to rely on them or you’ll never start building your app.
Two things you can’t get from surveys: actual conversion rates and pricing info. You can ask “would you sign up for this list” or “would you pay $x for this” but these are useless questions.
If you need this kind of knowledge, do an experiment. For most other information, surveys can be a solid, time saving approach.
I’m interested to hear your thoughts in the comments.