Here are my musings on the relative merits of quantitative versus qualitative research in web development. I hope you’ll find the information of value to your daily work. I’ve conducted various types of research online but it should be clear which method I think adds the most value to our clients’ businesses.
If this post interests you and you want to find out more you can’t beat Jakob Nielson’s website at Useit.com (he would be willing to gain usability at the loss of visual design – but his heart’s in the right place).
If you would like to further discuss Coast Digital’s experience of user reviews and web design work then drop us a line – we’d be delighted to speak with you about your business needs.
Quantitative research
Quantitative research has the potential to give you reams of data and strong statistical significance. It also gives you lots to talk about, but ultimately only provides feedback against the criteria and questions you asked. Ask the wrong questions and nothing of much value will come back.
Quantitative research often requires significant time investment at the inception stage – say the set-up of a user satisfaction questionnaire – but once the survey has been developed it can be rolled out across the internet for minimal overhead.
The beauty of quantitative research is that you can collect all this data and run some statistical analysis to prove that the results aren’t flukes, and, with a little interpretation, draw some conclusions and pretty bar charts. It’s great to wow people with the sheer weight of information but just how relevant is it in web development?
Qualitative research
Those with leanings towards mathematics often believe that qualitative research is the poor cousin of stats-based research; because the sample sizes are small there can be little statistical significance and nothing can be truly proven.
Qualitative research in web development does have its challenges:
- You need to source and recruit relevant test candidates.
- The person conducting the user testing or interviews can unwittingly ‘lead’ users into making statements they wouldn’t have made without prompting.
- Each individual user session takes a considerable amount of time to conduct, record and analyse.
- Expanding the sample size to compensate for a lack of statistics is both prohibitively expensive and needless.
However, what qualitative research lacks in statistics, it makes up for in depth of information and potential for insight. Read on…
Qualitative user review testing – a sound solution for web development
In web development or web redesign work you need specifics; information about areas of weakness or opportunity not generalisations or measures of feelings.
You might feel that you can cut corners by sitting down and brainstorming what features your new website should include and how users should interact with it. And you might think you’ve got some brilliant ideas, but, chances are they’re based on hunches and gut-feelings.
From my experience, if you are tasked with finding out why a website isn’t performing, you can’t beat getting people to use it:
- Set them objectives, goals, scenarios and sit back and watch.
- Do not get too drawn into the tasks themselves; simply sit in as an impartial observer and ask the user to “think aloud”.
- If the user gets stuck at any point in a process on a website ask them “what are you thinking?”
- Make notes, or even better, record the whole session on video and/or screen recorder and analyse the session later.
When it comes to valuable research outputs, you simply cannot beat a summary document highlighting the core areas that already work and those that need additional development work.
This document should be backed up and cross referenced with a copy of the actual videoed sessions. This way you should successfully remove conjecture and personal opinion from your research piece.
In conclusion
When it comes to web development I would encourage you to steer clear of bulky surveys and personal hunches. Instead, strike through the middle; invite a limited set of core users to use your website and observe them. You’ll be absolutely amazed by how many valuable and actionable items their feedback produces.