At GoSquared, we do user testing on a regular basis, and we have done so for years.
After chatting with lots of people from other companies, we recently started to realise there were a bunch of learnings we’ve gained over time, so we thought it’d be great to get everything we’ve learned about user testing written down in one (surprisingly big) blog post.
Happy testing!
Why do user testing?
Putting your product in the hands of real world users is the best way to validate any assumptions you’ve made during development. Test early and often. It’s cheaper and easier to correct assumptions early in development than after something’s already shipped.
User tests provide in-depth, candid feedback about your product. They’re also a great opportunity to do some market research and find out how well your product/pitch lands with your users.
Another benefit is that it’s the most reliable way of identifying ‘wow’ moments – the parts of your product that make a user go ‘wow’ and should be highlighted in your marketing material.
Having said all this, how do you actually go about conducting a user test? The internet is full of advice to help craft the perfect testing methodology. However, in our experience what holds people back, is not what to test, but the practical aspects of setting one up. How do you find a user? How do you record your findings? How long should they last?
This post will cover what works for us here at GoSquared.
Finding a user to test with
Finding a user is obviously a pretty crucial part of conducting a user test. However not all users are created equal. You need a clear understanding of what you’re trying to test and who the intended audience is. For example, don’t invite a sales person to test your new developer platform!
From easiest to hardest, here’s how we find people:
Ask a colleague – this is super low-effort and if you’ve never performed a user test before it’s a good way to practice with somebody sympathetic to the situation. They already know where your office is and they work for the same company which immediately gets rid of any red tape you may have around confidentiality. The ideal person would be somebody that had minimal involvement the development of the feature you’re testing. The only downside is that the feedback won’t necessarily be reflective of real world users if it comes from someone privy to a lot of your internal business or product decisions.
Ask a recent hire – for many of the same reasons as above. If you ask them to be part of test early on it’s a great way to familiarise them with the product and hopefully they they’ll still be free of any internal preconceptions and biases.
Utilise your work network – grab somebody from your shared office/workspace or ask somebody from a meet-up. They’ll be a greater degree of separation further than a colleague so hopefully will be able to provide a fresher perspective. You can always return the favour by doing a test for them in the future.
Ask an engaged user – hopefully they’ll be willing and enthusiastic because they get to give direct feedback on a product they already love. This is valuable feedback because they’re the people who will actually end up paying for the feature they’re testing!
Find a member of the public – use social media like LinkedIn or Twitter. Ask them not to research your product beforehand so you can observe their unbiased reactions during the test. The less they know about your product/company the better. Be warned that the quality of people that respond will wary wildly so you might need to offer an incentive for taking part (such as an Amazon voucher).
Once you’ve got a shortlist of testers, have a quick call with them to screen their technical capability, understand their current job role and asses their knowledge of the competitive marketplace. Ask them to avoid doing any research on your company before the user test. As a practical tip, make sure they know how to find your office!
Preparing for your test
Do you have consent to record the test?
Not everybody is comfortable being recorded so it’s best check ahead of time. Reassure them that the recording is purely for internal use only and won’t be shared publicly. You don’t necessarily need them to sign anything, getting consent over email (so you have it in writing) is usually good enough.
Ask the user for discretion
The product you’re testing is likely rough around the edges and not a true representative of its final form. Ultimately, it may never make it to market. Depending on how sensitive the product is and how much you trust the person, you might need a formal NDA (Non-Disclosure Agreement). If this is difficult, that shouldn’t put you off testing existing, publicly available parts of your product with them anyway.
Find somewhere quiet
Book a meeting room or find somewhere quiet where you won’t be interrupted. Wherever you choose, double check it has a decent wifi signal too!
Get two laptops
You’ll need two laptops, one for your notes and timekeeping, and a second for the test subject to use.
Test data
Depending on what you’re testing you may need to prepare a testing account with some dummy data in (unless of course you’re testing your onboarding in which case you may well want to observe them going through the account creation process themselves). Prototyping tools like Marvel are great for testing interaction and UI choices in the early stages of development.
Prepare your questions and/or a script
Standardising your test procedure and questions is essential if you want the test to be repeatable with multiple people. Don’t ask leading questions, keep them open-ended so the user can explore the answer more naturally. Check your timings so that the test won’t overrun or end up incomplete. It’s also a good idea to have a few non-essential bonus questions prepared so that you can keep your user busy if you’re under time.
Learn how to stream to Youtube
Setting up a private Youtube stream is a really handy way to record the user’s screen and have a video that is immediately backed up and ready to review after the test. This is a barebones quick guide to setting it up:
- Log into your Youtube acocunt on the test subject’s laptop.
Head to the ‘Creator studio’ and find the live events section here - Create a ‘New live event’ and tweak the privacy settings to that the stream is either private or unlisted (only people with a link to the video can view it). Set the ‘Type’ to ‘Quick’.
- When you click ‘Go live now’ it will open a new Google Hangouts window where you can test the microphone and share the laptop screen with the stream.
- Hit ‘Start Broadcast’ in the Google Hangout window to actually start recording.
We tend not to use the laptop webcam. Most laptops will struggle to record the screen and the webcam at the same time and your test subject will probably be more comfortable without them being filmed.
Tell the team to watch
Whether they watch the live stream or review the video later, your team will always pick up on extra things you missed and interpret feedback differently. If they’re watching along live, they can even prompt you to dig deeper on a specific interaction whilst the test is still ongoing.
Basic structure of the test
Every user test will be different depending on what you want to test, however there are some basic things that we think every test can benefit from. We aim to keep our user tests under an hour long so that the user doesn’t lose interest. Also – don’t forget to start your Youtube live stream before you begin!
User background – 5 minutes
It’s worth conducting a brief interview with the user so your team can contextualise any feedback. Ask about their background, existing product knowledge, job role and technical ability. What products/tools do they currently use? How much do they pay for them? Have they used your product before? What problems do they experience in their current role?
Set expectations – 5 minutes
Before jumping into the test, it’s extremely important to set the right tone, outline your goals and what you expect of the user during the test.
The test is for you to observe and record their genuine reactions to your product. It’s only going to be used internally and won’t be shared outside of the company. You need to encourage them to think aloud, verbalise their actions and be as candid and honest as possible – you won’t get offended by any criticism they make.
When something isn’t clear to the tester, encourage them to ask questions, but let them know you won’t necessarily answer most of them. You’re not deliberately being unhelpful, but in a real-world scenario the user wouldn’t have a product expert on hand to answer their every question.
Let them know that whatever happens, they can’t mess up. If they’re making mistakes, it’s your fault because the product/site hasn’t communicated the solutions well enough.
Explain the scenario – 2 minutes
Whether you found the perfect user or not, it can be helpful to give them a fictional scenario or role to help them frame the test. For example, they could be working for a small online business of 5-10 people and their boss has asked them to evaluate marketing tools.
Marketing site first impressions – 10 minutes
Unless the whole test is about your marketing site, it’s worth spending at least ten minutes at the start of a test to get candid feedback on it. After all, if your marketing site isn’t up to scratch you won’t need to worry about the usability of product because you’ll have nobody signing up to use it!
Have a web browser open on their screen showing a blank tab. Have your site open in a second, background tab ready for them to switch to. Ask them to browse the site for 30 seconds then switch back to the blank tab. Then follow up with these questions to see if anything stood out:
- What do you think the product does?
- Who should be using the product?
- What problem does it solve?
Then repeat the the exercise but give them free roam of the entire site for two minutes. Pay attention to the path they take, encourage them to describe why they clicked on certain pages and what they’re currently viewing. Good follow-up questions are:
- How would the product help their business?
- How much would it cost to use the product?
- Would you sign up for this service?
- What additional information would you need to convince you to sign up?
- Who do you think our closest competitor is?
Pay close attention to the language they use in their answers. This will help you evaluate if your tag line resonates, if they’re picking up on the right terminology, and if they’re mentally placing your product in the vertical/category that you’d expect.
The main test – 30 minutes
We try not to artificially sandbox the user within our product. We want them to behave exactly as they would in the real world.
Unless you’ve created the perfect product first time (rare), it’s highly likely your users will get stuck and struggle at certain points. The point of the test is to highlight these points so you can mitigate them in the future. However, it’s also extremely valuable to see how a user tries to deal with the problem during the test.
Some people will look for an answer on your site, perhaps they’ll ask for help via live chat, maybe they’ll google it. Others will simply give up. Observing this process is extremely valuable: it highlights discoverability issues, shows you the language people use to google you, and what sources of information they trust to give them answers.
Wrap up and feedback – 8 minutes
This brings your total testing time up to one hour. Use the last few minutes of the test to thank the tester and ask them how they thought the test went. Did anything particularly excite or frustrate them? Would they use the feature or product when it’s released? How does it stack up against competitors’ solutions? Would they pay for your solution? You can also use this opportunity to ask questions from any teammates you had watching the video stream.
Start user testing today!
That’s pretty much all there is to it. Hopefully it doesn’t sound too complicated. This is what we’ve found works for us, but obviously every product is different.
The best way to find what works for you is to try it out a few times and see for yourself. Got any other suggestions of tips for running user tests? Let us know on Twitter.
Now go forth and test!
