Thinking and Writing

Recruiting for Usability Tests — Are you Missing a Trick?

I was sitting with my 70 year old mother in law at the weekend doing the usual round of tech support that happens when visiting relatives. Watching her really struggle with her new banking app made me think about the design process and recruiting participants for usability testing.

On the surface the app had a nice and simple interface and I could see all the user journeys that could be carried out and could guess what the next actions they would want me to do. My mother in law however was struggling (and becoming increasingly frustrated) with basic concepts like scrolling below the page fold, disabled button states and the feature tour overlay which, through her eyes, were unclear.

Now I didn’t have anything to do with the design of this app and I don’t know what processes they went through but I can almost picture the project team patting themselves on the back for a job well done as round after round of usability testing sailed through without issue. I wonder though who they recruited for the tests and if by recruiting “safely” with people who are already pre disposed to apps and banking online they may have missed a trick spotting some basic usability flaws in the design?

Over the years I have setup and facilitated usability tests hundreds of times on numerous projects. The participant recruitment spec for these tests is normally based on personas and target audiences and this is either passed onto the recruiter or forms the basis of a screener survey which is distributed through various channels. Standard questions include things like age, gender, technology aptitude followed by a bunch of questions that match your personas or target audience. These screeners are purposely designed to weed out the outliers some of whom may not be ‘digital first’ and therefore may not be our target audience but maybe we should include more of these people in our tests to get more rounded feedback to act upon?

After all if you’re designing a digital product which potentially could be used by anyone with a bank account doesn’t it make sense to cater for the less digitally confident customer and if they can use it so can anyone?