All posts by Mat Walker

Using Skype for Moderated Remote Usability Testing

By | Research, Uncategorized, Usability Testing, UX | 8 Comments

I used Skype and Ecamms Call Recorder for Skype to conduct moderated remote usability testing around the world on a clickable Marvel app prototype. Heres how I got on.

I’ve been working for a Japanese sportswear company who wanted some usability testing done on a new feature they were planning for their fitness app. As the app has a healthy (pun intended) global user base they were keen to test with users from around the word which mean’t remote usability testing on a clickable prototype put together in Marvel app.

When I’ve conducted moderated remote usability testing in the past I’ve used screen sharing services like Webex and GoToMeeting but I’ve always found the requirement on the end user to install various plugins troublesome and the cost (at the time) was prohibitive.

My loose set of requirements this time around was:

  • No plugins to install
  • Low cost
  • Reliable
  • Record screen, audio and video of the participant

I settled on Skype in the end as its pretty ubiquitous these days and although the app would need setting up and installing the chances were that a lot of respondents would already have it up and running.  Because I had a few thousand potential participants I took a gamble and was explicit in the screener recruitment survey that we would be using and Skype on a desktop or laptop and this was a requirement. Despite this I still got a 20% response rate which was better than I normally get for face to face interviews (what also helped was a fairly loose set of requirements for participants and also doing the tests over the weekend).

Skype doesn’t have an inbuilt recording capability but there are a few plugins that can allow this. I used Call Recorder for Skype by Ecamm and at $29.95 was pleased with the results.

What went well with the testing:

  •  Low barrier to entry mean’t that I got a good response and including Skype as a requirement didn’t seem to put people off
  •  Not having to mess about with talking people through installing plugins on the day took a lot of the pressure off me and allowed me to focus on the testing
  •  The call and video quality was excellent so I was able to include excerpts in the final presentation deck for the client
  •  Conducting the testing at the weekend meant I got a good response and because it was remote meant I could spread it out over a couple of days without having to hire meeting rooms
  •  People seemed more at ease with the test and opened up a lot quicker because they were at home
  • It was cheap to run compared with hiring meeting room space

What went less well:

  •  Skype wasn’t quite as flawless as I hoped. Two out of the five couldn’t share their screen without Skype crashing.
  •  I also had the call drop a few times for no particular reason which was annoying and interrupted the flow

So would I use Skype again? Probably yes. Despite its flaws it was pretty easy to use for the participants and although its no substitute for face to face interviews it was still pretty effective and I got lots of good feedback. Conducting it over the weekend wasn’t ideal (for me!) but mean’t I could talk to more people and turn around the test results quickly.

5 years on … Lessons learnt from my time in the freelance UX trenches

By | Freelance, Portfolio, UX | 2 Comments

Five years ago this month I quit the comfort of my full time job and went freelance. I often get asked  about how to go about becoming freelance so I thought I’d mark this anniversary by listing a few things I’ve learn’t upon the way (in no particular order):

1. If you’re going to go freelance do it properly

When I left my full time job I spent the first 6 months not really sure whether I wanted to go perm or go freelance. The result was a bit of a mess of me half applying for jobs and half committing to the freelance life. In hindsight I should have focused more on the freelance thing and set myself a deadline where I could reassess my career.

Remember you’ll also need a few months money to live on while you wait for your first invoices to be paid. I had some inheritance so I gave myself 3 months to make a go of it. My plan was that If after the first two months It all went horribly wrong I’d still have a month to find a permanent job. I still have that money in the bank.

2. Find a mentor

I’m lucky that I know a lot of freelancers. When I first started out I found it really useful to meet up with them from time to time and pick their brains about freelance life and what I should be doing. Its worth getting to know some freelancers (doesn’t matter what line of work they are in) just to sense check your decisions and sound them out for advice.

3. Get a website 

I’ve had a crappy website for years. This year I spent a bit of downtime at the beginning of the year updating it to something only slightly less embarrassing. The result of which is that whereas before I didn’t get very many work enquires from it I now get 1 a day on average.  Wish I’d done it years ago. I bought a WordPress theme and hacked it about to meet my needs but you are probably much better at coding than me. Have a look at other freelancers websites for inspiration.

4. Get a (meaningful) portfolio

There have been plenty of people blog recently about the value of UX portfolios but I can talk from experience of having been on both sides of the hiring table that it is worth spending time on one. Just remember that nobody is interested in a list of projects with shiny screengrabs and Photoshop mockups. What hiring people want to read is a story about the projects you worked on starting with the objectives, what you did on the project and the outcomes. (I’ve got a longer blog post drafted on this which I’ll save for another time)

5. Use LinkedIn

I have a love hate relationship with LinkedIn but the fact is that I get a lot of work out of it so spend the time honing your profile. Think of the kind of skills people will be looking for an get those key words in your profile. Also keep the copy tight and to the point and use bullets to highlight key skills.

6. Network Network Network

Get along to as many networking events as you can. The UX (and web industry generally) is a friendly space to be in so get to know people as you can. I quite often pass work onto people I know if I can’t take it on and the reverse has happened to me as well. It also doesn’t hurt to do a few talks and get a bit of a name for yourself if you can.

7. DON’T PANIC

Some months are lean some months you’ve got work coming out of your ears. You can’t predict how much work you have so don’t bother trying. The only things I noticed is that from mid December to mid January things are quiet and also things quiet down a bit in August. Also remember that you are not a machine and take time off. Work will always be there when you get back.

8. No job is ever guaranteed

I’ve learn’t this the hard way but it doesn’t matter how many contracts you’ve signed no work is guaranteed so be prepared. I’ve had projects stop a couple of days before they were meant to begin and half way through (nothing to do with me!). Unfortunately it comes with the territory so make sure you’ve always got a couple of months money in the bank just in case it happens to you.

9. Keep track of your finances

Its much easier to sort out invoicing, expenses and tax as you go along rather than leave it to the last minute. I try and keep a 1-2 of hours a week to one side just to keep up to date on admin. There are lots of finance packages out there but I recommend Freeagent (heres a referral code if you fancy giving it a try for a 10% discount – 33n0dtl2). I know some people use spreadsheets but for the money its worth having something like Freeagent to streamline the process and leave you a bit more time for the more important things in life.

10. Set your rates

Don’t ever try and be the cheapest. Sell yourself on producing quality work and set your rates accordingly. I have a sliding scale of rates depending on who the client is, what the job is, where its based and how long its for. Having said that I tend not to worry about money too much. I do this job because I genuinely love it and would rather do an interesting project than work on something well paid. If your main motivation for becoming freelance is earning loads then there is plenty of well paid work in the finance and gambling areas.

11. Get your game face on 

You’ll be expected to hit the ground running when you start a new project and the people in your team will be looking to you for the answers. I rarely get to work with other UX people (when I do its a treat) so nobody will be there to cover your back. If you are serious about becoming freelance make sure you know your trade inside and out. There will be lots of occasions where you will need to learn as you go along but when you do this make sure you’re not making too much of a leap into the unknown as it will come back to bite you.

12. Dont be the Flash guy

Remember Flash? Don’t be that person who still only does that one thing. Try and keep ahead of the game and keep your skills sharp by reading blogs, following the right people on Twitter and going along to talks and conferences.

13. Document and record everything

I generate a lot of paperwork and sketches. I try and keep a record of everything I do for my portfolio and to evidence my work in case anyone ever asks why certain decisions have been made. I don’t keep hard copies but I photograph everything and file it away. You never know when it will be useful.

14. Find your preferred way of working

Theres no right way of working or type of project, its whatever suits you. Some UX freelancers prefer to work on retainers or will only work from home. I prefer working on bigger projects full time and generally work onsite. Its just whatever works for you.

So there you go, a few things I’ve learnt along the way. I hope if you are planning on making the switch you found this useful. At the moment there is still plenty of work to go around (especially for experienced people or those with specialisms) but the market is becoming a little crowded as more people make the switch. Fortunately as the industry becomes more established so does the quantity of work available so its still a viable option.

There are probably lots of things I’ve forgotten so feel free to ask and buy me a beer sometime and I’ll tell you all the sordid secrets!

Good luck!!

Some notes from interviews about iPads and social media with 11-16 year old school kids

By | User Research | No Comments

I’ve been doing a lot of work with children recently, working on a new app for schools. As part of this project I’ve interviewed around 30 children aged 11-16 from a state funded secondary school in Essex.

One of the many things that makes this school unique is that all students are given an iPad to use for their studies. Students are also permitted to take the iPads home and use them as part of their studies. There is little restriction as to what they do and install with the iPads although most social media sites are blocked by the school firewall.

Amongst other things we’ve talked about their use of the iPads and use and attitudes to social media. Rather than lose those notes I thought I’d jot them down here for my own future reference and for anyone else whose interested.

 

Students and social media

One thing that kept coming up in our discussions was their use of social media. Younger children especially didn’t see the need for platforms like Facebook at this stage of their lives. This quote from one of the students sums it up well:

“I don’t use Facebook because I don’t see the point. All the people I know are in the same town as me and in my class, I see them every day.  I want to know what they’re doing now not last week”

There was also talk about Facebook being a platform you’d use to talk to your relatives not your close friends but may become more relevant later in life when friends go separate ways after school.

Instant Messaging

Instant Messaging is the main form of communication with the young people I spoke with. BBM, iMessage, WhatsApp, Snapchat and SMS are the prominent platforms mainly due to their low cost and high adoption rate with their peers.

A picture paints a thousand words

So the saying goes and its true with the students I spoke with. Instagram was *very* popular amongst the group and the social media platform of choice. Along side capturing pictures of day to day life students were also using Instagram as a quick way to talk to their followers. This is especially true when they want to convey an emotional state to their followers which would require more effort to do as a Tweet or Status Update (For example a self portrait grimace or sad face). The conversation with their followers then continues within the Instagram app.

 Viral distribution of apps

Teachers told us anecdotally that a new app will spread very quickly across the school on various mobile devices. The most popular apps for this are games and they are spread via word of mouth. The apps which spread quickly also tend to be forgotten about just as quickly as they appeared as the next big thing comes along. Cost is also a driver with new apps, free and very cheap ones are more likely to spread for obvious reasons.

Buying and installing apps

Students are also responsible for what they choose to install on their iPads. They all have their own Apple accounts and are permitted to buy whatever they want. The main barrier to installing paid for apps is due to them not having debit or credit cards so they use iTunes gift vouchers instead.

iPad usage over time

On talking to students from various years and observing them during break times it appears that in the first years of secondary school the iPad is very much seen as a novelty and gets used both for games and study. As the students get older however the novelty of the iPad drops off quickly and it becomes a tool to help with studies. Games get uninstalled and a core set of apps are used regularly. Free apps are the most widely used.

Outputs from a session looking at favourite apps in school.

Outputs from a session looking at favourite apps in school (click to enlarge).

Personal smart phones vs School iPads

Most students we spoke with also had smart phones (mostly cheaper Andriod devices). These devices are taken to school and are used to communicate to friends with. They are also popular as they use 3G data connection and therefore bypass the school firewall because they don’t use the WiFi.

 iPads vs Paper

Students at the school could do project work on the iPads or on paper. Opinion was divided on which was better and came down to personal preference. Students who preferred iPads for project work cited the following reasons:

  • Ease of use in collating information,
  • Easier to work collaboratively
  • More ‘professional’ looking project work

With students who preferred more traditional methods the most common reason they spoke about was a distrust of the iPad to keep work safe. Some spoke of having bad experiences in the past with lost project work which led to this distrust.

The iPad as a distraction

When I’ve been talking to people about this project one of the first thing they generally say is whether the kids find the iPad distracting. From what I’ve been told by the teachers the answer is no. The students are allowed to listen to music during lessons (on headphones) which doesn’t appear to be a problem and they have to use the iPads in a lot of their lessons for research. There are however times when the teachers have said that the iPad has been a distraction (for example using iMessage is the new passing paper notes around the class) but the teachers also said that if they weren’t using the iPads they would be doing something else.

Bullying

When the subject of online bullying came up teachers said that it does happen on occasion but when it does the kids who have been bullied are encouraged to screen grab anything they find offensive and report it to the teachers.

 

Hope you find these notes and observations useful and if you have any thoughts or questions please do leave a comment or drop me an email.

The problem with Lean UX is …

By | Agile, Lean, UX | 2 Comments

The other day I wrote a quick response to the question ‘What are the weak points of the Lean UX design methodology?’ on Quora. A couple of people have asked me to explain a bit more about my post so I thought I’d elaborate a little.

The app I’m currently working on is focused on developing a Minimal Viable Product (or MVP)  for use in schools. Unfortunately the project has had strict time and budgetary constraints imposed on it so, from the outset, there was a real pressure to deliver as many stories as possible in the given time. (In a true Lean environment MVP wouldn’t be a deliverable as it is here but a state of flow but thats the situation we have.)

Due to the restrictions of this project its concentrated a lot of the Lean UX implementation headaches into a short period of time which prompted me to write this post.

One of the many problems with the ‘feature first’ approach is that in order to include as many features as possible any UX and design consideration is stripped so far back whats implemented could have a negative impact of the usability of the feature in question. While none of the changes made are huge compromises the cumulative effect can be damaging to the experience of using the product.

The other problem is that in order to deliver as quickly and efficiently as possible any delighter or exciter elements often get removed in order to fit the feature into the sprint. Once that happens there is almost zero chance of it ever getting retrofitted.

Andy Budd summarises the problem well in this blog post from a couple of years ago

“In the rush to deliver a minimum viable set of features (the threshold elements in the Kano model) we often ignore the elements that make the product really great (the exciters and delighters in the Kano model). As quality gets stripped back in preference for functionality, we slowly see our products become ever more minimal and ever less viable. It’s very rarely one thing that does it. Instead MVP often turns into death by a thousand paper cuts.”

In order to mitigate this problem each story or feature should come from the users themselves and should have a hypothesis which we, as a team, should seek to validate.

For example say I wanted to introduce a new brand of ketchup to the market. The hypothesis could read something like:

“We believe that lovers of spicy food will want to use our new brand of super spicy ketchup as a condiment with their food.

We will know this to be true when:

– We see a week on week increase in sales of our ketchup in supermarkets

– Our customers tells us they like the taste”

(not a great hypothesis but you get the idea!)

To continue with this rather clunky analogy – If we imagine that our ketchup is in the wild and our test supermarkets put it on a high shelf, out of reach for most people, we would probably find that hardly any ketchup is sold. From this you might assume that no one wants the new ketchup and the product is a failure. If however you put the same ketchup on a shelf which is easier for people to reach then we may find that more people buy it and our hypothesis has been proved correct and people love super spicy ketchup.

The point being if the feature we our trying to introduce isn’t given a fair chance by being implemented as well as it could be then we shouldn’t be surprised if hardly anyone uses it and its widely ignored.

So whats the solution? Well the bad news is there isn’t a silver bullet which will cure all the ills, more a few things that, if implemented well, may help.

  • Better integration between the devs and UX. This first one is a no brainer. Work together to design easily implmentable features with minimal overhead.
  • Spend more time talking to the PO so when they are grooming and prioritising the backlog they have the full picture and understand what the implications of their decisions are.
  • A key hypothesis for new features. What problem is this feature we’re implementing solving and how will we know when its succeeding? Include a hypothesis not just for the product but for each story and also include a KPI to test that its doing what its supposed to. Which leads nicely onto…
  • Once a feature is live keep testing it. Keep questioning its validity and seeing how the experience of its usage can be improved for the end user. Schedule points to revaluate features and see how the experience could be improved through iterative design.
  • Ditch the burn down chart. Burn down charts are only useful for management types who like graphs. The proof of progress should be in the features and stories which you’re pushing live which users using. (Theres a blog post in there somewhere but I’ll save that for another day!)
  • Continuous deployment – Don’t wait till the end of the sprint to release new stuff. Release early, release often.

Theres probably a bunch of other things that you could do but thats all I’ve got for the moment.

For the record I love Lean and Agile as a process for designing and building products, it just feels that sometimes in the excitement of getting stuff out there, we lose sight of the detail which separates the ok products from the great ones.

Free alternative to Silverback/ Morae / Camtasia

By | Usability Testing, User Research | 4 Comments

I was doing some usability testing last week and, due to a glitch on my Macbook Air*, wasn’t able to use my regular screen recording package of choice Silverback.

Time was against me so after a quick bit of Googling I came up with a free alternative to Silverback using some of the built in functionality with OSx namely Quicktime. Now when I say its an alternative to Silverback (and Morae or Camtasia for that matter) there are some draw backs:

  • The first being that while Quicktime can record screen movements and audio it doesn’t do the picture in picture video stuff that Silverback does.
  • The other minor drawback is that Quicktime produces some pretty hefty files before you export them so make sure you’ve got plenty of space on your hard drive.

It really is worth shelling out for Silverback but if you’re stuck (or short on funds) Quicktime does work at a stretch.

Heres how I used Quicktime for recording user testing sessions:

1) Open Quicktime then go to File -> New Screen Recording

Screen Shot 2014-01-17 at 15.24.17

2) Click the little drop down arrow on the right hand side of the popup window and choose the mic and also ensure that “show mouse clicks” is ticked if you want to emulate the same click effect that Silverback does.

Screen Shot 2014-01-17 at 15.25.13

3) Hit the red record button!

I do very little video work so this post may well throw up lots of “well, yeah, durr” comments but it was new to me so I hope it helps out someone else in the same predicament. Below is a little demo video I recorded earlier.

Quicktime screen recording demo

*NOTE: The glitch was a plugin problem but I’m also aware that at the time of writing this Silverback doesn’t work with new Macbook Airs or Macbook Pros so Quicktime may also be useful as a short term solution here as well if you’ve not already got a Silverback licence until the next version of Silverback comes out.

Looking for UX design and research help with your project? Get in touch