Mobile usability testing: Advice on how to use it

Introduction to mobile usability testing

In March 2011, UK smartphone penetration reached 33%. Another study found that data usage on UK mobile devices increased by over 75 % in the first quarter of 2011.

This means that it is becoming increasingly likely that significant numbers of your customers may want to use mobile devices to research and/or interact with your company. In order to ensure that you offer a compelling mobile experience to your customers, we recommend that you consider mobile usability testing. Mobile usability testing is a way to find out how well your existing (or planned) digital assets perform within the mobile environment. Here, we discuss some of the main factors any business should take into account when considering mobile usability testing.

Do you need mobile usability testing?

During our discussions with clients about mobile usability testing, we have found that there are 3 main factors which help to evaluate mobile usability testing's relevance to a business. Although none of these factors provides a perfect answer in isolation, together they can give you a good idea of whether mobile usability testing is likely deliver significant benefits:

Web Analytics
Look at the data on how your customers are currently accessing your existing digital assets. Do lots of people use mobile devices when accessing your website? If so, this might be an indicator that mobile usability testing might be worthwhile. Customer profile
Are your customers the type of people who are likely to be using mobile devices as a means of accessing data, information and services? In our experience, the main user groups that currently tend to use mobile devices in this way are younger users (15 24) and well-educated professionals. Content type
Some types ofcontent or services seem to suit the mobile environment better than others. During mobile usability testing sessions, we have found that time-critical and location-dependent content tends to be most popular on mobile devices. We have also found that people tend to use mobile devices for short, discrete tasks rather than long research-focused sessions. Do you have content with these characteristics?

Of course, there are many other factors in deciding whether mobile devices are important enough to your business to justify an investment in mobile usability testing (such as the likely profitability of mobile device users). However, we have found that the above-mentioned factors are a good way of beginning a conversation about the potential business value of mobile usability testing.

Best practice still applies to mobile usability testing

Although there are some special considerations when designing and conducting mobile usability testing sessions (which we will talk about later), many of the same skills apply. Here are some of the major areas which don't really change:

Tasks
It's still crucial that you define tasks for mobile usability testing which are relevant and valuable to your business and its customers. These tasks must, of course, be communicated to the test participant without any cues which could bias the usability testing. Recruitment
The importance of participant-recruitment might be said to be even higher in mobile usability testing sessions (when compared to more traditional usability testing sessions). The mobile audience contains many niches (based on, for example, participants' experiences of different mobile devices and services) this makes targeted recruitment essential. Talk Aloud
This method of usability testing where participants are encouraged to vocalise their thoughts during a usability testing session is still very relevant to mobile usability testing. Encouraging usability testing participants to vocalize their thoughts during a session remains a key factor in mobile usability testing.

At its heart, mobile usability testing sessions still involve human beings interacting with a piece of technology in order to achieve a goal. As such, it's unsurprising that many of the same usability testing techniques still apply - irrespective of whether it's a mobile usability testing session, or one based on a PC.

Field testing in mobile usability testing

A big difference between mobile devices and traditional desktop PCs (and even laptops) is that mobile devices are used in far more diverse and challenging real-world environments. This means that factors such as lighting, noise and location must be taken into account for mobile usability testing.

These extra factors are a potential argument for testing in genuine real-world locations (known as field testing). However, it is important to recognise that although field testing can be very valuable, it can also add significant costs to mobile usability testing sessions.

It's interesting to note that a recent study found that lab-based tests gave "sufficient information when testing [the] usability of a mobile application".[1] However, the study's authors highlighted that the application they tested was relatively simple and did not depend on a participant's location (it simply allowed participants to transfer files between remote computers and the mobile handset). In conclusion, the study stated that "more tests with different kinds of applications are needed.especially for applications with location information, [where] the laboratory setting may not give the best outcome."

Realistically, there is never going to be a single approach on how and when to use field testing that always delivers the best results for mobile usability testing. It is far more likely that good decisions about planning mobile usability testing sessions will be made on a project-by-project basis, according to a company's content and their customers' profiles. In our experience, a good use of field testing can be at these phases in a mobile usability testing project:

Beginning - identifying issues

End - validating design decisions

Using field testing in this way can often deliver the best Return On Investment, with laboratory-based tests being used iteratively within a mobile design project.

Devices in mobile usability testing

Another difference between mobile usability testing and its desktop/laptop counterpart is the far wider range of mobile devices with very different user experiences. This issue must be considered when planning your mobile usability testing. To a large extent, your choice of devices for a mobile usability testing session will determine its success (because you are less likely to improve an experience which you do not test)

In general, we recommend that you base this decision on device popularity (with your audience) and the disparity between the contender-devices' user experiences. Good inputs to this decision on your mobile usability testing plan include:

Web analytics

The devices which people are currently using to access your digital assets should be covered in your mobile usability testing.

Consumer profile
Which devices are your customers most likely to own? This can often be estimated from their age and income.

Instead of focusing on 1 or 2 devices, we usually recommend running mobile usability testing sessions across the 3 main categories of mobile device. Planning mobile usability testing sessions in this way affords a good coverage across most devices. The 3 main categories of mobile device are:

Touch screen mobile phones Non touch screen mobile phones Tablet PCs

When planning a mobile usability testing session, you will probably need to choose a device from each category. This decision is normally based on devices' popularity within your customer group and how generally representative a device is of its category.

Observing and recording in mobile usability testing

It is more difficult to observe and record a mobile usability testing session than a PC counterpart because smartphones do not (currently) allow you to run screen-recording or screen-sharing software in the background during testing sessions.

A good solution to this problem is to mount a camera onto the mobile device for the mobile usability testing session, with the camera looking down onto the screen and the participant's fingers. Because the camera is mounted onto the mobile device, the participant should be able to hold, move and use the device in a natural way. The moderator of the usability testing session would then view (and record) the camera's recording through a PC monitor.

Note it is important to run a practice mobile usability testing session in order to make sure that the camera's focus is fixed on the screen (and if possible also gives you good detail on the participant's finger movements).

Summary - mobile usability testing

Rising smartphone penetration is making mobile usability testing more important for many businesses, but it' still a good idea to check whether it is relevant for your company at the moment. Although many of the skills involved in planning and conducting a mobile usability test are the same as for traditional desktop-based usability testing, there are some differences with mobile usability testing. The main differences involve the greater potential benefit of field testing, including each mobile device category within your testing and using a device-mounted camera to observe & record sessions.