Anticipated use – actual use – after use

UX designers look at the actual use creating satisfaction for users – efficiency and effectiveness

• Satisfaction – tested through questionnaires but efficiency and efficacy can be tested through user tests 
• Efficiency – is it quick and easy 
• Effectiveness – can it be done 
• These will cause an increase or decrease the satisfaction level 
• Which can therefore increase or decrease a users reported experience level 
• The main thing to look for when user testing is efficiency and effectiveness 

You will find other things but don’t ask about them 

Tasks 

• Tests at the level of the task you want them to achieve 
• The problem area would be the sub tasks – working through the website Test the whole process
• Walk them past the known problem area
• What does the user need to do at a really high level that’s nothing to do with the interactive system? 
• Ask a question for the task – e.g. what would you do if you needed to buy a T-shirt with delivery on or before Friday 
• Mirror real life as much as possible 

Be broad with questions – don’t ask specific questions 

Inappropriate tasks 

• Pretender tasks – e.g. “you’re a bank worker from Spain”
• Silly test tasks – e.g. “when using the checkout can you use the name mr banana man”
• Derivative tasks –  e.g. “let’s assume that by mistake you added two products to your basket” 
• Tests tasks with clues –  e.g. “what is the opening time of the cafe in cafe”
• In precise test tasks –  e.g. “find and read the terms and conditions” 

Appropriate things to say in a user test 

• What would you like to do next 

Say as little as possible

• impartially 
  • We must not agree or show any hints to the user as to what we think 
  • Do not nod or smile
  • Do not give clues
• No leading questions 
  • We must be careful with our wording 
  • Do not bias your results 
• Put emphasis back on the user 
  • What would you do next 
  • What are your options now 
  • I’m sorry I can’t help you as you don’t want to influence what you do next 

Low and high fidelity 

• We should test out designs as much as possible 
• Usability gets better over time 
• We should test our designs when they are low, mid and high fidelity 
• It is easier to make changes to hand drawn prototypes and low fidelity 
• Less changes should need to be made at high fidelity 
• We test to uncover and fix problems 
• Not to validate that we as designers got it right 
• If you’re learning huge amounts every time you test your designs 
• Consider if you are testing your designs in the right way 

Process: planning, conducting, reporting 

• moderated in person, remote or unmoderated
• Test plan 
  • goal of the test 
  • Number of test participants 
  • Approx session length 
  • Time plan 
  • Cost estimate 
• Recruiting participants 
  • check back to out personas 
  • Recruitment screener – could make a survey and send it out and ask simple questions about people who are available for your tests to make sure they fit your personas  
  • Book participants 
• Write a test script 
• briefing 
• Pre-session interview
• Tasks 
• Post session interview
• Pilot it – a friend or colleague goes through it to makes sure it will work or will be understood 
• What should a briefing consist of 
  • Welcome the participant 
  • Introduce yourself 
  • Explain what they are there for 
  • We are not testing you, we are testing the interface – you can’t do anything wrong 
  • Please think out loud during the sessions as it helps me to understand your thought processes 
  • I’m recording the sessions to help me analyse things afterwards – might need to sign something or send in advance 
  • If you’re feeling uncomfortable at any point we can talk a break 
  • Please consider what you see today to be confidential – might need something to be signed maybe send something before 
  • I may not be able to answer any questions you ask as I don’t want to influence what you’d do next 
    • Do you have any questions before we begin 
• Pre-session interview 
  • Questions to establish you’ve got the correct persona 
  • Easy simple questions to put the participant at ease 
  • Loosely follow pre-interview question list be curious 
  • Find out anything you want to know in preparation for the tasks 
• Test tasks 
  • Wait until this point to show the prototype 
  • Consider giving an easy task to start with 
  • Hand the participant the task in writing 
  • Ask them to give you their understanding of what they have to do before they begin 
•  Post session interview 
  • Moving to subjective feedback 
  • How did you find the experience using this prototype today
  • What 2/3 things did you like the most 
  • What 2/3 things are in most need of improvement 
• Who should be in a usability test? 
  • Test participant 
  • Moderated – person conducting the test 
  • Note taker 
  • Observers 

Results 

• Analysing results
  • positive 
  • Minor problem – something that requires a fix but has little effect on the efficiency or effectiveness 
  • Major problem
  • Critical problem – participants unable to complete a task
  • Catastrophic problem – existential threat 
• Reporting results 
  • Backlog written as jobs to be done – as a user needs to be able to see the X at the same time as the Y
  • Any that aren’t picked up should be stored somewhere 
  • Complied into a report 
    • An executive summary 
    • 5-50 findings (including the positive) 
    • The test script 
    • Screenshots or pictures that supplement the description of important usability findings 
    • May contain recommendations on how to fix things