HCI Exercise 1

(INB.06021UF Human-Computer Interaction 3VU SS 2020)

Exercise 1: Heuristic Evaluation Plan (HE Plan)

First of all, read the sections in the lecture notes on heuristic evaluation and usability testing.

This exercise is just the plan, you do not do the actual evaluation yet.

1 Plan Your Heuristic Evaluation


2 Write the HE Plan

Write a short plan for the heuristic evaluation:

  1. Some title information indicating the group number, the title/topic of the evaluation, and the name of each group member.

  2. A description of the methodology behind a Heuristic Evaluation, in your own words.

  3. User profiles describing which groups of users you think are supposed to be addressed by the web site. Describe the goals and typical tasks for each user group.

  4. The names, characteristics, and environments of each evaluator.

  5. Each evaluator will evaluate the web site twice: once with a web browser on a PC (= desktop or laptop, running Windows, MacOS, or Unix), and once with a browser on a mobile device (= smartphone or tablet, running Android or iOS). You must use real devices and not an emulator or simulator.

    If you are using a mobile browser and your web site suggests you should go to the mobile version of the site, then do so. However, do not download and install any native apps.

    The mobile device should be operated in portrait mode. The browser on the PC should be operated in landscape mode.

  6. Coordinate between yourselves, so that each evaluator uses a different web browser to the other evaluators on each device:

    • On a PC, the priority order of browsers is Chrome, Firefox, Safari, Edge, and Brave.

      This means that if there are four evaluators in your group, one will use Chrome, one will use Firefox, one will use Safari, and one will use Edge. If, say, everyone has a windows PC, so no-one can run Safari on a Mac, then Brave is there as a backup.

    • On mobile, the priority order of browsers is Chrome, Firefox, Safari, Android Browser, and Brave.

      This means that if there are four evaluators in your group, one will use Chrome, one will use Firefox, one will use Safari, and one will use Android Browser. If, say, everyone has an iOS mobile device, so no-one can run Android Browser, then Brave is there as a backup.

  7. In an attempt to reflect real usage patterns, each evaluator should plan to use an ad blocker on one of their devices (and no ad blocker on the other device).

    If you find a problem which may be due to blocked content, turn the ad blocker off temporarily to see whether the problem persists. If the problem then goes away, document it as a problem apparently caused by the ad blocker.

  8. Explain how you will make make screen video recordings on the various platforms you have chosen to evaluate. See my guide to session capture on various platforms. You should plan to record your voice (with a live commentary while you work) as well as what happens on screen.

    On touch devices, see if your recording software can turn on display of touch events.

    The screen recording should be at most at FullHD resolution (1920×1080 or 1080×1920 pixels). If your device has higher resolution than FullHD, please plan to adjust the settings or make the browser window smaller. If your device has lower resolution than FullHD, record at its maximum resolution. Use a good microphone where possible, and make a test recording to make sure everything is working and the audio can be heard.

    Our preferred video format is MP4 (H.264 video and AAC audio). If your device/software can record in that format natively, that will be perfect. Otherwise, you will have to look at converting/transcoding your video clips after extraction.

    Plan to use the following recommended video settings:

    Output Video Resolution:1920×1080 (FullHD) [or 1080×1920] or lower
    Frame Rate:20
    Rate Control:VBR (Variable Bit Rate)
    Bit Rate:5000 Kbps (= 5 Mbps)

    Plan to use the following recommended audio settings:

    Codec:AAC (= mp4a)
    Sample Rate:44100 Hz
    Bit Rate:160 Kbps
    Bits per Sample:32
  9. Explain how you will extract video clips from the screen capture videos. All findings must be illustrated with a video clip.

    In Ex 2, each evaluator will only be handing in their extracted video clips, not their entire screen capture videos.

  10. Adapt and include the materials you will be using for the evaluation:

    • Copy the skeleton log file log-ee.txt from the materials to create two plain text log files for each evaluator:


      where EE is replaced by the initials of the evaluator in lower case. Take care to preserve the character encoding of the log files as UTF-8.

      If your name is Keith Andrews, then your initials in lower case are "ka". If two evaluators in your team both have the same initials, then assign one of them a variant, for example Ken Anderson might be assigned "kn". Please do not use any special characters, umlauts, or spaces in the file names.

      Each evaluator should fill in the metadata at the start of their two log files to match their (planned) evaluation environments.

    • Adapt the example spreadsheet helist.xlsx for your evaluation:

      • Enter the initials of your evaluators (yourselves) in the appropriate columns of the two tabs "Problems" and "Positives".

For the HE Plan:

3 Prepare Your HE Plan Directory

4 Upload Your HE Plan to Sapphire