HCI Exercise 1

Human-Computer Interaction SS 2023

Exercise 1: Heuristic Evaluation Plan (HE Plan)

First of all, read the sections in the lecture notes on heuristic evaluation. You will have to explain what these techniques are to your client (tutor) at the first client meeting.

This exercise only involves writing the plan, you do not do the actual evaluation yet!

1 Plan Your Heuristic Evaluation


2 Write the HE Plan

Write a short plan for the heuristic evaluation:

  1. Some title information indicating the group number, the title/topic of the evaluation, and the name of each group member.

  2. A description of the methodology behind a Heuristic Evaluation, in your own words.

  3. User profiles describing which groups of users you think are supposed to be addressed by the web site. Describe the goals and typical tasks for each user group.

  4. The names, characteristics, and environments of each evaluator.

  5. Coordinate between yourselves, so that the evaluators use a mix of devices and browsers to inspect the web site:

    • Two of the evaluators should evaluate on a mobile device (= smartphone or tablet, running Android or iOS), in portrait orientation, each using a different approved mobile web browser.

      On mobile, the approved browsers are: Chrome, Firefox, Safari, and Samsung Internet Browser.

    • The other evaluator(s) should evaluate on a PC (= desktop or laptop, running Windows, MacOS, or Unix), in landscape orientation, each using a different approved PC web browser.

      On PC, the approved browsers are: Chrome, Firefox, and Safari.

    It is OK for one evaluator to use (say) Chrome on PC, and another evaluator to use Chrome on mobile, since they are different devices.

    You must use real devices and not an emulator or simulator.

    If you are using a mobile browser and your web site suggests you should go to the mobile version of the site, then do so. However, do not install a native app, even if your web site suggests you should do so.

  6. In an attempt to reflect real usage patterns, one of the evaluators on mobile and one of the evaluators on PC should plan to use an ad blocker.

    If you find a problem which may be due to blocked content, unblock ads temporarily to see whether the problem persists. If the problem then goes away, document it as a problem apparently caused by ad blocking.

  7. Before you start evaluating, plan to reset the browser to a fresh state:

    • Make sure all add-ons and extensions are disabled (except for an ad blocker, if you are using one via an extension)).

    • Delete all cookies.

    • Delete all temporary files (clear the cache).

  8. Plan to try out your assigned website first with all cookies enabled and then with most cookies disabled (only necessary cookies enabled) to see if there is any difference.

    If you find a problem which appears to be due to disabling cookies, document it as such.

  9. Explain how you will make make screen video recordings on the various platforms you have chosen to evaluate. See my Guide To Session Capture on various platforms. You should plan to record your voice (with a live commentary while you work) as well as what happens on screen.

    On touch devices, see if your recording software can turn on display of touch events.

    The screen recording should be at most at FullHD resolution (1920×1080 pixels in landscape orientation or 1080×1920 pixels in portrait orientation). If your device has higher resolution than FullHD, please plan to adjust the settings or make the browser window smaller. If your device has lower resolution than FullHD, record at its maximum resolution. The standard browser GUI should be included in the recording.

    If the native resolution of your device is too high in one or both directions, and you cannot adjust the settings or make the browser window smaller, you will have to record at higher pixel resolution and later transcode down uniformly to at most FullHD. Do not distort the video and do not create black or empty strips to either side (or top and bottom). See my Guide to Video Transcoding. Unfortunately, transcoding always results in a loss of quality.

    Use a good microphone, and make a test recording to make sure everything is working and the audio can be heard.

    Our preferred video format is MP4 with H.264 video and AAC audio. If your device/software can record in that format natively, that will be perfect. Otherwise, you will have to look at converting/transcoding your video clips after extraction.

    Plan to use the following recommended video settings:

    Output Video Resolution:1920×1080 (FullHD) [or 1080×1920] or lower
    Frame Rate:20
    Rate Control:VBR (Variable Bit Rate)
    Bit Rate:5000 Kbps (= 5 Mbps)

    Plan to use the following recommended audio settings:

    Codec:AAC (= mp4a)
    Sample Rate:44100 Hz
    Bit Rate:160 Kbps
    Bits per Sample:32
  10. Explain how you will extract video clips from the screen capture videos. All findings must be illustrated with a video clip.

    In Ex 2a, each evaluator will only be handing in their recreated/extracted video clips, not their entire screen capture videos.

  11. Adapt and include the log files you will be using for the evaluation:

    • Assign each evaluator a two-letter shorthand code based on their initials. For example, Keith Andrews would be “ka” in lower case and “KA” in upper case.

      If two evaluators in your team both have the same initials, then assign one of them a variant. For example, Ken Anderson might be assigned "kn" in lower case and “KN” in upper case.

    • Copy the skeleton log file log-ee.txt from the materials to create a plain text log file for each evaluator:


      where EE is replaced by the initials of the evaluator in lower case. Do not use any special characters, umlauts, or spaces in the file names.

      For example, if your name is Keith Andrews, the log files should be named log-ka.txt.

    • Each evaluator should fill in the metadata at the start of their log file to match their (planned) evaluation environments. Enter the name of the corresponding evaluator. Take care to preserve the character encoding of the log files as UTF-8.

    • Each negative finding (problem) will be assigned an ID of the form EE-Neg01, EE-Neg02, etc. Each positive finding will be assigned an ID of the form EE-Pos01, EE-Pos02, etc. In each log file, replace the initials “EE” in the example problem and positive finding IDs with the initials of the corresponding evaluator in upper case.

  12. Adapt the example spreadsheet helist.xlsx for your evaluation:

    • In the sheet (tab) “Problems”, replace the generic codes E1 through E4 with the upper case initials of your evaluators (yourselves) in the columns “Found By” and “Severity”.

    • In the sheet (tab) “Positives”, replace the generic codes E1 through E4 with the upper case initials of your evaluators (yourselves) in the columns “Found By” and “Positivity”.

For the HE Plan:

3 Prepare the Presentation

4 Prepare the HE Plan Directory

5 Upload Your HE Plan to Sapphire