Assignment 1

Deadline: July 9th

In this assignment you will be conducting a cognitive walkthrough similar to the one done in the Why Johnny Can’t Encrypt research paper we read.

The goal is to perform an expert cognitive walkthrough evaluation of a current encryption tool.

Groups

You are allowed to do this assignment on your own or with one other person. If you are in a group of 2, only one final report needs to be written up.

Steps

Step 1: Select a tool to evaluate

You can select any encryption tool with the exception of WhatsApp and Signal as the correct usage of these have been partially covered in the ECE458 course. If you are unsure, I recommend Wikipedia’s list of encryption tools. In particular, consider tools listed at the bottom of that page under the “disk encryption”, “email clients,” and “OTR” messaging categories. Download and install (or, if applicable, simply enable) the tool you chose.

Step 2: Pick a task and its correct usage

Think about what the main purpose of the tool is and then determine the correct usage of the tool for that task. Some tools will have only one main task, but others may have several. You only need to do the activity for one main task, but you do need to cover all the aspects of that task. For example, if key exchange is necessary before encrypting, then you need to consider key exchange and encrypting.

You may need to consult documentation or guides to identify the correct way to use the tool securely. Even if you use this tool yourself already, make sure that you are correctly handling issues like: keys, backup, and verifying identities of communication partners (if there are any).

Step 3: Conduct a cognitive walkthrough

There are formal and informal cognitive walkthroughs. A formal cognitive walkthrough is a precisely planned study conducted by several human-computer interaction experts. The experts use a persona, make formal action sequences, and fill out structured reports as part of the process.

For this assignment you will be doing a ligher version closer to an informal cognitive walkthrough. The goal of such a walkthrough is to find the big problems but follow a less documentation-heavy process. This approach is more commonly used by professionals like HCI designers and software engineers to quickly evaluate a user interface while still following a structured approach. Like most HCI methodologies, one of the key goals of a cognitive walkthrough is to get you to slow down and seriously consider the interface in front of you from the perspective of a real user.

In this study you should assess the usability of the design for a University student who has never taken any security courses. That means they are smart, experienced with using comptuers, experienced with the Internet, but may have no idea of how encryption works or what keys even are.

Download the spreadsheet and fill in the task you are assessing at the top. Also fill in the correct actions necessary to complete that task (step 2). Copy and paste the existing action blocks as needed to ensure that all correct actions are listed.

Now go through your selected software and fill in the answers to the 4 cognitive walkthrough questions for each action. If you are in a group of 2, then both members need to fill in the spreadsheet seperately.

Step 4: Identify big issues

Look at the spreadsheet(s) you created and use them to identify what issues you consider to be most problematic. When dealing with a client or writing a research paper you often have to synthesize the low level findings into higher-level issues.

Final Write-up

You will be submitted a report and a spreadsheet. If there are two members then both members should fill out their own spreadsheet, so there will be two spreadsheets submitted.

Report

The report should contain:

  1. Names of the group member(s)
  2. Name of the tool you chose
  3. Paragraph 1: In your opinion, what were the largest usability issues with the tool you evaluated?
  4. Paragraph 2: What usability flaws identified in the Johnny paper still persist in this tool? Describe them.
  5. Paragraph 3: What usability flaws identified in the Johnny paper have been addressed to your satisfaction by this tool? How were they addressed?

If you believe any of those paragraphs is not applicable (e.g., the tool has no usability flaws not described in the Johnny paper), instead briefly explain why you believe it is not applicable.