Chapter 7

Survey Design

Slides

Handounts

Required Reading

Learning Outcomes

Understand

  • Research questions guide survey design.
  • Wording can have a large impact on answers.
  • How to apply study design lessons to surveys.

Apply

  • Take a survey and pay attention to the questions you are being asked. Are any of them leading? Do they accurately allow you to express your skills or opinions?
    • Most in-store receipts have links to take a survey
    • Phone surveys sometimes happen
    • If you call a service center you may be asked to take a survey
    • Fill out a product review, it normally includes a mini survey

Lecture Notes: Survey Scales

Survey Scales about Security and Privacy

Writing good survey questions requires careful thought and a good knowledge of the information you are trying to measure. Some concepts are also challenging to measure like “security attitude”. To solve this problem, research teams create what are known as “survey scales”. A scale is a set of questions that are well written and have been shown to reliably measure a concept.

Scales have several useful properties. First off, writing good survey questions is challenging, so a pre-written scale is just easier to use. Secondly, if multiple researchers all ask the exact same questions, it becomes possible to compare answers across research studies. Finally, researchers using a scale can assume (and cite) that it reliably measures a concept.

Below are four scales from usable privacy and security. Each measures a different set of concepts.

  • IUIPC

    • Naresh K. Malhotra, Sung S. Kim, and James Agarwal. 2004. Internet Users’ Information Privacy Concerns (IUIPC): The Construct, the Scale, and a Causal Model. Information Systems Research 15, 4 (2004), 336–355.
    • https://doi.org/10.1287/isre.1040.0032

  • SSDSES

    • Daniel Votipka, Desiree Abrokwa, and Michelle L. Mazurek. 2020. Building and Validating a Scale for Secure Software Development Self-Efficacy. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–20.
    • https://doi.org/10.1145/3313831.3376754

  • SeBIS

    • Serge Egelman and Eyal Peer. 2015. Scaling the Security Wall: Developing a Security Behavior Intentions Scale (SeBIS). In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 2873–2882.
    • https://doi.org/10.1145/2702123.2702249

  • SE-6

Other Useful Scales