Principles of Survey Design
Business Research Methods
RESE-1170
Tutorial 7
March 2023
• Considered biases in questionnaire
design
• Looked at good and bad survey design
• Considered how to identify variables
and scales.
Lecture learning
outcomes
Recap
Lecture recap
What is ‘quantitative research’?
What are its strengths?
What are its limitations?
This image, by Nguyen Dang Hoaning Nhu is liscenced under CC-BY
What is quantitative research?
Deals in numbers rather than
rich descriptive data.
Requires a large sample of
participants (statistical
power).
Hypotheses drafted a priori,
specific, testable.
Tests for relationships
between variables.
Allows for generalisations.
Most quantitative research in
the social sciences uses the
survey method (but there are
others).
Leading questions and biased outcomes
Reflecting on the video, why
would such a practice of
leading questions be a
problem?
Because such data produced by the survey will not accurately reflect the views
of the sample population.
Because such data can be used by those in positions of power with vested
interests to justify pre-made decisions.
Because decisions that result from this data may not produce the best
evidence-based policy and practice.
Reflections on the video
Questions should be:
1. Clear and accurate
2. Neutral, using unbiased language
3. Covers all possible responses
4. Has a clear cognate structure (comprehensible for the reader)
Questions should not be:
1. Ambiguous (when did you leave school?)
2. Vague and unprecise language (do you drink a lot?)
3. Leading (do you agree with the myth that…?)
4. Double-barrelled (do you think that students should have more classes in Business
Research Methods and Employment Law?)
5. Questions which technical jargon (Do PRI staff perform better than DFR staff within
the ICU?)
The questions we ask in surveys are important.
Are these good questions, or not?

Group activity 1
Your tutor will provide 1:1 feedback to any student who has come with a draft
of the first assessment.
While this happens, you will work in small teams to examine two surveys.
Your task is to identify the problems with the bad survey, and the key features
of the good survey.
Both of these are on Moodle.
Good and bad survey design
Reflection on the survey
This image, by Emily Morter is licensed under CC-BY
Group activity 2
Within the social sciences, many variables cannot be measured directly, but
instead require ‘scale constructs’.
Scale constructs are empirical measurements for theoretical constructs.
For example, ‘student engagement’, or ‘employee engagement’ or ‘customer
satisfaction’, or ‘employee performance’ are all conceptual terms that have no
direct measurements. Researchers theorise these concepts, and then build
survey questions to measure them.
Let’s take a look now at some examples.
Scale constructs
Take an example of ‘student engagement’
Different researchers and organisations will define “student engagement” in different ways,
because they will conceptualise it in different ways. How they are defined will shape how we
understand “engagement”, and how we ‘measure’ it.
Handelsman et al (2005) conceptualised “student engagement” as having four components:
Skills engagement, emotional engagement, participation/interaction engagement, and
performance engagement.
Let’s take an example:
Created by: Dr Scott Tindal, 2022
Handelsman, M., Briggs, W., Sullivan, N.
and Towler, A. (2005) A measure of
college student course engagement. The
Journal of Educational Research 98 (3):
184-192.
Examples of scale constructs
Created by Scott Tindal, 2022 16
Your tutor will now give you some journal papers, and from these you will:
1. Identify the underlying concepts.
2. Identify the specific survey questions that the researcher used to measure
the construct.
Task 2
This image, by Ben Allen is licensed under CC-BY
Next week we will start to do some quantitative data analysis using Microsoft
Excel.
To prepare, please make sure that you have Microsoft Excel installed on your
laptop. If you don’t already have it, feel free to use the University’s license:
https://www.gre.ac.uk/it-and-library/software/microsoft

If you get stuck, email IT Service Desk Please bring your laptop next week. We will also be able to provide a small
number of laptops during the session if you don’t have/can’t bring a laptop
with you. .
Next week