Today's lesson
Designing the survey
Until now you have read other people's data. Today the class writes one survey together, takes it out on campus, and collects data of its own.
1 The question
What we are trying to find out
Our question
What is related to York students’ mental health?
Every question we put on the survey has to earn its place against that. If a question does not help answer it, or does not feed one of our tests, it does not go on.
Notice how that question is worded. Not “what is harming York students’ mental health,” which would assume the answer before we collect anything, and would promise a cause our survey cannot show. Related is what a one-time survey can deliver — and it leaves room for things that go with better mental health, not only worse.
2 The logic
The test decides the question
This is backwards from how people usually write surveys. Do not start with what you are curious about. Start with the test you want to run, because each test needs a particular shape of question.
Independent-samples t-test
Why it exists
It compares two separate groups of people on one number.
So you need
One yes/no question, to split people into two groups. One number question, to compare them on.
For example
“Do you work while in school? Yes / No”“On a scale of 0 to 10, how stressed have you felt this week?”
Which answers
Do students who work feel more stressed than students who don’t?
Paired-samples t-test
Why it exists
It compares the same people in two situations. Everyone gives two numbers.
So you need
Two number questions that ask the same thing in two different situations.
For example
“How many hours do you usually sleep on a weeknight?”“How many hours do you usually sleep on a weekend night?”
Which answers
Do students sleep less on weeknights than on weekends?
One-sample t-test
Why it exists
It compares one group to a fixed number from outside the survey.
So you need
One number question, plus a benchmark you bring with you.
For example
“How many hours did you sleep last night?”
Which answers
Do York students sleep less than the recommended 7 hours?
Simple linear regression
Why it exists
It checks whether one number rises or falls as another rises.
So you need
Two number questions, both answered by the same person.
For example
“How many hours did you sleep last night?”“On a scale of 0 to 10, how stressed have you felt this week?”
Which answers
Do students who sleep less report more stress?
Read down the middle row of those four cards and you have the whole design rule: yes/no splits people, numbers get compared.
3 Writing them
Two sets of rules
Yes/no questions — the ones that make groups
- Exactly two answers. Yes or no. Freshman / Sophomore / Junior / Senior has four, so it cannot split people into two groups for a t-test.
- Everybody lands in one group only. That is what makes the groups independent — different people, each measured once.
- Watch for overlap. “Do you take day or night classes?” breaks, because some students take both. Fix it by asking “Do you take any night classes? Yes / No.”
Number questions — the ones that get compared
- Ask for an actual number. Hours, counts, or a 0 to 10 rating. Not bands like “5–10 hours.”
- Use 0 to 10, not strongly agree to strongly disagree. Two reasons, and you already know both:
- Agree and strongly agree are words. Somebody would have to turn them into numbers before averaging, and nobody measured whether the step from agree to strongly agree is the same size as the step from neutral to agree. That is the ordinal problem from Week 3.
- Eleven choices spread people out. Five bunch them up, and a real difference gets harder to see.
- For regression, the same person must answer both.
4 Build it
Fill the slots
We need these to run our tests. Everything else is optional.
Yes / NoSplits people into two groups. Something that divides York students in a way that might matter for mental health.
NumberThe thing we compare. Hours, a count, or 0 to 10.
NumberA second number, so we can see whether it rises or falls with the first.
SpareOne or two more, if the class wants options. Three yes/no questions means everyone can run a different t-test.
What each slot buys you
Slots 1 and 2 give you an independent-samples t-test. Slots 2 and 3 give you a regression. Two number questions asking the same thing in two situations would give you a paired t-test as well.
5 Who answered
Demographics, and why only a few
Demographics do not run any of our tests. They tell a reader who the results apply to — “40 people answered, most were 18 to 24, three quarters were York students.”
- Two or three, no more. Every extra one takes time and makes a person easier to identify. We promise anonymity, and these results go on public portfolio pages.
- “Prefer not to say” on every one.
- Put them at the end, while people are still paying attention for the questions that matter.
Age: exact number, or ranges?
Ask for the exact number. You can always group exact ages into ranges later; you can never turn ranges back into ages. And a real number can go into a regression.
The cost is privacy, so: collect the exact age, publish only ranges or an average. Never put the raw spreadsheet on a public page.
6 Out on campus
Collecting it
One form, one QR code
Everyone scans the same code onto their phone. One survey means one bigger dataset, and a bigger dataset means every test is easier to read.
Say what it is before they tap
“I’m a public health student. This is a short anonymous survey about student mental health for my class. It takes about two minutes, and it’s completely voluntary.” That sentence is not a formality — it is the standard you will be held to in this field.
Nobody’s name, ever
Responses are anonymous because your portfolio is public.
Write this down for your limitations section
Whoever is standing outside the library this afternoon is not York College. The people you reach are the people who were there, and who agreed. That is a convenience sample, and it belongs in your write-up.
7 After
What you will do with it
Once the responses are in, the data gets posted the same way the writing-hand data was, and each of you analyzes it yourself.
- One t-test — pick a yes/no question and a number question.
- One regression — the two number questions.
- Three sentences each: direction, size, belief. Then one plain sentence, with no causation.
- Your limitations, including the convenience sample and however many people actually answered.
You already have the prompts for both tests — the t-test prompt is in Week 8, and the regression write-up frame is in Week 7.
8 Leaving with this
One line
Decide the test first, and the questions write themselves. Decide the questions first, and you may end up with answers no test can use.