Populations, samples, and sampling

A population is the set of people, animals, or other units that a conclusion is meant to cover. A sample is the subset actually studied. The two words are relative to a sentence. The population of “adults in this city” is not the population of “patients at this clinic,” and a sample of students in one course is neither, unless the sentence is carefully reduced to the course. Cozby and Bates treat sampling as the link between the people in the room and the people in the claim (Cozby & Bates, 2020). A beautiful design on the wrong set answers a different question than the one in the introduction.

A sampling frame is the list, or the procedure, from which the sample is actually drawn: a registrar’s roster, a clinic’s appointments this month, the people who saw a posted notice. The frame is not the population. People missing from the frame cannot be selected, and people who appear twice can be overrepresented. A phone survey whose frame is landlines has already excluded a large part of a modern city. The bias is in the frame before anyone refuses.

Sampling error is the ordinary difference between a sample statistic and the population value, the jostle that comes from drawing a finite number of people. It shrinks, in a probability sample, as the sample gets larger. Sampling bias is a systematic miss: some kinds of people are more likely to be in the sample than others, and a larger sample of the same biased procedure reproduces the miss more precisely. Size is not a cure for bias. A huge convenience sample is still a convenience sample.

A probability sample is one in which each unit in the frame has a known, nonzero chance of being included. Simple random sampling gives every member, and every combination of a given size, an equal chance. It is the clean reference case. It requires a frame and a way of drawing that is not the researcher’s hand. Stratified random sampling divides the population into strata that matter for the question, such as age bands or regions, and draws a random sample inside each. The point is to guarantee representation of strata that a single simple draw might, by bad luck, underfill. The strata have to be known before the draw.

Cluster sampling draws groups rather than individuals: schools, then students inside the chosen schools; city blocks, then households. It exists because a list of every person is often unavailable and because travel to a scattered simple random sample can be impossible. The statistical price is that people inside a cluster resemble each other, so the effective amount of information is less than the head count suggests. A methods reader who sees “we sampled 40 classrooms” should not treat that as 40 times the number of children in independent draws. The chapter on inference will care about this. The design fact belongs here.

Probability sampling is what licenses the ordinary language of margins of error in polls. The margin is a statement about sampling error under a stated plan, not a statement that the questions were good or that everyone answered honestly. A probability sample with a bad questionnaire is a precise estimate of the answers to a bad questionnaire. Measurement and sampling are different failures, and a paper can have either, or both.

A convenience sample is whoever can be reached: the class, the clinic waiting room, the people who clicked. Most laboratory psychology is built on convenience samples, especially students. That fact does not make the studies useless. It changes the sentence. Random assignment can still support a causal contrast inside the people who showed up. It cannot turn those people into a miniature of a nation. Quota sampling fills preset numbers of kinds of people and is still nonprobability if the people inside the quota were grabbed rather than drawn. The quota can look like stratification and lack the random draw that makes stratification a probability plan.

Purposive sampling selects cases because they carry the phenomenon: expert practitioners, people who lived through a particular event, clinics that run a particular program. The selection is deliberate and the inference is about those kinds of cases, not about a rate in a population. Snowball sampling asks participants to recruit further participants. It reaches networks that have no public list. It also inherits the network’s limits. A snowball started among friends of the researcher will not wander into a disconnected community just because the sample got large.

Henrich, Heine, and Norenzayan argued that a great share of behavioral science has been conducted with participants who are Western, educated, industrialized, rich, and democratic, and that those participants are unusual on many psychological measures rather than a default human (Henrich, Heine, & Norenzayan, 2010). The paper is a warning about external validity, which this course treats fully in the generalization chapter. The sampling consequence is immediate. A convenience sample of university students is a sample of a particular slice, and the slice should be named in the same sentence as the result. Hiding it in a methods footnote and then speaking about “people” is how a local finding becomes a counterfeit universal.

Even a perfect frame loses people. Nonresponse is the failure to obtain data from units that were selected. If the people who refuse are different on the variable of interest, the achieved sample is biased even though the draw was random. A poll about debt that the most indebted people hang up on will look too calm. Reporting the response rate is part of describing the sample. A low rate is not an automatic ruin, and a high rate is not an automatic blessing, but silence about the rate is a refusal to let the reader judge.

The distribution of a statistic across many samples from the same population, the sampling distribution, is the mathematical object that lets a later course say how far a sample result might sit from the population value. That machinery is Statistics for Psychology, the next course on this map. This chapter’s job was the design of the draw and the honesty of the sentence that follows from it. A reader who can say what the population was, what the frame was, whether chances were known, and who refused, can already tell whether the paper’s “we found that people…” is a report or an exaggeration.