Sociological research methods

Evidence that other people can check. Ask ten friends whether working during college hurts grades, and you will hear ten confident answers, each built on a few people they happen to know. Sociologists start with the same kinds of questions, but they answer them with research methods: planned, systematic ways of collecting and analyzing information so that someone else could repeat the work and see whether they get the same result (Conerly et al., 2021).

Lesson 1 gave you the sociological imagination: the habit of looking for patterns behind personal stories. This lesson gives you the tools for testing whether a pattern is real. One example question runs through the whole lesson: do students who work long hours earn lower grades?

There is a common sequence, usually called the scientific method. It traces back to Auguste Comte's hope, from Lesson 1, that society could be studied with the same care as nature. Figure 2.1 shows six steps (Conerly et al., 2021):

• Ask a question narrow enough to answer.
• Review the research. A literature review finds what other researchers have already learned, so you do not repeat their work or their mistakes.
• State a hypothesis, a testable prediction about how two or more things are related.
• Design and run a study that collects the data.
• Draw conclusions. Do the data support the hypothesis, or not?
• Report the results so that others can check them and try to repeat them.

Not every sociologist starts with a hypothesis. Some use an interpretive framework: they enter a group's world first, watch and listen, and let questions and explanations grow out of what they see. This approach fits Max Weber's idea of Verstehen from Lesson 1, understanding what actions mean to the people doing them (Conerly et al., 2021). Sociologists also distinguish two kinds of data. Quantitative data are numbers, such as hours worked or grade point average. Qualitative data are words, images, and observations, such as an interview about why a student took a second job.

You name your variables and decide exactly how to count them. A variable is any trait that can change from person to person or over time. In a hypothesis, the independent variable is the suspected cause, and the dependent variable is the outcome that depends on it. In the example, hours worked per week is the independent variable, and grades are the dependent variable.

Next comes an operational definition: a precise statement of how a variable will be measured. "Working long hours" is vague. "Working 20 or more hours a week at a paid job, as the student reports it on a survey" is operational. "Grades" becomes "the cumulative grade point average on the student's official record." Two researchers using the same operational definitions can compare their results; two researchers using their own private meanings cannot.

Not necessarily. When two variables change together, they have a correlation. A positive correlation means they move in the same direction; a negative correlation means one goes up as the other goes down. The hypothesis "more hours at work, lower GPA" predicts a negative correlation.

A correlation alone does not prove causation, that a change in one variable produces a change in the other. The classic example: ice cream sales and drownings rise and fall together. Ice cream does not cause drowning. Hot weather drives both, because more people buy ice cream and more people swim in summer. A hidden factor like hot weather is called a third variable, and a correlation it creates is called a spurious correlation (Conerly et al., 2021). Figure 2.2 draws the trap.

The same trap waits in the example question. Suppose students who work 20 or more hours a week do have lower grades. Family income could be a third variable: students from lower-income families may need to work more hours and may also have had fewer school resources before college. To claim that the work itself lowers grades, a researcher has to rule out explanations like that. Researchers do it by controlling for the third variable, comparing students who work different hours but come from families with similar incomes.

If the 1,000 are chosen the right way. The whole group a researcher wants to understand is the population, for example all undergraduates in the United States. Studying everyone is almost never possible, so researchers study a sample, a smaller group drawn from the population.

The key is a random sample, in which every member of the population has the same chance of being chosen. Chance spreads the sample across the whole population, so it tends to look like the population in income, age, background, and everything else, including traits the researcher never thought to measure. A convenience sample, made of whoever is easiest to reach, can be badly lopsided (Conerly et al., 2021). Figure 2.3 shows the difference.

You check it for two things. Reliability means consistency: the measure gives the same result when used again under the same conditions. Validity means accuracy: the measure captures what it claims to measure (Conerly et al., 2021).

A bathroom scale that always reads five pounds heavy is reliable but not valid: it is consistent, and consistently wrong. In the example study, asking students to recall their GPA from memory might be less valid than using official records, because people tend to remember their grades as a little better than they were. Figure 2.4 shows the idea as shots at a target.

It depends on the question. Sociologists rely on four main methods (Conerly et al., 2021):

• Surveys ask people questions, through questionnaires or interviews. They are the most common method in sociology.
• Field research gathers data in a natural setting by watching, listening, and often taking part. It includes participant observation, joining a group's daily life in order to study it, and ethnography, a detailed description of a whole community or way of life.
• Experiments change one variable on purpose, under controlled conditions, and measure what happens to another.
• Secondary data analysis uses data that someone else already collected, such as government statistics or earlier surveys. Content analysis applies the same idea to documents and media, such as news stories or television shows.

Figure 2.5 lets you see each method at work in a real study. Choose a method.

Watching people can change what they do. In the 1920s and early 1930s, researchers at Western Electric's Hawthorne Works near Chicago studied how lighting and work breaks affected factory output. The story that grew out of those studies is that workers produced more simply because they knew they were being observed, an effect now called the Hawthorne effect. Later economists reanalyzed the original lighting data and found the effect much weaker than the legend (Levitt & List, 2011). The caution still holds: any researcher who is visible may be changing the very behavior under study.

Yes. Employers asked on a survey whether they treat applicants fairly will almost all say they do. Devah Pager tested what employers actually did. In 2001, pairs of trained young men, matched on age, appearance, and speaking style, applied for about 350 entry-level jobs in Milwaukee, Wisconsin, with résumés of equal strength. Within each pair, the men took turns presenting a criminal record. The only planned differences were the record and, across pairs, race. The outcome Pager measured was a callback, an invitation to interview or a job offer (Pager, 2003).

Figure 2.6 shows the results. A criminal record cut callbacks in half for white applicants. Black applicants with no record received fewer callbacks than white applicants with one.

This design is called an audit study, and it shows the advantage of an experiment: because the researcher controlled everything except the variables of interest, the differences can be read as effects, not just correlations. Its limit is just as clear. The study covered one city and one kind of job in one year, so later researchers repeated audits in other cities to see whether the pattern held.

Honesty, safety, and respect. The rules exist because researchers have broken them. From 1932 to 1972, the U.S. Public Health Service ran the Tuskegee study of untreated syphilis in about 600 Black men in Macon County, Alabama: 399 with the disease and 201 without. Researchers never obtained the men's informed consent, and many of the men believed they were getting free government health care. Treatment was withheld even after penicillin became widely available as the standard cure in the late 1940s (Centers for Disease Control and Prevention [CDC], 2024). Public outrage over Tuskegee led to the National Research Act of 1974 and then the Belmont Report (1979), which set three principles for research with human beings: respect for persons, beneficence (do good and avoid harm), and justice (National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, 1979).

Sociology has its own cautionary case. For Tearoom Trade (1970), Laud Humphreys secretly observed men's anonymous encounters in public restrooms, wrote down their license plate numbers, found their home addresses, and later interviewed some of them in disguise as part of an unrelated health survey. The men never agreed to be studied, and exposure could have ruined their lives. The study is still taught as an example of what not to do.

Today, universities require an institutional review board (IRB) to approve research with people before it begins. The American Sociological Association's Code of Ethics (2018) asks sociologists to obtain informed consent, meaning participants know what the study involves and agree freely; to protect confidentiality; and to avoid harm. Max Weber added a further standard: value neutrality, the effort to keep personal values from distorting how data are collected and reported (Conerly et al., 2021). Sociologists debate how fully anyone can meet that standard, but nearly all agree on the goal of reporting what the data show, including results they did not hope for.

Ask five questions before you believe or share it:

• Who was studied? A random sample of the population, or whoever was handy?
• How was it measured? Look for the operational definitions.
• Correlation or causation? If it was not an experiment, what third variable could explain the link?
• What method? Each method sees some things clearly and misses others.
• Has anyone repeated it? One study is a start. Repeated findings are knowledge.