Observational methods

Observational methods record behavior as it occurs, rather than asking people to summarize themselves. The record can be made in the setting where the behavior usually happens or in a setting arranged so that the behavior can be seen. The scientific difference between a glance and a study is the coding scheme: a set of defined categories, applied to specified intervals or events, so that a second observer could produce the same record. Without the scheme, the “observation” is a memory of what struck the researcher, which is a source of hypotheses and not a data set.

Naturalistic observation watches behavior in its ordinary setting, with the researcher trying not to reshape it. The gain is that the behavior is the one the setting actually produces. The cost is control. The researcher cannot make the interesting event happen on schedule, and the variables that co-occur in the setting are tangled. Naturalistic observation is at its best for description and for discovering what a later experiment should manipulate. A causal claim from a purely naturalistic record has to survive the third condition from the opening chapter, and it usually cannot, because rivals were free to travel together.

Systematic observation uses the coding scheme in a disciplined way, often in a setting the researcher has partly arranged so that the behaviors of interest will occur. A playroom with a standard set of toys, a discussion with a standard topic, a driving simulator with a standard route: the setting is constrained so that codes can be compared across people. Systematic observation gives up some of the mess of ordinary life in exchange for a record that can be counted. The paper should say which trade it made.

The observer can be hidden or obvious, and can be a bystander or a participant. Those choices change both the ethics and the data. Concealed observation in a place where people expect privacy is an ethics problem from the earlier chapter, not a clever method. Observation of public behavior, where there is no reasonable expectation of privacy and the researcher does not intervene, is often treated as lower risk. The expectation is the issue. A public plaza is not a changing room.

Reactivity is the change in behavior that occurs because people know they are being observed. People become neater, kinder, quieter, or more industrious when a clipboard appears, and the change can fade as they habituate. Reactivity does not make observation useless. It means the first minutes may be about the observer. A long enough record, an unobtrusive measure, or a concealed record where ethics allow, are the usual responses. An unobtrusive measure is a trace of behavior that does not require the person to perform for the researcher: wear on floor tiles, settings of a thermostat, the contents of a recycling bin. Traces have their own ambiguities. A worn tile does not say who walked, or why.

Participant observation puts the researcher inside the setting as a member, openly or not. The gain is access to meanings that a bystander cannot see. The costs are reactivity if the role is known, deception if the role is hidden, and a loss of distance that makes coding drift toward the researcher’s loyalties. A participant observer who has come to love the group is a poor sole coder of the group’s faults. The coding scheme and a second coder are the correction, where a second coder is possible. Where it is not, the limit belongs in the conclusion, in the same size type as the insight.

Interrater reliability, from the reliability chapter, is the spine of an observational study. Two observers apply the scheme to the same intervals, and agreement is reported with a statistic that respects chance, such as Cohen’s kappa when the codes are categories. A scheme with dozens of subtle categories will often fail this test until the categories are fewer and the boundaries are written as decisions, not as vibes. “Aggressive” is not a code. “Hits, shoves, or snatches an object from another child during the interval” can be a code. The examples in the manual are part of the method.

A case study is an intensive description of one person, one group, or one event. It can generate hypotheses, falsify a universal claim (one clear counterexample ruins “always”), and preserve detail that a count throws away. It cannot estimate a frequency in a population, and it cannot separate the many things that were true of that case at once. Calling a case study an observation does not promote it. The promotion happens when a scheme is applied across cases so that a comparison exists. Psychologists sometimes use “case study” as a compliment for depth. Depth is real. Generality is a different virtue, purchased with more cases and a sampling plan.

Archival research analyzes records that already exist: official statistics, newspaper accounts, social-media posts, school files. The researcher does not create the behavior and often did not create the record. The advantage is scale and the absence of reactivity to the researcher. The disadvantage is that the record was made for someone else’s purpose. What was not written down is invisible, and what was written down may be what the institution wanted to see. An archival study of arrests is a study of arrests, not a study of offenses. The gap between those two is a construct-validity problem sitting inside a convenient data set.

Observational work is the home of the first goal of science, description, done with a discipline that a diary does not have. Frequencies, sequences, and co-occurrences are legitimate results. A sequence can suggest temporal precedence, which a one-time questionnaire cannot. It still rarely eliminates alternatives, because the observer did not assign the first event. A child who is shoved and then cries has a time order. The shove may itself have been a response to an earlier act the code missed. The discussion can say “followed.” It earns “produced” only with more design than a single observed pair.

The practical standard for this chapter is a methods paragraph a stranger could enact. Where were people observed, for how long, with what expectation of privacy, under what coding rules, with what agreement between observers, and which behaviors were defined out of the scheme? A paragraph that says “participants were observed and aggressive behavior was noted” has not reported a method. It has reported an intention. The next chapter turns to the other great nonexperimental tool, asking people about themselves, which has a different set of ways to go wrong and the same duty to say exactly what was done.