Sources of ideas, hypotheses, and the literature

A research question names a relationship, a difference, or a description that evidence could settle. “Why are people the way they are?” is a mood. “Among adults who have just been interrupted, does a written plan for the unfinished task reduce how often they return to it in the next hour, relative to an unrelated writing task?” is a research question. The second sentence names the people, the behavior, the comparison, and the time. It can come out the other way. That possibility is what makes it worth running.

A hypothesis is a tentative answer to the question, stated so that a result could support it or count against it. A prediction is the hypothesis translated into the operations of one study: which scores should be higher, in which condition, on which measure. Cozby and Bates keep those two words apart because a hypothesis can be right in spirit while a particular prediction fails for a boring reason, such as a measure that barely moves (Cozby & Bates, 2020). When the paper says “we hypothesized,” the reader should be able to point at the comparison the hypothesis required. A hypothesis that is rewritten after the results are known, and then presented as the hypothesis that guided the study, is a different object. It is a description of what was found, wearing the clothes of a forecast.

Ideas arrive from more than one place, and the place does not certify them. Common sense, a puzzling observation, a practical failure, and an existing theory are all legitimate sources. Common sense is a useful source and a poor authority, because common sense also contains contradictions: absence makes the heart grow fonder, and out of sight is out of mind. A study is what decides which saying holds, for whom, and under which conditions. The source of the idea can be a kitchen, a clinic, a novel, or a previous paper. The scientific status of the idea begins when it is stated as a question with a procedure.

A theory is an organized set of ideas about how a set of variables relate, offered so that it explains existing findings and generates new ones. A theory is larger than a single hypothesis. The hypothesis is one stake in the ground. The theory says why that stake should be there and where the next stakes should go. If the theory is only a new name for the finding, it generates nothing. A useful theory tells a researcher what to measure next, what should happen if a condition is changed, and which popular belief the theory says is wrong.

Theories in behavioral science are evaluated by the company they keep with evidence, not by how satisfying they feel. A theory that organizes many findings and survives attempts to embarrass it is earning its place. A theory that is adjusted after every contrary result, so that no result could touch it, has become unfalsifiable. That standard was the point of the previous chapter. Here the practical consequence is about reading. When a paper invokes a theory, the reader asks which claim of the theory this study actually tests, and what the theory would have required if the result had gone the other way.

Past research is the other great source, and it is not the same thing as a theory. A single surprising experiment can suggest a question. A pattern across many experiments can suggest that the question has moved. Replication is an attempt to run a study again, either as closely as possible or with a deliberate change that the theory says should not matter, or should matter in a stated way. Exact repetition checks whether the finding was a one-room accident. A changed repetition checks the reach of the finding. Both are research. Neither is a failure of imagination.

A literature search is the systematic hunt for what has already been claimed, with what methods, and with what results. The point is to stop a researcher from spending a year on a question that was answered, or from missing the measurement problem that sank the last three attempts. In psychology the main index is PsycINFO, which catalogs journal articles, books, and dissertations in the behavioral sciences. Google Scholar and a journal’s own site are useful supplements. A search box is not a literature. A literature is the set of papers the question actually depends on, found with terms that match how that field names its variables, and read past the abstract.

Search terms are part of the method because the wrong word returns the wrong century of work. A researcher who types only the everyday name of a behavior will miss the papers that use the construct’s technical name, and a researcher who types only the technical name will miss applied reports that never adopted it. Cozby and Bates recommend starting broad, then combining terms, then following the references of the papers that clearly belong. The references of a relevant paper are a map drawn by someone who already did part of the search. The papers that cite that paper, which a database can list, are the map of what happened afterward.

A primary source is the report written by the people who ran the study. A secondary source is someone else’s summary: a textbook chapter, a review, a news account, a lecture. Secondary sources are how a reader finds the primary ones and how a field teaches its outline. They are not a substitute for the primary report when the claim is going to be used. Summaries drop the procedure, round the numbers, and sometimes attach a moral the authors did not draw. If a study is important enough to cite for a finding, the finding should be checked against the paper that produced it. This course uses textbooks as the map and treats the empirical claim as living in the primary report.

Reading for a methods course is a extraction, not a highlight of the stirring sentences. For each paper that might matter, the notes should hold: the question in one sentence, the design, who was studied and how they were obtained, the operational definitions of the main variables, the comparison that carries the conclusion, the result in the units the paper used, and the limitation the authors admit or the design makes obvious. A note that says only “this study showed that rewards help” has thrown away the method. The next study cannot build on a verb.

A literature review inside a new paper is the argument that these extracted pieces add up to a reason for the new study. The review is not a tour of every paper the author enjoyed. It is a path from what is known to the specific gap this study fills: a missing comparison, a confound that keeps recurring, a population the finding has never met, a mechanism that was inferred and never measured. A gap that is only “nobody has done this exact thing with this exact group” is weak if there is no reason to expect a difference. A gap tied to a theoretical conflict or a repeated design flaw is a reason to run participants.

Review articles and meta-analyses are the efficient entrance when a literature is large. A narrative review organizes the studies by theme. A meta-analysis combines results statistically and, when it is done well, shows the variation among studies rather than pretending they were one study. A meta-analysis is still a secondary source with respect to any one experiment. It is a primary source with respect to its own coding decisions: which studies were included, how effects were converted to a common unit, and how the authors handled studies that found nothing. Those decisions are part of the result. A reader who cites “the meta-analysis” without those decisions is citing a mood about the literature.

The end of a good search is a sentence of this shape: given these findings, this unresolved comparison, and this reason to care, the next study should do this. The sentence should be specific enough that a reader could tell, a year later, whether the study did that thing. Vague programs (“more research is needed on this important topic”) are how literatures stall. The needed research is a particular contrast that the existing papers do not contain.

Two ethical notes belong here because the literature is where they first appear, even though the next chapter is the ethics chapter. First, the search should include results the researcher does not like. A file of only the confirming papers is a speech, not a review. Second, work that found no difference is part of the literature when it can be found. Journals have historically preferred differences, so the published record can look cleaner than the set of studies that were run. A methods student learns to treat a neat story with suspicion until the procedures are in view. That suspicion is not cynicism. It is the same falsifiability rule applied to other people’s papers, which is the only way it is a rule at all.

The practical product of this chapter is a habit. Before a procedure is built, the question is written in one sentence, the hypothesis is written so that it can lose, the prediction names the measure, and the closest primary papers have been read for their procedures rather than for their adjectives. The next chapter turns to the constraints on how those procedures may treat the people who provide the data.