Lesson 01 argued that psychological science is a way of thinking — empirical, falsifiable, self-correcting. This lesson asks the obvious follow-up: compared to what? When you want to know whether venting helps anger, whether a supplement works, or whether a study skill pays off, you have four possible sources: your own experience, your own reasoning, an authority figure, and systematic research. The first three feel trustworthy precisely because they're yours (or your mentor's). The case of this lesson is that each fails for specific, well-documented reasons — and that knowing how they fail is what makes you able to read research intelligently.
The through-lineWhy your own experience is the worst evidence
Personal experience feels like the most trustworthy thing you own. It's actually the least reliable — for structural reasons, not because you're careless. The problems are built into how experience works, and no amount of intelligence exempts you.
Problem one: experience has no comparison group. The classic historical case is Benjamin Rush, the most famous physician in revolutionary America. During Philadelphia's 1793 yellow-fever epidemic, Rush bled his patients aggressively — sometimes removing most of the blood a body can spare — and he "saw" it work, because some bled patients recovered. What Rush never collected was the number he actually needed: how many patients recovered without bloodletting. That number wasn't gathered until the 1830s, when the French physician Pierre Louis pioneered the "numerical method" — counting outcomes in bled versus less-bled patients — and found bloodletting gave no advantage. Two thousand years of confident medical experience dissolved on contact with one comparison group.
The lesson generalizes. Every "it worked for me" claim is secretly a claim about four numbers: times you did X and got better, did X and didn't, skipped X and got better anyway, skipped X and didn't. Experience hands you exactly one of those cells — vividly — and whispers that it's enough. It never is.
Problem two: everyday life is confounded. Suppose you start a self-compassion journaling practice and, two weeks later, feel noticeably less stressed. Was it the journaling? Maybe. But midterms also ended, you started sleeping more, the weather improved, and you expected to feel better, which itself moves the needle. In research language, a confound is an alternative explanation that changed along with the thing you're crediting. Real life runs all its variables at once; nothing in your daily experience isolates one cause. Research designs exist precisely to hold the world still while one thing moves. Your Tuesday cannot do that.
Problem three: research is probabilistic, and you are a sample of one. Findings in psychology predict proportions and tendencies, not every individual case. "Smoking causes lung cancer" is true even though someone's grandmother smoked until 95; her case doesn't refute the claim any more than one lottery winner refutes the odds. When you test a claim against your own single, vividly remembered, emotionally invested case, you're using the weakest possible sample evaluated by the least neutral possible observer.
The catharsis idea — punch a pillow, scream into the void, let it out — feels obviously true, and personal experience seems to confirm it: you vent, and eventually you feel calmer. Bushman (2002) put it to a controlled test with roughly 600 angered participants who either sat quietly or hit a punching bag (some while picturing the person who had insulted them). Then everyone got the chance to blast their provocateur with loud noise. Catharsis predicts the venters should be calmest. Instead, the group that vented most aggressively was the most aggressive afterward, and sitting quietly beat venting. Venting doesn't drain anger — it rehearses it. Your experience said otherwise because time passing and distraction were confounded with the venting. This is the whole lesson in one study: what "makes sense," what you've personally felt, and what's true are three different things, and only a comparison group can sort them out.
The bias catalogueThe machinery that fools everyone
Why is experience so persuasive if it's so unreliable? Because the mind runs on shortcuts. In a landmark Science paper, Tversky and Kahneman (1974) showed that people judge probability and frequency using heuristics — fast mental rules of thumb that are usually good enough but produce systematic, predictable errors. That word "systematic" is the point: these aren't random slips that wash out; they push everyone's judgment in the same wrong direction, which is why intuition can be confidently, collectively mistaken. Kahneman (2011) later framed this as fast, automatic "System 1" thinking doing the driving while slow, effortful "System 2" mostly approves whatever System 1 hands it. Four biases from this literature are worth knowing cold.
The availability heuristic. You estimate how common something is by how easily examples come to mind (Tversky & Kahneman, 1974). Their original demonstration is elegant: asked whether English has more words that start with R or have R as the third letter, most people say "starts with" — because words starting with R are easy to retrieve — when third-letter-R words are actually more numerous. The same machinery makes rare, vivid dangers (shark attacks, plane crashes) feel more common than mundane deadly ones (flu, the drive to the airport), because news coverage feeds retrievability, not frequency. Ease of recall gets silently mistaken for truth.
Illusory correlation — the missing-cell problem. People "see" relationships that aren't there because co-occurrences are memorable and non-occurrences are invisible. Chapman and Chapman (1967) showed clinicians (and then naive undergraduates) drawings from a projective test randomly paired with symptom descriptions. Observers confidently "discovered" the same diagnostic signs practicing clinicians swore by — big eyes drawn by paranoid patients, and so on — even though the pairings were random and the relationships literally did not exist in the materials. The confirming pairs stuck in memory; the disconfirming ones never registered. Your friend texts right after you thought of them, the ER "always" fills up on a full moon — same mechanism, one filled cell of a 2×2 table masquerading as a pattern.
Confirmation bias. Nickerson (1998), in the definitive review, defined it as the seeking and weighting of evidence in ways that favor what you already believe — usually without any intent to deceive yourself. It shows up in what you search for, which sources you click, how hard you scrutinize agreeable versus disagreeable findings, and what you remember later. Among the classic results Nickerson reviews: give people with opposing views on capital punishment the same mixed body of evidence, and both sides walk away more convinced they were right, because each side credits the supportive studies and dismantles the rest. You can watch this in any diet debate — an omnivore and a vegan can read the identical study on meat and health and each find it confirms their menu. The bias isn't stupidity; it's asymmetric effort.
Overconfidence. People's confidence systematically outruns their accuracy. Moore and Healy (2008) showed "overconfidence" is really three distinct errors: overestimation (thinking you performed better than you did), overplacement (thinking you're better than others — the "everyone's an above-average driver" effect), and overprecision (being too sure your beliefs are exactly right). Overprecision is the stubbornest of the three: ask people for ranges they're 90% sure contain the true answer, and the truth lands inside those ranges far less than 90% of the time. For a scientist-in-training, the takeaway is blunt — your felt certainty is not a data point.
And one meta-bias supervises the rest. Pronin, Lin, and Ross (2002) found that people readily agree these biases are real and widespread — in other people. Their participants rated themselves as less susceptible than the average American to bias after bias, a pattern the authors named the bias blind spot. Reading this section, you probably nodded along while quietly exempting yourself. That's the blind spot working in real time. Knowing the catalogue doesn't immunize you; building comparison groups into your reasoning does.
Which bias is this? Tap a scenario to reveal the answer.
Illusory correlation is, at bottom, a failure to fill in a 2×2 table. Any claim of the form "X goes with Y" needs four numbers, not one: times X and Y happened, X without Y, Y without X, and neither. Bloodletting, psychic texts, lucky socks, full-moon ERs — all collapse the moment you demand the absent cells. When someone offers you a striking anecdote, your first question is never "wow, really?" It's "and how often did it not happen?" Make that reflex automatic and you've inoculated yourself against half the bad claims you'll ever meet.
Looking inward fails tooThe introspection illusion
Here's the twist that makes this more than a list of quirks: you can't fix any of it by looking inward, because introspection itself is unreliable. In one of the most cited papers in psychology, Nisbett and Wilson (1977) argued that people often have no direct access to their own mental processes. In their best-known demonstration, shoppers evaluated four identical pairs of stockings laid out in a row and showed a strong position effect — the rightmost pair was preferred about four to one. Asked why they chose it, people cited fabric, sheerness, quality. Asked directly whether position influenced them, they denied it, some looking at the researcher like he'd lost his mind. The choices were driven by something participants could not see and confidently explained with reasons that weren't the cause.
Nisbett and Wilson's conclusion — we "tell more than we can know" — is that when you explain your own behavior, you're not reading out an internal process; you're doing what an outside observer would do: generating a plausible story from your culture's stock of theories. The story arrives instantly and feels like memory, which is exactly what makes it treacherous. This finding echoes through the rest of the course. It's why "just ask people why they did it" is not a research method, why self-report questionnaires are designed with such paranoia (Lesson 07), and why experiments manipulate causes rather than asking participants to introspect them (Lesson 11). It's also the humbling capstone of the bias catalogue: the one observer you can consult anytime, for free, about your own mind — is an unreliable narrator.
AuthorityCan't I just trust the experts?
If experience and introspection are out, the tempting fallback is authority: find a credentialed expert and believe them. Lesson 01 met this move as Peirce's "method of authority," and it's not crazy — expertise is real, and deferring to your dentist beats freelancing. But authority is only as good as the evidence underneath it, and a credential tells you someone has spent years in a field, not that this particular claim is based on systematic data rather than the expert's own (biased, unconfounded-by-nothing) personal experience. Experts accumulate vivid cases, remember their hits, and get little clean feedback on their misses — which is to say, expertise runs on the same machinery the bias catalogue just dismantled, with higher confidence and better vocabulary.
Psychology tested this directly, and the result is one of the field's most uncomfortable findings. Meehl (1954) compared clinical prediction — an expert integrating information by judgment — against statistical (actuarial) prediction — a simple formula combining a few measured variables — across every study he could find, on questions like who will succeed in training or relapse after treatment. The formula tied or beat the expert in essentially every comparison. Thirty-five years later, Dawes, Faust, and Meehl (1989) reviewed roughly a hundred studies in Science and found the conclusion unchanged. Grove, Zald, Lebow, Snitz, and Nelson (2000) then meta-analyzed 136 studies: mechanical prediction was about 10% more accurate on average, substantially outperformed clinicians in a third to half of studies, and was substantially outperformed in only a handful. A regression equation with three predictors, applied consistently, routinely embarrasses twenty years of seasoned intuition.
Why? Because formulas have no availability heuristic. They don't overweight the vivid recent case, don't see illusory patterns, and never have an off day. Kahneman (2011) offers the fair boundary condition: intuitive expertise is real where the environment is regular and feedback is fast and unambiguous — chess players and firefighters genuinely develop trustworthy pattern recognition. But in noisy environments with slow, murky feedback — predicting therapy outcomes, hiring, admissions, stock picking — confident intuition is mostly confidence. So the rule for weighing authority: trust experts most when they can point to the systematic evidence behind the claim, and least when the warrant is "in my experience." The best experts volunteer the evidence unprompted. The ones who bristle at the question are telling you something too.
Finding the real thingHow to find and weigh actual research
So the trustworthy source is systematic research. Where does it live, and how do you tell the genuine article from things dressed like it?
Research lives in peer-reviewed journals. Before publication, a manuscript goes to an editor and two or more independent experts who critique the design, analysis, and conclusions — usually demanding revisions, often recommending rejection; top psychology journals reject most of what they receive. Peer review is the field's quality filter, and it matters. But treat it as necessary, not sufficient. Reviewers don't re-run studies or audit raw data, flawed papers get through, and — the modern complication — predatory journals now exist that will publish nearly anything for a fee while performing the theater of review. They have official-sounding names, websites, even fake editorial boards: pseudoscience in a lab coat. Checks that take two minutes: is the journal indexed in a real database (PsycINFO, PubMed)? Does its editorial board contain identifiable scholars? Do other scientists cite it?
Inside legitimate journals, learn to tell the two basic article types apart, because they answer different questions. A primary empirical article reports new data the authors collected themselves — you can recognize it by its Method and Results sections, and it's the original source everything else paraphrases. A review article reports no new data; it synthesizes the existing studies on a question. The most powerful review is a meta-analysis, which statistically combines the effect sizes from many studies into one overall estimate — the closest thing science has to a final word, though even that word gets revised. When you write papers for this course, your citations trace back to primary empirical sources; reviews are how you find them and frame them.
| Source | What it is | Best for | Watch out for |
|---|---|---|---|
| Meta-analysis | Statistical synthesis of many studies' effects | The overall verdict on a question | Only as good as the studies (and search) that went in |
| Primary empirical article | Original peer-reviewed study; has Method & Results | The actual evidence; what you cite | One study is one data point, never the final word |
| Review article / chapter | Expert synthesis, no new data | Mapping a literature fast | Author's framing; check the primaries it leans on |
| Science journalism | News story about research | Discovering that a study exists | Caveats stripped, claims inflated — trace it up the chain |
| Social media / AI summary | Unvetted paraphrase of a paraphrase | A lead, at most | May be distorted — or entirely invented |
The decay of findings into headlines has itself been studied. Sumner et al. (2014) analyzed 462 press releases from UK universities alongside the journal articles beneath them and the news stories built on them. Roughly a third to 40% of press releases exaggerated the research — stronger advice, causal language for correlational findings, human claims from animal studies. And the exaggeration propagated: when the press release inflated the claim, 58–86% of the resulting news stories inflated it too; when the release stayed honest, only 10–18% of stories exaggerated. Read that carefully: much of the distortion enters at the university's own press office, before a journalist ever touches it. So "chocolate is the new antidepressant!" was probably "small short-term mood association, n = 34" three hops upstream. When a claim matters to you, climb the chain to the primary source — the caveats you're missing are where the truth lives.
Start at your library's databases — PsycINFO indexes psychology specifically and lets you filter to peer-reviewed empirical work. Use Google Scholar for breadth, and lean on two features nobody teaches: the "Cited by" link, which jumps forward in time to show who built on a study (and whether anyone failed to replicate it — Lesson 15 will explain why you should check), and the library-links setting, which routes you to full text your school already pays for instead of a paywall. Then read with a purpose: abstract first — what's the claim, what's the evidence? — then jump to the section that answers your question. Nobody reads empirical articles front-to-back like novels, including the people who write them.
The newest hazard: generative AI writes confident, fluent summaries — and sometimes invents sources that don't exist. A chatbot will hand you a citation with a real-sounding author, a plausible journal, a clean DOI, and the paper is entirely fabricated. It's the availability heuristic weaponized: the citation looks right, so it feels right. Rule for this course and beyond: an AI summary is a lead, never a source. Before you cite anything an AI gave you, find the primary source yourself — search the title in Scholar, confirm the authors and journal, open the actual paper. If you can't find it, it probably isn't real. (Yes, this happens in submitted student papers. Yes, professors check.)
SourcesCited in APA 7
Bushman, B. J. (2002). Does venting anger feed or extinguish the flame? Catharsis, rumination, distraction, anger, and aggressive responding. Personality and Social Psychology Bulletin, 28(6), 724–731. https://doi.org/10.1177/0146167202289002
Chapman, L. J., & Chapman, J. P. (1967). Genesis of popular but erroneous psychodiagnostic observations. Journal of Abnormal Psychology, 72(3), 193–204. https://doi.org/10.1037/h0024670
Dawes, R. M., Faust, D., & Meehl, P. E. (1989). Clinical versus actuarial judgment. Science, 243(4899), 1668–1674. https://doi.org/10.1126/science.2648573
Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12(1), 19–30. https://doi.org/10.1037/1040-3590.12.1.19
Kahneman, D. (2011). Thinking, fast and slow. Farrar, Straus and Giroux.
Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. University of Minnesota Press.
Moore, D. A., & Healy, P. J. (2008). The trouble with overconfidence. Psychological Review, 115(2), 502–517. https://doi.org/10.1037/0033-295X.115.2.502
Nickerson, R. S. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175–220. https://doi.org/10.1037/1089-2680.2.2.175
Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231–259. https://doi.org/10.1037/0033-295X.84.3.231
Pronin, E., Lin, D. Y., & Ross, L. (2002). The bias blind spot: Perceptions of bias in self versus others. Personality and Social Psychology Bulletin, 28(3), 369–381. https://doi.org/10.1177/0146167202286008
Sumner, P., Vivian-Griffiths, S., Boivin, J., Williams, A., Venetis, C. A., Davies, A., Ogden, J., Whelan, L., Hughes, B., Dalton, B., Boy, F., & Chambers, C. D. (2014). The association between exaggeration in health related science news and academic press releases: Retrospective observational study. BMJ, 349, Article g7015. https://doi.org/10.1136/bmj.g7015
Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185(4157), 1124–1131. https://doi.org/10.1126/science.185.4157.1124