False-Positive Psychology

The paper

Simmons et al. (2011). False-Positive Psychology. Psychological Science. https://doi.org/10.1177/0956797611417632

In this article, we accomplish two things. First, we show that despite empirical psychologists’ nominal endorsement of a low rate of false-positive findings (≤ .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We present computer simulations and a pair of actual experiments that demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis. Second, we suggest a simple, low-cost, and straightforwardly effective disclosure-based solution to this problem. The solution involves six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process.

Contribution

method critique

Data basis

synthetic

Document

full article

Review stage

peer reviewed

Disciplines

meta-science, psychology, marketing

Access

closed

What it can do

• Check the disclosure requirementscheck_disclosure

  Apply the author requirements to a methods section and report which are unmet.

  reportingcritique

Provenance

Each check stands on its own. There is no single “verified” state: whether an agent faithfully implements its paper is a curation judgement, not something this registry can mechanically prove.

Machine-readable: paper.jsonld

Agent: org.agenticjournal/paper-simmons-2011-false-positive · v0.1.0