Can Fact-checking Prevent Politicians from Lying?
Abstract: Journalists now regularly trumpet fact-checking as an important tool to hold politicians accountable for their public statements, but fact checking's effect has only been assessed anecdotally and in experiments on politicians holding lower-level offices. Using a rigorous research design to estimate the effects of fact-checking on presidential candidates, this paper shows that a fact-checker deeming a statement false false causes a 9.5 percentage points reduction in the probability that the candidate repeats the claim. To eliminate alternative explanations that could confound this estimate, I use two types of difference-in-differences analyses, each using true-rated claims and "checkable but unchecked'' claims, a placebo test using hypothetical fact-check dates, and a topic model to condition on the topic of the candidate's statement. This paper contributes to the literature on how news media can hold politicians accountable, showing that when news organizations label a statement as inaccurate, they affect candidate behavior.
Data Available Upon Request
Checking How Fact-checkers Fact-check
Abstract: Fact-checking has gained prominence as a movement to revitalize truth-seeking ideals in journalism. While fact-checkers are often assumed to code facts accurately, few studies have formally assessed fact-checkers' overall performance. I evaluate the performance of two major fact-checkers, Fact Checker and Politifact, comparing their interrater reliability using a method that is regularly utilized across the social sciences. Surprisingly, only 1 in 10 statements is fact-checked by both fact-checkers. Among claims that both evaluate, fact-checkers perform fairly well on outright falsehoods or obvious truths. However, the agreement rate is much lower for statements in the more ambiguous scoring range (i.e., ``Half True’’ or ``Mostly False’’). The results suggest that fact-checking is difficult, and challenging to validate. Fact-checkers rarely evaluate the same statement, do not use directly comparable scales, and disagree more often than one might suppose. Politicians' strategic ambiguity may impede the fact-checking movement's goals, at least in some cases. Replication Codes and Data (Available upon Request)
The Limited Effect of Presidential Public Appeals
(With Annie Franco and Justin Grimmer)
Abstract: Presidents regularly make appeals to the public, making use of the presidential ``bully pulpit". The literature disagrees on what effects presidential public appeals have in part because existing research designs are ill equipped to assess the effect of presidential speeches. We use a wide array of new data and an interrupted time series design to identify the effect of the president's public appeals. Using a collection of serendipitous surveys, we show that presidents have almost no effect on survey responses identifying salient issues, public opinion on the issue the president makes a speech about, or about the president himself. But presidents are able to exercise a short term influence on news outlets and social media, but the effect dissipates quickly. Our findings refute several important theories and introduce a new puzzle: why do presidents continue to make addresses targeted at the public that have such a small effect?