Media Narratives of Police Use-of-Force

What do we know about media coverage of police use-of-force and its effect on public opinion? While it is axiomatic to say that news media influences public opinion, surprisingly little work has been done to understand how the news media frames the narrative(s) surrounding police use-of-force over time. This is not to say that no one has studied media effects on public opinion surrounding police use-of-force. Lawrence (2000)¹ and Fridkin and colleagues (2017)² are just a couple of works I have found immensely influential. However, these works typically examine media surrounding a specific controversial use-of-force event. Or, if they examine news media coverage over a longer period of time, the sample size is relatively small. This ‘small-n’ problem is a function of the constraints that conventional qualitative work places on researchers, namely time.

What is largely missing from the literature, and what impedes a better understanding of media effects on public opinion of police use-of-force, is longitudinal work with large corpora of news stories. I aimed to begin to fill in that gap in our understanding with my dissertation work. A brief and consolidated portion of that work makes up this post.

A New Corpus of News Stories

I have compiled a new dataset of 259,575 English-language media stories mentioning police use of force within the United States, covering the period of 1990 to 2021. The sources for these stories came from 2,770 different publications/platforms, including national, regional, and local newspapers (such as Wall Street Journal, Washington Post, New York Times, Tampa Tribune, Wisconsin State Journal, etc.), cable news television transcripts (FOX News, CNN, MSNBC, etc.), broadcast news transcripts (NBC, ABC, CBS, FOX), radio transcripts (NPR, etc.), magazines (Newsweek, American Rifleman, etc.), news wires (e.g., AP), and government press releases (Department of Justice, Federal Register, etc.).

Due to the computational demand for such a large corpus, I restricted the corpus to the following newspaper publications: USA Today, New York Times, Washington Post, St. Louis Post, Atlanta Journal-Constitution, and Wall Street Journal. I chose these publications guided by Enns’ (2016)³ work. Enns explains that using newspapers, while not perfect, is often the best option when working with news time-series data.

The corpus was further narrowed by restricting the analysis to headlines. Headlines were chosen as the unit of analysis for three reasons. First, research has found that more than half of news consumers do not attend to information beyond headlines. Accordingly, if there is an association with public opinion, one may expect headlines to have more of an effect than the actual stories since the majority of headline viewers do not read the body of the story and thus form their impression of the information being provided from the headlines. Second, Lawrence (2000) recognizes this in her research by suggesting that framing effects may be more detectable in the headlines than in the whole story. Finally, although of lesser importance, reducing the corpus to headlines reduces the computational demand on the machine-learning algorithms employed.

When the corpus is narrowed in the manner described above, I am left with 29,035 headlines consisting of 331,593 words, spanning 1990 to 2021. The mean word count per headline is 11.42, with a standard deviation of 7.33.

One simple but informative way to gain an initial understanding of a corpus is to plot the most frequent terms within it. The figure below plots the twenty most frequent terms within the headline corpus. One can begin to discern prevalent themes in headlines. For example, the five most frequent terms in the headlines are police officer shooting black man. Looking at the other fifteen most frequent words, one can see that ‘ferguson’ was a frequent topic, as well as ‘trump’ and ‘protests.’

However, unigram frequency analysis, while helpful, is an unsharpened instrument for understanding a corpus. Next, I move beyond using simple term frequencies as a measure by identifying latent themes that consider probabilistic word relationships.

Topic Modeling

For the particular question at hand, topic modeling is ideal. I prefer using structural topic modeling (STM) for several reasons I do not take the time to explain in this post. If you are interested in learning more about STM, you can read more about it in this paper I co-authored with Ian T. Adams. In a nutshell, STM identifies underlying latent variables through an unsupervised machine learning process in which themes are inferred from the distribution of words across topics. The topics are generated inductively from the data, utilizing a probabilistic approximation of Bayesian inference. Essentially, STM counts words, determines the probability of words appearing with other words within and across documents (i.e., headlines), and estimates the best fit for latent themes based on those probabilities.

What do the News Media’s Narrative Look Like Across Time?

There are two different ways of looking at the prevalence of each topic across the 32-year timespan studied (the Garbage Can Topic is omitted). Recall that structural topic modeling allows each document to represent a proportion of topics (i.e., a document may have multiple topics within it). For the purpose of the first figure below, the histograms represent the distribution of topics each document is most strongly associated with. The next figure plots loess smoothed estimates of topic proportions across time; that is, how prevalent a topic is across time and documents, rather than forcing each document into one topic like in the first figure.

When documents are not forced into one topic, we see some interesting patterns with the Controversies Topic (ebbs and flows across time with additional peaks in 1990 and 2000), the Race Topic (peaking during the Ferguson and Floyd periods), and the Protest and Reform Topic (showing a positive trend across time with a strong peak during the Floyd period).

Furthermore, the Just the Facts Topic appears to be the inverse of the Controversies Topic. That is, when policing controversies are not occurring, the Just the Facts Topic dominates, and vice versa. It is also noteworthy that the Just the Facts Topic drops to its lowest proportional levels during the Ferguson and Floyd periods. This is a different pattern than what is seen when forcing each document into one topic.

When examining the Just the Facts Topic histogram, it suggests that Just the Facts-related documents increase along with the controversial incidents of Ferguson and Floyd, suggesting that the media pays more attention to police use-of-force in general following significant incidents—a sort of forward-feeding effect. However, when examining topic proportions, a more nuanced pattern emerges. While more attention is paid to additional incidents of police use-of-force, even when official statements are being made, proportionally speaking, those headlines contain fewer official statements within them than during other periods. This suggests that other topics (i.e., Race Topic, Protest and Reform Topic, National Politics Topic, Ferguson Topic, Floyd Topic) are being combined with official statements, providing a narrative of race issues, political issues, and protest issues much more than in other periods.

Do News Media Narratives have an Association with Public Opinion?

In previous work⁴, Ian T. Adams and I demonstrated how public opinion of legally reasonable police use-of-force has steadily become more disapproving across the past four decades. We used three General Social Survey (GSS) questions that have been consistently asked since 1973 to examine public attitudes toward police use-of-force. We constructed an additive scale measuring public perceptions of legally reasonable police use-of-force by combining the following three questions:

• POLHITOK: Are there are situations you can imagine in which you would approve of a policeman striking an adult male citizen?

• POLESCAP: Would you approve of a policeman striking a citizen who was attempting to escape from custody?

• POLATTAK: Would you approve of a policeman striking a citizen who was attacking the policeman with his fists?

What these three questions share is the presumption of legal reasonableness under the legal and professional guidelines by which police officers’ actions are judged.

The table below provides the minimum and maximum values seen across the studied period for each question. POLESCAP experienced the largest percentage increase (19.91%), with POLHITOK and POLATTAK experiencing similar increases (12.54% and 13.43%, respectively). While POLATTAK has the lowest maximum disapproval rating of all three measures, it is a stark observation that in 2021, one in five respondents in a nationally representative survey answered that they disapproved of a police officer striking someone who was physically attacking the officer.

The figure below plots all three disaggregated measures alongside the public opinion measure validated through Confirmatory Factor Analysis for comparison purposes. The public opinion measure is what is used in the next analysis.

Finally, we regress the topics onto the public opinion measure. Recall, the public opinion measure is disapproval of legally reasonable police use-of-force. Thus, a positive coefficient indicates that a narrative increases public disapproval of legally reasonable use-of-force. A negative coefficient indicates that a narrative decreases disapproval of legally reasonable use-of-force.

As you can see in the plot above, all narratives except for ‘Just the Facts’ have a significant effect on disapproval of legally reasonable use-of-force. Bayesian methods are used, so one can think of the error bars as 95% of the probability distribution around the mean for each coefficient. Further, all variables are centered, so coefficients are interpreted as standard deviations.

The Floyd, Ferguson, Protest & Reform, and National Politics Topics have increased public disapproval of legally reasonable use-of-force. The Race and Controversies Topics decreased public disapproval of legally reasonable use-of-force.

Conclusion

What do the above findings tell us? There is much more than I can go into here (which is why I wrote a dissertation on this topic!). However, I’ll end with a few thoughts.

• Public opinion tends to be stable, with substantial, sustained movement in one direction being rare. Public disapproval of legally reasonable police use-of-force has not been stable and consistently increasing over the past three decades.

• The National Politics Topic had the largest effect on public opinion. Increasing partisanship within the US may be driving individuals to ‘take sides’ on the issue of police use-of-force in line with their identified political party’s views. However, public disapproval of legally reasonable police use-of-force has been growing amongst individuals of all partisan identifications (not shown here).

• Interestingly, the Race Topic coefficient was negative. A rebound effect has been identified in past literature whereby calling attention to a particular undesirable topic (i.e., alleged racism in policing) may lead individuals to stronger resistance to that idea. I do not know if that is what is going on here, but it is one possible explanation.

• Other research has found that there is often a sizable increase in unfavorable attitudes toward the police in the wake of a highly publicized use-of-force event. Therefore, the findings that the elevated attention news media provided to the events in Ferguson and events surrounding the murder of George Floyd had a high probability of affecting public opinion is plausible.

My dissertation work does not provide conclusive answers to these questions, except that there is clearly a relationship between media coverage of police use-of-force and public disapproval of legally reasonable use-of-force (and that, likely, public opinion follows media coverage and framing rather than vice versa; analyses not shown here). However, for the purposes of this post, I hope one of the primary takeaways is that while we know words and narratives matter, criminal justice research has struggled with ways in which to measure narratives. I hope that the tools briefly showcased here will lead to further development of quantitative text-based methods. I hope to have more to share on these methods in the near future.