Abstract

Context: Educating the public through information campaigns is a commonly used policy approach to public health problems. Yet, experimental methods that assess the impact of information campaigns may misestimate their effects by failing to account for respondents' willingness to receive new information.

Methods: This article uses a doubly randomized survey experiment conducted on a nationally representative sample, where some subjects are randomly assigned to an informational treatment about opioids while other subjects are given the choice of whether to receive treatment or not, to examine how public willingness to seek new information shapes the way they update their preferences about policies related to the opioid epidemic.

Findings: Among those likely to receive information, treatment has a large positive effect on increasing support for policies that address the opioid epidemic by about one half of a standard deviation. Among those who would avoid this information, preferences appear to be unmoved by treatment. These effects would be missed by standard experimental designs.

Conclusion: While redressing information asymmetries is only one part of a public health strategy for addressing the opioid epidemic, our findings highlight the importance of access to and receptiveness toward new information.

Knowledge problems pervade the US opioid epidemic. Health professionals face continued uncertainty in understanding, assessing, and treating patient pain, especially chronic noncancer pain (Bonnie et al. 2019; IOM 2011; NAS 2017).1 Regulators and health professionals face ongoing challenges estimating the scope of the opioid epidemic and the effectiveness of efforts to address opioid use disorder, overdose, and death (Barocas et al. 2018). First responders, communities, and families at the frontlines of the epidemic grapple with ever-evolving terrains and uncertainties over how to promote care, safety, and recovery. These frontline members of the public are crucial actors in efforts to address the opioid epidemic. While knowledge alone cannot solve the opioid epidemic, redressing information asymmetries about opioids features prominently in long-standing justifications for regulatory policy making and for public health information campaigns.

Educating the public through information campaigns constitutes a commonly used policy approach to public health problems. Experimental approaches to assessing the impact of information campaigns, however, may misestimate the effects of such campaigns by failing to account for respondents' willingness to receive new information. Using a survey experiment of a national sample of 1,000 respondents conducted through the YouGov platform, this article examines public willingness to seek new information and update their preferences for health policies related to the opioid epidemic. Is the public receptive to new information on the sources of the opioid epidemic and possible remedies? What are the potential implications of the public's heterogeneity for the development and distribution of health information? We offer some preliminary insight on these puzzles. Notably, among individuals who are likely to receive information, our informational treatment has a large positive effect on increasing support for policies to address the opioid epidemic by about one half of a standard deviation. Among individuals who are likely to avoid this information, policy preferences appear to be unmoved by our treatment. These effects would be missed by standard experimental designs.

Promoting Public Health through Redressing Information Asymmetries

Educating the Public

Promoting public health through public information constitutes an enduring and foundational part of US health policy making at the federal, state, and local levels. Systematic reviews of public health information campaigns find evidence that information campaigns have had a positive impact on public health concerns ranging from tobacco use (National Cancer Institute 2008) to heart disease (Rochella 2002). Such campaigns are more likely to yield durable improvements in public health, however, when they occur in conjunction with commensurate policy changes and local service provision (Wakefield, Loken, and Hornik 2010).

Despite the potential benefits of information campaigns, formidable challenges confront efforts to educate the public on public health and disease prevention. In general, members of the public vary in their likelihood of encountering information (Zaller 1992); and when they do, they may be inclined to discount information that does not comport with their prior predispositions (Kunda 1990) or that comes from sources they deem not credible (Lupia 2013). Without sufficient context, reports of research findings can exacerbate fatalistic beliefs or induce information overload.2

These general concerns about the effectiveness of information campaigns manifest in opioid-specific experiments as well. In the context of opioids information and messaging, like other public health challenges, frames and images matter. Referring to the same type of intervention, using the phrase “overdose prevention sites” garnered more public support in experimental conditions than the phrase “safe consumption site” (Barry, Sherman, and McGinty 2018). The images that experiments use to portray opioid addiction also bear on public respondents' support for punitive policies or support for expanded insurance coverage (Kennedy-Hendricks, McGinty, Barry 2016). More broadly, the ways in which public health messages are framed can augment partisan differences in public opinion (Gollust, Lantz, and Ubel 2009). Moreover, correcting inaccurate myths about disease or interventions does not necessarily yield an improvement in behaviors that promote public health (Nyhan and Reifler 2015). These challenges manifest in regulatory agencies' information-based public health strategies as well.

Information, Education, and Regulation of Pharmaceuticals

Information as a policy instrument to influence health behaviors can target individuals directly, and it can pass through intermediaries such as physicians or pharmacists. The FDA, for instance, engages in both direct and indirect public education through postmarketing regulatory communication that comes in several forms including drug labels, medication guides, safety alerts, and warnings. Part of the logic of government-initiated information distribution assumes consumer empowerment: that information will facilitate patients' abilities to engage actively in their pharmaceutical choices, rather than rely solely on physicians as intermediary experts (Grossman 2014). In practice, patients and consumers struggle to understand the pharmaceutical information they receive, leading to frequent calls for clear, simple language when communicating with patients and consumers (Hoek et al. 2011). The effectiveness of these materials depends on consumers reading the label and absorbing the intended message. Yet, studies consistently demonstrate that patients and providers “do not consistently heed [drug] labels” developed to guide and support safe and effective drug use (IOM 2007: 59).3 When they do take the time to read a warning, they may or may not accurately act in ways consistent with the intent of the warning (Dusetzina et al. 2012). Consumers, for example, may interpret right-to-know disclosures as a strong warning against using the product at all, when the warning was intended to elicit a more moderated consumer response (Bar-Gill, Schkade, and Sunstein 2018).

While some studies find low consumer understanding of drug warnings overall (Ip et al. 2015), others highlight the risks and disadvantage facing consumers with limited English skills (Bailey et al. 2011) or limited health literacy (Yin et al. 2013).4 Pharmacies do not typically provide drug prescribing and safety information in patients' native languages (Bailey et al. 2011). Despite guidelines calling for the information on drug labels and in medication guides to be written at a 6th–8th grade reading level, medication guides, on average, are written at an 11th–12th grade reading level (Wolf et al. 2006). Moreover, many consumers lack the contextual knowledge needed to understand the words and figures presented in drug information, which makes it difficult or impossible for them to use the information appropriately, even if they successfully read it (Ben-Shahar and Schneider 2014). Low health literacy leads patients to have a harder time understanding medication guides (Wolf et al. 2012; Wolf et al. 2014), to be more likely to misinterpret the risks of a medical procedure (Ben-Shahar and Schneider 2014), or to be more likely take an unsafe dosage of over-the-counter medications (Yin et al. 2013). Uncertainty over the effectiveness of information campaigns also arises because of the piecemeal nature of most studies. Much of what we know about individual-level responses to safety information comes from studies of particular drugs (Kuehn 2012),5 yielding uncertainty over whether the findings extend to other disease indications or patient groups.

In the context of the opioid epidemic, the FDA, along with other federal agencies, has deployed a range of information-based policy approaches.6 These have included the 2003 FDA warning letter on oxycodone, the 2009 FDA education campaign for prescribers and patients, and the 2012 implementation of the Risk Evaluation and Mitigation Strategies for all opioids. Public health advisories have also emerged from the surgeon general, including the recent call for greater access to naloxone.7 Similarly, many state legislatures have enacted laws mandating the development and inclusion of education about opioids abuse in school curricula.8

Empirical Puzzles

Redressing information asymmetries, alone, will not solve the opioid epidemic. Yet efforts to inform the public remain part of the core mission of many agencies and often a key component of more holistic strategies to address this issue. Existing research has demonstrated the broad importance of the framing and content of information about opioids for the general public. Yet broader theories of public opinion imply a more complicated process only partly captured by past work.

Consider two examples using Zaller's (1992) general Receive-Accept-Sample model as a framework to illustrate how the results from an experiment may differ from the dynamics in the world. First, suppose that, as Zaller contends in his reception axiom, exposure to information is correlated with engagement on that issue. In an experiment where information is provided about a complex issue like opioids, the effects of such novel information may be large, since many people are encountering information they have never received before (and are unlikely to otherwise receive in the course of their daily lives). Yet among the small subset of those engaged with the issue—people who care about opioids and follow the issue closely—the effects may be small or nonexistent, as they already know the information being provided. Second, suppose exposure is correlated with prior beliefs. People likely to encounter the information respond as one would hope, becoming more informed and supportive of particular policy, while people who would choose to avoid this information resist the information provided because it conflicts with their prior beliefs, and they move their opinions in the opposite direction of the treatment's desired effects (Zaller's 1992 resistance axiom). Depending on the size of these two groups, the effects we observe in an experiment might be close to zero, leading us to conclude a campaign is ineffective when the campaign has the desired effect among the population it is likely to reach. Rather than try to infer these processes through measures like education or knowledge that may proxy for exposure and measures like partisanship or ideology that may proxy for resistance, we detail in the next section how scholars can learn more by trying to directly incorporate these dynamics into the design and analysis of their studies.

Data and Design

When will the public seek out information on opioids and what consequence will that information have? Standard survey experiments provide only a partial answer to these questions, offering an estimate of the effect of treatment when everyone receives it (Gaines, Kuklinski, and Quirk 2007). Instead, scholars and policy makers may wish to know the effects of some informational campaign on those likely to receive it. They may also wish to know how such messages will be received by audiences unlikely to be exposed to such information as divergent responses may suggest the need for alternative communication strategies.

We explore these questions through an experimental design in which some subjects are given an opportunity to choose whether to receive information about opioid abuse in the US, while others are randomly assigned exposure to these facts: information that they may or may not have otherwise chosen to avoid. Experimental designs of this type are common in public health, where they are generally known as patient preference trials (Long, Little, and Lin 2008; Rücker 1989; Torgersen, Klaber-Moffett, and Russell 1996). They have been used in political science to assess the effects of negative campaigns and partisan media (Arceneaux, Johnson, and Murphy 2012; Gaines, Kuklinski and Quirk 2007; Knox et al. 2019).

Our particular study employs a parallel design in which some subjects are randomly assigned to treatment while other subjects are given the choice of whether to receive treatment or not. Subjects were asked two questions about opioid abuse in the US: whether they personally knew someone who had been addicted to prescription painkillers or other opioids and for an assessment of the extent to which opioid abuse was a major problem in the area in which they live. Subjects were then randomly assigned to one of two experimental conditions. Approximately 30% of subjects were assigned to a simple experimental design in which they were randomly assigned to treatment or control with equal probability. Treated subjects were presented with a brief set of informational facts (figure 1) about the opioid crisis taken from an information project by the Truth Initiative, a nonprofit organization,9 before answering a set of outcome measures detailed below. The treatment aims to mimic the kind of information presented in public health campaigns and often cited in media coverage of the opioid epidemic. Subjects in the control proceed directly to the questions.

The remaining 70% of subjects were assigned to a selection experiment in which they were given the option, shown in figure 2, of whether to read some more information on the issue before providing their own on the views on the issue.

Subjects who responded “yes” received the information presented in figure 1. Subjects who responded “no” proceeded directly to our battery of outcome measures. For subjects who received the informational treatment, the survey paused for 5 seconds before respondents could advance to the next section.

Figure 3 provides a summary of our experimental design and helps illustrate two key strengths of parallel design. First, as the right-hand side of the figure shows, we can represent the average treatment effect from a standard survey experiment (E[Y|D = T] − E[Y|D = C]) as the weighted average of the treatment effects among those likely to seek out information τs and those likely to avoid such information τa. Second, as the left-hand side of figure 3 shows, we can estimate the proportion α of the people likely to seek out information on opioids when given the chance. With these quantities we can estimate what Knox et al. (2019) refer to as the Average Choice-Specific Treatment Effects (ACTEs) by taking the difference between the average outcome in the selection condition, (that is, the weighted average of those who selected both into and out of treatment, E[Y|C = C]), and the average of those in the control (in this study, those randomly assigned to read the treatment, E[Y|C = E, D = C]), and weighting this estimate by the proportion of people likely to select that treatment:

Similarly, we can recover the effect of treatment on those likely to avoid it (τa) by taking the difference between the average outcome among those assigned to treatment, E[Y|C = E, D = T], and the average outcome among those allowed to select into or out of treatment, E[Y|C = C], and weighting this difference by the proportion of those likely to avoid that treatment (1–α).

Standard errors for a ratio of estimates can be constructed via the delta method (Cameron and Trivedi 2005).

Data, Outcomes, and Measurement

The data for our study come from a larger survey of 1,000 respondents conducted by the Taubman Center for American Politics and Policy in the fall of 2018 and fielded by polling firm YouGov. YouGov uses matched sampling with poststratification weights to obtain a nationally representative sample from its online panel (Rivers 2006). The median respondent in our sample was a 47-year-old white woman with some college experience who identifies as being ideologically moderate and a political independent.

We consider three sets of outcomes designed to measure whether our treatment changed respondents' (1) objective knowledge about the opioid crisis, (2) beliefs about the primary cause of this crisis, and (3) support for general policy measures to address this issue. We measure factual knowledge with a binary indicator for whether subjects could correctly recall the percentage of heroin users who started with a prescription painkiller. To assess subjects' beliefs about the causes of the opioid crisis, we use two binary indicators capturing whether subjects believe the primary cause of the present situation is health care providers or illicit drug use. To capture policy preferences, we use a battery of seven items measuring agreement with various approaches to address the opioid epidemic through education, training, regulation, and the criminal justice system on a 7-point scale (1 = strongly disagree, 7 = strongly agree). For simplicity of discussion, we produce two scales from these seven items using principal components analysis to capture general support for more punitive and more treatment-oriented approaches to the opioid crisis. We use standard measures of demographic (age, income, education, gender, and race) and political variables (partisanship and ideology) to describe who is likely to seek out or avoid information and to estimate heterogeneous effects by subgroup to compare our estimates of the overall average treatment effects (ATE) and choice-specific ACTEs. Our survey also contains information on where subjects report receiving information about opioids, and how the opioid crisis makes them feel. These questions, however, were asked posttreatment, and we use them primarily for descriptive and exploratory purposes.

Results

We begin our discussion with a descriptive analysis of where people report receiving information about opioid addiction. Next, we examine the characteristics of who, when given the opportunity to receive more information about this crisis, chooses to receive or avoid that information. Then, we assess the effects of that information, looking first at whether this information increases objective knowledge about the crisis and then whether such information alters what people believe is the primary cause for this crisis and their beliefs about policies to address this crisis. Finally, we explore some possible explanations for why the effects of information appear to differ by the likelihood of receiving it.

Where Do People Get Information about Opioids?

Table 1 presents some descriptive statistics of the self-reported frequency with which people encounter information about opioid addiction from various sources.10 We see that the news media is far and away the most frequent source of information about opioid addiction with 40% of respondents reporting they frequently received information from the media and another 32% reporting receiving information from this source more infrequently. About half of all respondents report receiving at least some information from health care professionals, 43% report they received information from government agencies, and just over a third report having received information from family and friends. Local schools and pharmaceutical companies were less common sources of information for about a quarter of respondents. Overall, only about 20% of respondents reported receiving no information from any of the six sources, 9% of respondents reported receiving at least some information from all six sources, and nearly two-thirds of respondents reported receiving information from more than one source.

Who Seeks Out Information about the Opioid Crisis?

The results from the previous section suggest a number of policy-relevant patterns about the public's likelihood of encountering information on the opioid crisis. First, a significant portion of the public, about 1 in 5, reports receiving no information from any of the six sources we listed. Second, the remainder of our sample reports receiving information from a relatively diverse array of sources, with the media being the primary and most frequent source of information. We now consider characteristics of who is likely to seek out such information, when given the choice (albeit in the context of taking an online survey) to seek out additional information on the opioid crisis.

Of the 712 subjects assigned to the choice condition in our experiment, 395 elected to receive more information about the opioid crisis (55%), while 317 chose not to receive this information (45%). Figure 4 examines how these two groups of respondents differ along eight dimensions. Subjects who chose to receive additional information were about 8% more likely to report knowing someone who had struggled with addiction (46% vs. 38%, p < 0.05) and marginally more likely to report that opioid addiction was a major problem in their community (41% vs. 35%, ns) compared to subjects who opted not to view the additional information. People who elected to receive additional information were also more likely to have higher levels of education and income, less likely to be racial minorities, and more likely to identify as Democrats and liberals.

Effects of Information

The results from figure 4 suggest that people open to receiving information about the opioid crisis differ in a number of ways from those who, given the choice, might opt to avoid such information. It seems likely, then, that the effects of such information will vary conditional on the likelihood of receiving it.

We examine this possibility first with regard to the objective knowledge subjects gained from our experiment. Figure 5 shows the average treatment effect (ATE) in green, compared to average choice-specific treatment effects (ACTEs) among those likely to choose or avoid our informational treatment in red, as well as conditional average treatment effects (CATEs) in blue for the subgroups from the previous section. Point estimates, confidence intervals, and p-values are provided in table 2. Overall, the treatment appeared to increase the probability that respondents would correctly identify the percent of heroin users who started with prescription pain killers by 25 percentage points from 30% in the control 55% in treatment (p < 0.05). The effects are slightly larger for those likely to seek out this information (29 percentage points) compared to those who would avoid this information (21 percentage points), but the differences are nonsignificant. The effects are generally similar in size and sign across various subgroups, ranging from a minimum, nonsignificant increase of 6 percentage points for independents and a maximum increase of 47 percentage points among liberals (p < 0.05).

These initial results provide evidence that people did in fact receive and understand the treatment. But it should not be too surprising that when you provide people with simple, factual information many of them are able to recall this information when prompted shortly thereafter. The more interesting question is what people do with this information and how they update their beliefs and preferences.

Figure 6 examines the effects of treatment on subjects' beliefs about the primary cause of the opioid crisis—specifically, the extent to which subjects are likely to attribute responsibility for the present crisis to health care providers (left-hand panel) or illicit drug use (right-hand panel). Point estimates, confidence intervals, and p-values are provided in tables 3 and 4. Three features of these results are particularly striking. First, in both cases, a standard survey experiment would suggest that our informational treatment had no effect. Second, these null results for the ATE mask significant substantive differences uncovered by our ability to estimate ACTEs for people likely and unlikely to receive our treatment. Specifically, subjects who would opt to receive information when given the chance are more likely to attribute blame to health care providers and less likely to attribute blame for the current crisis to illicit drug use. In contrast, when subjects who would prefer not to receive any additional information about the opioid crisis do receive that information, they are no more likely to blame health care providers and in fact are about 15 percentage points more likely to say the current crisis is caused primarily by illicit drug use (p < 0.05). Third, standard subgroup analysis is unlikely to detect this heterogeneity as most of the CATEs are nonsignificant; and no clear pattern emerges across the few estimates that are significant. For example, why should personal experience and race condition attributions about health care providers but not about illicit drug use? Or, why should ideology and partisanship, significant predictors of subgroup heterogeneity in the previous section, not predict more variation here? Instead, by incorporating the dynamics of choice directly into the design and analysis of experiment, we have uncovered effects that we might have otherwise missed.

Next, we examine the extent to which our brief informational treatment influenced subjects' general support for policies to address the opioid crisis through more treatment-focused approaches emphasizing education, regulation, and health care, and more punitive approaches emphasizing arresting dealers and users. Figure 7 presents the ATE, ACTEs, and CATEs from our analysis using our summary measures of policy beliefs (point estimates, confidence intervals, and p-values in tables 5 and 6). Again, we see a similar pattern of results to what we found when considering causal attributions.

Among those likely to receive this information, treatment has a large positive effect on increasing support for treatment-oriented policies to address the opioid epidemic but no effect on support for more punitive approaches. Among those who would avoid this information, preferences appear to be unmoved by treatment. These effects would be missed if we looked only at the ATE from our experiment, and analysis of subgroups yields some idiosyncratic evidence of heterogeneity.

As noted above, people who chose to receive information about the opioid crisis differed in a number of observable ways from those who elected to avoid such information, but those differences fail to predictably explain the divergent pattern of responses we see among those likely and unlikely to receive our treatment. We conclude with a brief, exploratory analysis of some of the ways these two groups—information seekers and avoiders—differ in their emotional evaluations of the opioid crisis. While primarily a descriptive exercise, it may help explain why, when presented with the same information, people seem to respond in divergent manners. Figure 8 presents the average responses across each of our four treatment groups to a set of questions asking subjects how often they felt angry, empathetic, indifferent, powerless, resentful, and sad toward individuals addicted to opioids. In no case did our brief informational treatment appear to influence subjects' summary emotional evaluations of this larger issue. But as the bottom set of averages illustrate, people willing to seek out information on the opioid crisis feel very differently toward those addicted to opioids than those who would avoid such information do. Those open to receiving information are less likely to report feeling indifferent and resentful and more likely to report feeling empathetic, powerless, and sad. As an early exploratory exercise, these descriptive results may help shed light on why information effects these two groups differently.

Limitations, Alternative Explanations, and Future Research

In this section we consider some possible threats to the validity of our inferences as well as some alternative explanations for our reported results. First, while we believe our approach possesses greater external validity than standard forced-exposure designs common to other experimental studies, it is hard, if not impossible, to perfectly capture the informational choices of everyday life. Alternative designs might vary the type of information (e.g., factual statistics vs. personal narratives) or the format through which information is conveyed (text vs. audio), as well as the number of choices presented although increasing the diversity of choices requires imposing additional assumptions and analytical constraints (see Knox et al. 2019). While the present experiment may not perfectly capture real life (few if any do), we believe it provides a reasonable approximation of the kind of choices people make when deciding whether to read an article or throw out an informational postcard. Another fruitful direction for further research would be to examine the dynamics of choice over time (Broockman and Kalla 2016; Druckman, Fein, and Leeper 2012).

Second, a threat to the internal validity of our estimates arises from the possibility that when offered the choice to read information, people process that information differently than if they'd been randomly assigned to that information (Evans and Stanovich 2013; Kahneman 2011). For example, dual-process models of human cognition posit two systems for how individuals process information. System 1 captures the automatic processes like heuristics that individuals use to process information efficiently, while system 2 describes the more active, deliberate thinking individuals engage in when presented with a task that requires cognitive effort. It is possible, then, that most subjects in the experimental arm of the survey are engaged in the default of system 1 processing, while subjects given a choice are prompted to engage in more effortful, system 2 processing. A related concern comes from the possibility that our choice condition creates demand effects, where respondents are adjusting their responses to match the presumed desired responses of researchers (Orne 1962).

While these are important and valid concerns, we believe our main results still hold for the following reasons. First, with regard to demand effects, Mummolo and Peterson (2019) suggest online survey experiments may be less affected by such concerns and present evidence showing little difference in treatment effects even when subjects are provided with information and incentives related to an experiment's purpose (see also de Quidt, Haushofer, and Roth 2018). Second, if our study is subject to demand effects, it seems unlikely that the bias would be particularly larger among subjects given the choice to receive information compared to subjects randomly assigned to receive it. And indeed, for our measure of factual knowledge, the ATE and ACTEs are quite similar.

Where these estimates diverge, it is possible that these differences reflect a pattern of systematically different cognitive processing induced by our experimental treatment; but, again, we believe several factors weigh against this interpretation as the sole explanation for our results. First, for all subjects who received the information, the survey paused for a brief period (5 seconds) to ensure that subjects had time to read the facts. Second, the total survey times for respondents who selected to read the article where not statistically distinguishable from those randomly assigned to read the article. Third, we believe the descriptive differences in emotional evaluations of the opioid crisis by decision to encounter information are more consistent with persistent differences in how people process that information rather than design-induced changes in the way people cognitively processed our informational treatment. Still, we cannot rule out this interpretation—and future studies using similar designs might explicitly include measures designed to distinguish between attitude changes arising from system 1 versus system 2 processing.

Finally, we have broadly argued that our results show how the effects of information about the opioid crisis vary based on the likelihood that a person encounters it. For policy makers, we believe the effect of some intervention on its likely recipients is often of particular interest. Of course, policy makers and scholars may also wish to know why a treatment had the effect it did and why that effect varied. Here too, we believe our design offers added insights above standard approaches. As figure 4 shows, people opting into and out of our informational treatment differ along a number of theoretically relevant dimensions—people who received the information were more likely to have personal experience with the opioid crisis, to have higher levels of income and education, more likely to be white, and to be more Democratic and liberal. In further exploratory analysis conditioning on these subgroups, we find a similar pattern of effects to the main results among those without a college degree, but generally no effect among those with a 2-year degree or higher. With regard to partisanship, we find some suggestive and surprising patterns. For example, independents who would choose to receive information are more likely to attribute blame for the opioid crisis to health care providers, while independents who would avoid information are less likely to do so. Our claim is not that these factors don't matter, but instead that many of them are likely to matter in ways that condition both exposure and response to information. By allowing researchers to observe this process directly, our design creates opportunities and insights for future research to explore why one message worked the way it did or how a different message might be tailored to reach a particular audience in the context of the opioid crisis specifically or in the context of other public information campaigns more broadly.

Conclusion

Having a better-informed public is a foundational component of public health in the US. Yet, how to inform the public effectively remains elusive. Our parallel design, in which some subjects are randomly assigned to treatment while other subjects are given the choice of whether to receive treatment or not, offers a window into the public's heterogeneity that could benefit future information-focused efforts related to opioids. While our experimental intervention increased basic knowledge of the opioid crisis among all respondents, how that information shaped respondents' beliefs varied by how likely they were to encounter it. Among those likely to receive information, treatment altered attributions of blame for the opioid crisis, decreasing the likelihood of attributing the crisis to the behavior of users and increasing the likelihood of attributing the crisis to the behavior of health care providers. Respondents who were open to receiving this message were more supportive of treatment-oriented policies to address the opioid epidemic by about one half of a standard deviation. Among those who would avoid this information, policy preferences appear to be unmoved by treatment; and encountering this information increased the probability of blaming drug users for the current crisis. While redressing information asymmetries is only one part of a public health strategy for addressing the opioid epidemic, our findings highlight the importance of access to and receptiveness toward new information.

Recognizing that mitigating the current epidemic will require a wide range of interventions, our results highlight important terrain for future research.11 For one, the current structure of FDA impact analyses insufficiently considers the downstream effects of its information-based regulatory interventions. By that, we mean that the FDA does not sufficiently analyze whether the ways in which it reports information to the public—through labels, medication guides, advisories—will reach different kinds of patient groups (including differences by socioeconomic status, health literacy, English proficiency, and gender, among others). Nor does the agency assess potential disparate effects of those information campaigns on different groups. Future research could analyze whether there are ways government agencies' information-based efforts could incorporate the public's heterogeneity more fully into information designs and what effects that might have on information access and use.

For another, has the opioid epidemic damaged public trust in health care providers as credible sources of information? And, if so, what are the implications of this damage for future efforts to convey important information to the public via health care professionals? Recent surveys suggest “the public placed the most blame on doctors who inappropriately prescribe painkillers” when asked “who is mainly responsible for the growing [opioid] problem” (Blendon and Benson 2018). Given the importance of receiving information from credible sources, future research should examine who the public—or different subpopulations of the public—deem credible.

In the words of one former FDA director, drug safety and effectiveness “is a complex problem that is not solved by government, not solved by FDA alone, and not solved for just one drug” (FDA 2000). Information asymmetries are only one part of the complex set of reasons that the opioid epidemic emerged. Redressing information asymmetries will be only one part of mitigating the crisis. But, even within the narrow sphere of informational approaches, the results of our experiments highlight the complexity of understanding the effects of information.

Acknowledgments

We are grateful to the Taubman Center for American Politics and Policy for including our survey experiment on the 2019 fall YouGov survey.

Notes

1.

On physician payments and associations with opioid prescriptions, see Hadland et al. 2019.

2.

These findings focus on the differences that emerge between television and print reports of research findings in the case of cancer (Gollust, Fowler, and Niederdeppe 2019).

3.

Scholarship on mandated disclosures similarly suggests consumers often skip these warnings (Ben-Shahar and Schneider 2014).

4.

Regarding limited English skills, some states, including California and New York, require prescription containers to be translated. Laws do not require the translation of materials such as medication guides (Regenstein et al. 2012).

5.

For instance, much is known about the effects of warnings associated with antidepressant use (Parkinson et al. 2014; Valluri et al. 2010).

6.

The FDA has, of course, also deployed a range of regulatory approaches that go beyond information. The FDA can constrain patients' and providers' access to drugs through restrictions on who can prescribe the drugs, how drug refills can occur, and where refills can occur (GAO 2011: 17).

7.

HHS.org, Office of the Surgeon General. “U.S. Surgeon General's Advisory on Naloxone and Opioid Overdose.” www.surgeongeneral.gov/priorities/opioid-overdose-prevention/naloxone-advisory.html (accessed January 20, 2019).

8.

See the National Conference of State Legislature's “Injury Prevention Legislation Database | Opioid Abuse Prevention” (www.ncsl.org/research/health/injury-prevention-legislation-database.aspx; accessed June 15, 2019) for a searchable database. For example, in 2017, Pennsylvania passed HB 178, which directs the state's Department of Health and Department of Drug and Alcohol Programs to develop a model curriculum on the issue.

9.

See “Get the Facts” (www.thetruth.com/the-facts; accessed October 15, 2018).

10.

These informational questions were asked posttreatment, and percentages were calculated with responses for subjects in the control condition only (N = 143). Results are unchanged in terms of relative rankings using the full sample of respondents.

11.

On the range of community-based interventions for managing the epidemic, see Fraser and Plescia 2019.

References

References
Arceneaux, Kevin, Johnson, Martin, and Murphy, Chad.
2012
. “
Polarized Political Communication, Oppositional Media Hostility, and Selective Exposure
.”
Journal of Politics
74
, no.
1
:
174
86
.
Bailey, Stacy Cooper, Agarwal, Nisha, Sleath, Betsy, Gumusoglu, Serena, and Wolf, Michael S.
2011
. “
Improving Drug Labeling and Counseling for Limited English Proficient Adults
.”
Journal of Health Care for the Poor and Underserved
22
, no.
4
:
1131
43
.
Bar-Gill, Oren, Schkade, David, and Sunstein, Cass R.
2018
. “
Drawing False Inferences from Mandated Disclosures
.”
Behavioural Public Policy
3
, no.
2
:
209
27
. doi.org/10.1017/bpp.2017.12.
Barocas, Joshua A., White, Laura F., Wang, Jianing, Walley, Alexander Y., LaRochelle, Marc R., Bernson, Dana, Land, Thomas, Morgan, Jake R., Samet, Jeffrey H., and Linas, Benjamin P.
2018
. “
Estimated Prevalence of Opioid Use Disorder in Massachusetts, 2011–2015: A Capture–Recapture Analysis
.”
American Journal of Public Health
108
, no.
12
:
1675
81
.
Barry, Colleen L., Sherman, Susan G., and McGinty, Emma E.
2018
. “
Language Matters in Combatting the Opioid Epidemic: Safe Consumption Sites versus Overdose Prevention Sites
.”
American Journal of Public Health
108
, no.
9
:
1157
59
.
Ben-Shahar, Omri, and Schneider, Carl E.
2014
.
More Than You Wanted to Know: The Failure of Mandated Disclosure
.
Princeton, NJ
:
Princeton University Press
.
Blendon, Robert J., and Benson, John M.
2018
. “
The Public and the Opioid-Abuse Epidemic
.”
New England Journal of Medicine
378
, no.
5
:
407
11
.
Bonnie, Richard, Schumacher, Mark A., Clark, J. David, and Kesselheim, Aaron S.
2019
. “
Pain Management and Opioid Regulation: Continuing Public Health Challenges
.”
American Journal of Public Health
109
, no.
1
:
31
34
.
Broockman, David, and Kalla, Joshua.
2016
. “
Durably Reducing Transphobia: A Field Experiment on Door-to-Door Canvassing
.”
Science
352
, no.
6282
:
220
24
.
Cameron, A. Colin, and Trivedi, Pravin K.
2005
.
Microeconometrics: Methods and Applications
.
Cambridge
:
Cambridge University Press
.
de Quidt, Jonathan, Haushofer, Johannes, and Roth, Christopher.
2018
. “
Measuring and Bounding Experimenter Demand
.”
American Economic Review
108
, no.
11
:
3266
302
.
Druckman, James N., Fein, Jordan, and Leeper, Thomas J.
2012
. “
A Source of Bias in Public Opinion Stability
.”
American Political Science Review
106
, no.
2
:
430
54
.
Dusetzina, Stacie B., Higashi, Ashley S., Dorsey, E. Ray, Conti, Rena, Huskamp, Haiden A., Zhu, Shu, Garfield, Craif F., and Alexander, G. Caleb.
2012
. “
Impact of FDA Drug Risk Communications on Health Care Utilization and Health Behaviors
.”
Medical Care
50
, no.
6
:
466
78
.
Evans, Jonathan St. B. T., and Stanovich, Keith E.
2013
. “
Dual-Process Theories of Higher Cognition: Advancing the Debate
.”
Perspectives on Psychological Science
8
, no.
3
:
223
41
.
FDA (Food and Drug Administration)
.
2000
. “
Gastrointestinal Drugs Advisory Committee Meeting Transcript: Lotronex
.”
June
27
. wayback.archive-it.org/7993/20170403222328/https://www.fda.gov/ohrms/dockets/ac/cder00.htm#Gastrointestinal.
Fraser, Michael, and Plescia, Marcus.
2019
. “
The Opioid Epidemic's Prevention Problem
.”
American Journal of Public Health
109
, no.
2
:
215
17
.
Gaines, Brian J., Kuklinski, James H., and Quirk, Paul J.
2007
. “
The Logic of the Survey Experiment Reexamined
.”
Political Analysis
15
, no.
1
:
1
20
.
GAO (US Government Accountability Office)
.
2011
.
Prescription Pain Reliever Abuse: Agencies Have Begun Coordinating Efforts, but Need to Assess Effectiveness
. GAO 12–115.
Washington, DC
:
US Government Accountability Office
.
Gollust, Sarah E., Fowler, Erika Franklin, and Niederdeppe, Jeff.
2019
. “
Television News Coverage of Public Health Issues and Implications for Public Health Policy and Practice
.”
Annual Review of Public Health
40
, no.
1
:
167
85
.
Gollust, Sarah E., Lantz, Paula M., and Ubel, Peter A.
2009
. “
The Polarizing Effect of News Media on the Social Determinants of Health
.”
American Journal of Public Health
99
, no.
12
:
2160
67
.
Grossman, Lewis A.
2014
. “
FDA and the Rise of the Empowered Consumer
.”
Administrative Law Review
66
:
627
76
.
Hadland, Scott E., Rivera-Aguirre, Ariadne, Marshall, Brandon D. L., and Cerda, Magdalena.
2019
. “
Association of Pharmaceutical Industry Marketing of Opioid Products with Mortality from Opioid-Related Overdoses
.”
JAMA Open
2
, no.
1
:
e16007
.
Hoek, Janet, Gendall, Philip, Rapson, Lara, and Louviere, Jordan.
2011
. “
Information Accessibility and Consumers' Knowledge of Prescription Drug Benefits and Risks
.”
Journal of Consumer Affairs
45
, no.
2
:
248
74
.
IOM (Institute of Medicine)
.
2007
.
The Future of Drug Safety: Promoting and Protecting the Health of the Public
.
Washington, DC
:
National Academies Press
.
IOM (Institute of Medicine)
.
2011
.
Relieving Pain in America: A Blueprint for Transforming Prevention, Care, Education, and Research
.
Washington, DC
:
National Academies Press
.
Ip, Eric J., Tang, Terrill T.-L., Cheng, Vincent, and Cheongsiatmoy, Derren S.
2015
. “
Impact of Educational Levels and Health Literacy on Community Acetaminophen Knowledge
.”
Journal of Pharmacy Practice
2
, no.
6
:
499
503
.
Kahneman, Daniel.
2011
.
Thinking, Fast and Slow
.
New York
:
Farrar, Straus, and Giroux
.
Kennedy-Hendricks, Alene, McGinty, Emma E., Barry, Colleen L.
2016
. “
Effects of Competing Narratives on Public Perceptions of Opioid Pain Reliever Addiction during Pregnancy
.”
Journal of Health Politics, Policy and Law
41
, no.
5
:
873
916
.
Knox, Dean, Yamamoto, Teppei, Baum, Matthew A., and Berinsky, Adam J.
2019
. “
Design, Identification, and Sensitivity Analysis for Patient Preference Trials
.”
Journal of the American Statistical Association
. doi.org/10.1080/01621459.2019.1585248.
Kuehn, Bridget M.
2012
. “
Effects of FDA Safety Warnings May Vary
.”
Journal of the American Medical Association
307
, no.
9
:
894
95
.
Kunda, Ziva,
1990
. “
The Case for Motivated Reasoning
.”
Psychological Bulletin
108
, no.
3
,
480
98
.
Long, Qi, Little, Roderick J., and Lin, Xihong.
2008
. “
Causal Inference in Hybrid Intervention Trials Involving Treatment Choice
.”
Journal of the American Statistical Association
103
, no.
482
:
474
84
.
Lupia, Arthur.
2013
. “
Communicating Science in Politicized Environments
.”
Proceedings of the National Academy of Sciences
110
,
suppl. 3
:
14048
54
.
Mummolo, Jonathan, and Peterson, Erik.
2019
. “
Demand Effects in Survey Experiments: An Empirical Assessment
.”
American Political Science Review
113
, no.
2
:
517
29
.
National Academies of Sciences, Engineering, and Medicine
.
2017
.
Pain Management and the Opioid Epidemic: Balancing Societal and Individual Benefits and Risks of Prescription Opioid Use
.
Washington, DC
:
National Academies Press
.
National Cancer Institute
.
2008
. “
The Role of the Media in Promoting and Reducing Tobacco Use
.”
Tobacco Control Monograph No. 19. Bethesda, MD
:
US Department of Health and Human Services
.
Nyhan, Brendan, and Reifler, Jason.
2015
. “
Does Correcting Myths about the Flu Vaccine Work? An Experimental Evaluation of the Effects of Corrective Information
.”
Vaccine
33
, no.
3
:
459
64
.
Orne, Martin T.
1962
. “
On the Social Psychology of the Psychological Experiment: With Particular Reference to Demand Characteristics and their Implications
.”
American Psychologist
17
, no.
11
:
776
83
.
Parkinson, Kristy, Price, Joseph, Simon, Kosali I., and Tennyson, Sharon.
2014
. “
The Influence of FDA Advisory Information and Black Box Warnings on Individual Use of Prescription Antidepressants
.”
Review of Economics of the Household
12
, no.
4
:
771
90
.
Regenstein, Marsha, Andres, Ellie, Nelson, Dylan, David, Stephanie, Lopert, Ruth, and Katz, Richard.
2012
. “
Medication Information for Patients with Limited English Proficiency: Lessons from the European Union
.”
Journal of Law, Medicine, and Ethics
40
, no.
4
:
1025
33
.
Rivers, Douglas.
2006
. “
Understanding People: Sample Matching: Representative Sampling from Internet Panels
.” YouGovPolimetrix. www.websm.org/uploadi/editor/1368187057Rivers_2006_Sample_matching_Representative_sampling_from_Internet_panels.pdf (accessed
January
14
,
2020
).
Rochella, Edward J.
2002
. “
The Contributions of Public Health Education toward the Reduction of Cardiovascular Disease Mortality: Experiences from the National High Blood Pressure Education Program
.” In
Public Health Communication
, edited by Hornik, Robert C.,
73
83
.
Mahwah, NJ
:
Lawrence Erlbaum Associates
.
Rücker, Gerta.
1989
. “
A Two‐Stage Trial Design for Testing Treatment, Self‐Selection, and Treatment Preference Effects
.”
Statistics in Medicine
8
, no.
4
:
477
85
. doi.wiley.com/10.1002/sim.4780080411.
Torgerson, David J., Klaber-Moffett, Jennifer, and Russell, Ian T.
1996
. “
Patient Preferences in Randomised Trials: Threat or Opportunity?
Journal of Health Services Research and Policy
1
, no.
4
:
194
97
. doi.org/10.1177/135581969600100403.
Valluri, Satish, Zito, Julie M., Safer, Daniel J., Zuckerman, Ilene H., Mullins, C. Daniel, and Korelitz, James J.
2010
. “
Impact of the 2004 Food and Drug Administration Pediatric Suicidality Warning on Antidepressant and Psychotherapy Treatment for New-Onset Depression
.”
Medical Care
48
, no.
11
:
947
54
.
Wakefield, Melanie A., Loken, Barbara, and Hornik, Robert C.
2010
. “
Use of Mass Media Campaigns to Change Health Behavior
.”
Lancet
376
, no.
9748
:
1261
71
.
Wolf, Michael, King, Jennifer, Wilson, Elizabeth A. H., and Curtis, Laura M.
2012
. “
Usability of FDA-Approved Medication Guides
.”
Journal of General Internal Medicine
27
, no.
12
:
1714
20
.
Wolf, Michael S., Bailey, Stacy C., Serper, Marina, Smith, Meredith, Davis, Terry C., Russell, Allison L., Manzoor, Beenish S., Belter, Lisa, Parker, Ruth M., and Lambert, Bruce.
2014
. “
Comparative Effectiveness of Patient-Centered Strategies to Improve FDA Medication Guides
.”
Medical Care
52
, no.
9
:
781
89
.
Wolf, Michael S., Davis, Terry C., Shrank, William H., Neubergerd, Marolee, and Parker, Ruth M.
2006
. “
A Critical Review of FDA-Approved Medication Guides
.”
Patient Education and Counseling
62
, no.
3
:
316
22
.
Yin, H. Shonna, Mendelsohn, Alan L., Nagin, Perry, van Schaick, Linda, Cerra, Maria E., and Dreyer, Benard P.
2013
. “
Use of Active Ingredient Information for Low Socioeconomic Status Parents' Decision-Making Regarding Cough and Cold Medications: Role of Health Literacy
.”
Academic Pediatrics
13
, no.
3
:
229
325
.
Zaller, John.
1992
.
The Nature and Origins of Mass Opinion
.
Cambridge
:
Cambridge University Press
.