The Patient-Centered Outcomes Research Institute (PCORI) was established as part of the Affordable Care Act to promote research on the comparative effectiveness of treatment options. Advocates hoped this information would help reduce wasteful spending by identifying low-value treatments, but many conservatives and industry groups feared PCORI would ration care and threaten physicians' autonomy. PCORI faced three challenges during its first decade of operation: overcoming the controversy of its birth and escaping early termination, shaping medical practice, and building a public reputation for relevance. While PCORI has won reauthorization, it has not yet had a major impact on the decisions of clinicians or payers. PCORI's modest footprint reflects not only the challenges of getting a new organization off the ground but also the larger political, financial, and cultural barriers to the uptake of medical evidence in the US health care system. The growing attention among policymakers and researchers to provider prices (rather than utilization) as the driver of health care spending could be helpful to the political prospects of the evidence-based medicine project by making it appear to be less as rationing driven by costs and more as an effort to improve quality and uphold medical professionalism.
The central purpose of the Affordable Care Act (ACA) is to expand access to affordable health insurance, but its secondary goal is to curb the growth of medical spending. Among the ACA's cost-control provisions that many experts hoped would “bend the cost curve” were limits on tax breaks for high-cost employer-sponsored health insurance (the Cadillac tax), accountable care organizations (ACOs), bundled payments, and an expert commission to develop recommendations for reducing excess Medicare spending growth (the Independent Payment Advisory Board). The premise behind these provisions is that a great deal of US medical spending is wasteful, and that it is possible to curb costs without damaging quality. While the ACA has achieved savings through cuts in reimbursements to Medicare providers, the cost controls mentioned above have not been successful. Both the Cadillac tax and the Independent Payment Advisory Board sparked backlash from stakeholder groups, including unions, employers, providers, and the medical products industry. Neither ever went into effect, and both provisions were ultimately repealed on a bipartisan basis (Oberlander and Spivack 2018; Suderman 2020). While ACOs and bundled payments have survived, they have produced only modest savings (Frakt 2019b; Colla and Skinner 2019).
Perhaps the least-understood element of the ACA's cost-control framework is the Patient-Centered Outcomes Research Institute (PCORI), which was created to fund and disseminate research findings on the comparative effectiveness of treatments, diagnostic tests, and other options. Surprisingly little scientific evidence exists about the benefits and harms of many medical services. When more than one treatment is available for a condition—such as drugs and a medical procedure—there is rarely good evidence about which one is most beneficial (Donnelly 2010). While estimates vary about what fraction of the treatments provided to patients is supported by evidence, some studies place the figure at under half (IOM 2007). As Austin Frakt observes, “The British Medical Journal sifted through the evidence for thousands of medical treatments to assess which are beneficial and which aren't. According to the analysis, there is evidence of some benefit for more than 40 percent of them. Only 3 percent are ineffective or harmful; a further 6 percent are unlikely to be helpful. But a whopping 50 percent are of unknown effectiveness. We haven't done the studies” (2019a: 15).
Experts have long hoped that better evidence about what works and for whom could identify opportunities to reduce low-value spending (Wilensky 2006; Ashton and Wray 2013). As Peter R. Orszag and Ezekiel J. Emanuel (2010: 602), who were key advocates for an expanded government role in supporting comparative effectiveness research during their tenure in the Obama administration, wrote, PCORI “will empower physicians and patients with new information regarding the effectiveness of various medical technologies and interventions. The integration of the PCORI's research findings with decision supports, guidelines, and other aspects of electronic health records should greatly enhance the information that physicians and patients can use in choosing the right tests and treatments for a particular situation.” In short, advocates hoped PCORI would provide the informational foundation of a high-performance delivery system.
In this essay, I discuss three key challenges that PCORI faced during its first decade of operation: to overcome the controversy of its birth and escape early termination, to begin to have a meaningful impact on medical practice, and to build a public reputation among key stakeholders for relevance. While PCORI has survived and won reauthorization, it has not yet had significant impact on the clinical behavior of physicians or the reimbursement and coverage decisions of payers. PCORI's relatively small impact on key stakeholders reflects not only the challenges of getting a new organization off the ground but also the larger political, financial, and cultural barriers to the uptake of medical evidence in the US health care system. Removing these barriers will require physicians to embrace a more robust understanding of their professional obligations to promote scientific research and ensure its appropriate use. Public opinion research suggests that if physicians (who enjoy a high degree of social trust) were to signal their support for the evidence-based medicine (EBM) project, it would lessen the public's anxiety about losing access to medical services. That, in turn, would make it more likely that politicians would embrace needed changes in the policy and regulatory environment to promote the uptake of evidence (Patashnik, Gerber and Dowling 2017). Until these broader changes happen, PCORI's influence is likely to remain modest. However, the growing attention among policymakers and health economists to provider prices (rather than utilization) as a key driver of health care spending could be helpful to EBM's political prospects by making this project appear to be less as rationing driven by costs and more as an effort to improve quality and uphold medical professionalism.
Why Worry about the Lack of Medical Evidence?
Before reviewing PCORI's development and performance, it is useful to pause and ask why people should care about the delivery of care unsupported by evidence in the first place. One reason is to reduce waste in the US health care system. While some patients fail to receive recommended medical services (McGlynn et al. 2003), the overutilization of low-value services is a major contributor to inefficiency. During the 2000s (the period when the ACA's cost-control ideas like ACOs, bundled payments, and comparative effectiveness research were being incubated), Dartmouth economists focused policy makers' attention on the geographic variation in utilization and spending in Medicare (Fisher et al. 2003a, 2003b; see also Wennberg 1984). Their research showed that the additional services provided in high spending regions did not produce better outcomes for Medicare enrollees, even after controlling for differences in illness rates among patients and regions. These findings were interpreted to mean that spending in high-utilization/high-cost areas could be reduced without adverse effects on health (CBO 2007).
Policy conclusions based on research on variations in Medicare spending during this period tended to implicitly assume that per capita spending patterns in Medicare and the commercial insurance sector were strongly correlated (Chernew et al. 2010). A 2010 study by Harvard economists (Chernew et al. 2010) and a subsequent study three years later (Newhouse et al. 2013) showed that this assumption is not correct, and that provider prices (not the quantity of medical services consumed) is the key to variations in spending in private health plans. In an important recent study based on an analysis of a massive amount of previously unavailable claims data, Zack Cooper and colleagues (2019) demonstrated that prices in commercial insurance markets serving people under age 65 vary dramatically. The authors found about half of the spending variation for the privately insured is driven by price variation across regions, and half is driven by quantity variation. By contrast, 95% of the spending variation in Medicare (which sets prices administratively) is due to variation in the quantity of care delivered.1 Taken together, this body of research suggests that policy makers who seek to reduce the overall growth of US medical spending must develop strategies to control prices and quantities. 2
As appropriate as the current focus is on prices, it remains important to address the costs to patients and society associated with treatments that are based on weak evidence of their comparative effectiveness, such as proton beam therapy. One reason to retain a focus on such services is government budget deficits (Skinner, Goodman, and Fisher 2015). While excessive prices are a major problem in commercial insurance markets, the prices US providers are paid by the Medicare and Medicaid programs “are not hugely out of sync with the regulated prices set in other countries” (Cooper 2019). There may be limits to how much lower such regulated prices can be pushed. Policy makers who seek to curb inefficiencies in programs like Medicare and Medicaid may need to consider stronger reforms to reduce the overuse of low-value services.
A second and even more important reason to be concerned about the delivery of dubious or wasteful treatments is patient welfare. What ultimately matters is not how much treatments cost, but whether they effectively treat illness. When patients are overtreated or receive treatments that work less well than alternatives do, they suffer harms (Brownlee 2010). One unnecessary procedure can lead to a cascade of others, such as giving older men no-longer-recommended prostate-specific antigen tests, which can lead to unnecessary biopsies and complications (Ganguli 2020).
In sum, the delivery of medical services not based on solid evidence was a serious problem before the enactment of the ACA in 2010. It remains an important concern today.
While information about the comparative benefits and harms of treatment options is a public good, proposals to establish a comparative-effectiveness agency sparked controversy during the ACA debate. Drug and medical device companies, along with many Republicans in Congress, feared the agency would ration care, chill biomedical innovation, and interfere with the doctor-patient relationship (Avorn 2009; Iglehart 2010; Sorenson, Gusmano, and Oliver 2014). To dispel these concerns, lawmakers designed PCORI as an independent, nonprofit institute and gave it a fairly small budget and narrow authority. In contrast to the National Institute for Health and Care Excellence in the UK, PCORI has no linkages to policy-making bodies. It was barred from mandating coverage, reimbursement, and other policy decisions, and from using quality-adjusted life years as a threshold to determine what services are cost effective (Neumann 2010). In sum, Congress recognized that there is too little solid medical evidence, and created PCORI to subsidize its production. However, legislators were wary of giving PCORI regulatory powers. The term patient-centered in PCORI's name appears to have been added at the behest of the medical products industry (Ashton and Wray 2013). This strategy may have been intended to keep PCORI tethered to the demands of patient groups (some of which receive industry subsidies) who seek unfettered access to treatments and medical technologies (Keller et al. 2019).
Unlike the Independent Payment Advisory Board (which Congress repealed in 2018) PCORI has passed the political survival test. Congress reauthorized PCORI through 2029 on a bipartisan basis as part of a year-end spending package in December 2019. The renewal featured relatively minor tweaks to PCORI's financing, governance, and research mission (Friends of PCORI Reauthorization 2019).3 More far-reaching changes, such as a bipartisan Senate proposal to establish an expert advisory panel for “high impact research” to advise PCORI on ways to better target medical conditions and services that have a large impact on national health spending, did not make it into the legislation. Republicans' support for PCORI has increased over the past decade as they have gained confidence that PCORI would not use its research to issue payment and coverage recommendations. Former member of Congress Phil Gingrey (2017), MD (R-Ga.), an ob/gyn who served as chair of the House GOP Doctors Caucus, told a reporter that he initially feared PCORI would “morph into a mechanism to ration care,” but that his thinking evolved. “As I look back on it now, I can clearly see that that section we were so worried about, in regard to comparative effectiveness research interfering with the doctor-patient relationship and the patient not having sufficient input in research and decision-making, [that didn't happen],” he said (quoted in Frieden 2018). PCORI's reauthorization also reflected support among patient groups, medical schools, and researchers who wish to see the PCORI grant funding stream continue.
In its 2018 Annual Report and fact sheets circulated during the debate over its reauthorization, PCORI summarized its main accomplishments along three dimensions—generating evidence, achieving impact, and influencing others. PCORI has awarded more than $2.6 billion in research grants over its first decade through more than 770 projects (PCORI 2020). PCORI reported that 891 publications in peer-reviewed journals have resulted from studies fully or partially funded by the institute through late 2017 (GAO 2018). PCORI noted that its research had appeared in 39 citations of clinical guidelines. The institute also highlighted several of its funded studies whose results, if applied, could improve outcomes for patients and the health care system. For example, one PCORI study demonstrated that the 7.1 million aged 30 and older with type 2 diabetes who do not use insulin don't have to check their blood sugar daily. PCORI stated this could avoid 10 billion blood glucose tests with the average patient saving about $1,600 in testing supply costs over five years (PCORI 2019). With respect to policy influence, PCORI noted that the Center for Medicare and Medicaid Services had used the results of its studies as the basis for its national coverage decision on implantable cardiac defibrillators (7).
While these are noteworthy achievements for a new research institute, the results produced by its studies have not yet had a major impact on the decisions of clinicians or payers. PCORI was slow out of the gates in sponsoring comparative effectiveness studies, even taking into account its relatively small budget compared to the NIH and other agencies. Many of PCORI's early grants focused on building infrastructure and promoting patient education and engagement to fulfil the “patient-centered” aspect of its mission (Kaiser 2019). The Center for American Progress, a liberal think tank, issued a critical report on PCORI's early progress (Tanden et al. 2014). While PCORI's research program began to accelerate by 2016, it has not yet had a major influence on the use of low-value treatments. The General Accountability Office reported that stakeholder organizations believe that PCORI has “changed the way research is conducted for the better, such as prioritizing research outcomes that are most meaningful to patients” (GAO 2018: 13–14). However, officials representing payers and health insurance plans told the GAO that “PCORI's priorities did not fully align with their needs, such as their needs for CER on certain high-cost conditions, medications or treatments” (GAO 2018: 14). As William V. Padula and R. Brett McQueen (2019: 757) wrote, “Despite its many achievements, PCORI is not directly associated with economically meaningful improvements in health system efficiency.” Nor have its studies had a major impact on payers who set coverage and reimbursement policies. PCORI's research “hasn't really informed our decision-making as of yet,” said Kate Berry, senior vice president of America's Health Insurance Plans (quoted in Kaiser 2019). In sum, PCORI's focus on patient engagement is shaping the way some research is conducted and its results are appearing in journals, but it has not yet had a significant impact on clinical practice or on spending or payment decisions.
Building a Reputation
Commenting on PCORI's inability to “rattle off five or 10 things” the agency has done that have had a major impact on medical practice, Zeke Emanuel argues that its research agenda hasn't been “bold enough” (quoted in Kasier 2019: 1180). It seems fair to conclude that PCORI has (understandably) wished to avoid a repeat of the experience of the Agency for Health Care Policy and Research, which famously ran into opposition from back surgeons when it questioned the effectiveness of spinal-fusion surgery (Gray, Gusmano, and Collins 2003; Sorenson, Gusmano, and Oliver 2014). Studies of the development of bureaucratic capacities at agencies like the Food and Drug Administration suggest that building an organizational reputation for relevance and vigilance takes time (Carpenter 2010). It requires strategic leaders who can navigate a challenging political environment, finding ways to avoid generating a backlash among powerful groups while simultaneously fashioning new ideas and solutions that promote the public's diffuse interest in effective governance. As bipartisan support for PCORI's mission is consolidated and it develops a reputation among experts for being an honest information broker, it has the potential to broaden its coalitional base to include not only researchers and patient groups but also physicians, payers, and policy makers. Without the incorporation of such critical stakeholders, PCORI will lack the political support it will need to commission studies that take a hard look at the benefits of popular but unproven treatments. In sum, PCORI's strategy of steering clear of political controversy probably was essential to its survival over the past decade, but at some point it will need to produce evidence on the big questions of waste and inefficiency in US health care if it is to live up to the aspirations of its architects. As Gail Wilensky—fomer director of the Medicare program under George H. W. Bush and one of the most influential advocates for comparative effectiveness research—stated, “I've been a little concerned by what I thought was the excessively cautious strategy that PCORI used in its early years, which is not hard to understand because of its almost near-death experience and the political pushback they were facing. I think it's time to be over it, especially now that they have gotten reauthorized” (quoted in Castellucci 2020).
To be sure, PCORI won't be able to ensure the uptake of medical evidence alone. The experience of the past decade suggests that cost-control reforms like ACOs and bundled payments are too weak to give actors a financial incentive to increase their use of evidence. More robust—and more controversial—shifts in public policy will be needed to make evidence carry significant weight when doctors decide how to treat patients, when payers decide what to cover or reimburse, and when firms decide what medical innovations to invest in. The prevailing political winds are, if anything, blowing in the opposite direction. The FDA is relying on less-rigorous data than in the past to approve drugs (Darrow, Avorn, and Kesselheim 2020). And despite growing awareness of the risks of certain medical devices, the FDA rarely demands “high-quality evidence to support their approval” (Redberg 2018).
To make evidence count in US medicine, changes in Medicare policies may be needed, given that what Medicare does often shapes the behavior of commercial insurers. Thus far, Congress has been unwilling to grant Medicare the authority to deny coverage to therapies that are expensive but deliver marginal health benefits. One potential compromise would be to allow Medicare to incorporate considerations of value into policy decisions while preserving patients' access to services by experimenting “with reference pricing—a practice under which insurers pay a single reference price for all treatments with similar therapeutic effect, allowing patients who want less cost-effective treatments to pay any difference out of pocket” (Bagley, Chandra, Frakt 2015: 6). Using this approach to deter less cost-effective care would likely generate opposition from industry and providers, but it might be more palatable than alternatives that limit consumer choice. If it were implemented widely across different medical therapies, it would strengthen incentives for the medical products industry to prioritize the development of high-value innovations.
Ultimately, transitioning to an evidence-based health care system will require changes not only in the regulatory and policy environment but also in the norms and behavior of the medical profession. The reaction of interventional cardiologists to the recent ORBITA study is illustrative of the need for change (Al-Lemee et al. 2018). Each year, millions of heart patients with clogged arteries receive an angioplasty in which stents are inserted to reduce chest pain. The procedure is expensive and carries risks, yet it became the standard of care on the basis of little hard data. The ORBITA study found that patients who received a stent experienced no improvement in chest pain or exercise tolerance compared to patients who received a placebo procedure. Interventional cardiologists saw the study as an “attack” on their specialty and “lashed out at the trial and the investigators with angry and inaccurate retorts” (Francis 2018). The self-protective reaction of cardiologists to ORBITA is similar to the behavior of other specialty groups when placebo-controlled trials have challenged treatments in their respective practice areas (Patashnik, Gerber, and Dowling 2017).
For medical evidence to count more, physicians and medical societies will need to embrace a more robust understanding of their professional obligations to “uphold scientific standards, to promote research, and to create new knowledge and ensure its appropriate use” (ABIM 2005). In practice, this means using their expertise and authority to identify gaps in the evidence base, to lobby PCORI and other funding agencies to prioritize grants to fill those gaps, and to embrace strong findings (whether they confirm or disconfirm clinical practices). Too often, physicians and medical societies fail to live up to these norms, and physicians and societies who do exhibit leadership on EBM initiatives are often criticized rather than praised by their peers. There are some signs that the culture of the medical profession is beginning to shift, but many clinicians and societies continue to adopt a defensive posture. On the hopeful side, research shows that if physicians were to accept leadership responsibility for promoting high-value, evidence-based care, the public's trust in physicians would allow the medical profession to have a large influence on the policy debate and the public's fears of rationing would be calmed if not eliminated (Patashnik, Gerber, and Dowling 2017).
The political and ideational linkages between CER and the technocratic health care cost-control agenda of 2008–10 clearly helped prepare the ground for the creation of PCORI. These linkages have weakened over the past decade due to increases in hospital market consolidation, changes in the political environment, and accumulating research on the causes of spending variations in the commercial insurance sector. However, the increased attention among policymakers and researchers to provider prices (rather than utilization) as a key driver of US health care spending could help reframe the EBM project. The less EBM is associated in political discourse with cost control, the better may be its future political prospects because it will be seen less as a pretext for rationing driven by costs and more as an effort to improve quality and uphold medical professionalism. The best argument for EBM has never been that it will save money. The most compelling argument is that it is essential to improving patients' health and well-being. While solving the medical evidence problem will remain a political challenge, the professional obligation of doctors to promote patient welfare is an underutilized resource that should make one skeptical about claims about the impossibility of strengthening the role of EBM in US health care over the long haul.
I wish to thank Jon Oberlander for very helpful comments and suggestions, as well as Alan Gerber and Conor Dowling for their collaboration on the preface to the forthcoming paperback edition of our book Unhealthy Politics: The Battle over Evidence-Based Medicine, which stimulated this essay.
While Medicare prices and private prices are only weakly correlated, Cooper et al. (2019) found a strong correlation (0.6) for quantities of services delivered per beneficiary in a given geographical area. In other words, regional utilization patterns for Medicare patients are associated with regional utilization patterns in the under-65 population (Skinner, Goodman, and Fisher 2015).
Many argue that the major reason why the US spends far more than other wealthy democracies on health care is high prices (Anderson et al. 2003). It is important to recognize, however, that “decomposing [cross-national] differences in health care spending into price and quantity is more difficult than it might seem, and there are important challenges in drawing policy inferences from such analyses” (Baicker and Chandra 2018: 986). As Zack Cooper (2019) notes, “At a high level, there is evidence that the US has more high-technology care than is present in other nations. For example, the vast majority of robotic surgery machines are sold in the US. Indeed, even for similar procedures, such as hip replacements, surgeons in the US might be more likely to carry out these surgeries using costlier negative-pressure body exhaust suits than surgeons in other countries. This would make US procedures appear to have higher prices, but those prices would actually reflect differences in the way care is delivered in the US.”
These modifications include an increase in mandatory appropriations to offset elimination of the transfers from the Medicare Trust Funds into the PCORI Trust Fund; the option to add two more seats, for payers/purchasers, to the PCORI Board of Governors; and new authorization for PCORI to consider the “full range of outcomes data,” including “potential burdens and economic impacts of the utilization of medical treatments, items, and services [including medical out-of-pocket costs] on different stakeholders and decision-makers respectively.” Some observers believe the latter change may allow PCORI to give more attention to costs in its research, which in turn may increase the influence of PCORI's findings on the decisions of Medicare and other payers, but whether this will happen remains to be seen. See reauthorizepcori.org/app/uploads/2019/12/PCORI-Approps-Bill.pdf.