Analytic Narratives in Political Economy

Abstract

Analytic narratives combine rational choice theory—often extensive-form, game-theoretic models—and qualitative evidence to understand historical and institutional questions in political economy. This article begins by characterizing the features of analytic narratives that distinguish this method from others in economics and political science. Analytic narratives gained prominence in the 1990s as the result of several factors, including the development of game theory, a turn away from general equilibrium theorizing, and the increasing interest in institutions and economic history. Based on the initial responses and subsequent methodological debates, four criticisms of analytic narratives are discussed: those based on claims to originality, the value of rational choice theory, case studies and external validity, and the merits of qualitative evidence. As the article argues, differing disciplinary approaches and perspectives generated many of these criticisms. Nevertheless, from a methodological perspective, analytic narratives remain an effective and distinctive method for analyzing the political economy of institutions.

1. Introduction

In the fall of 1993, a group of scholars gathered at the Center for Advanced Study in the Behavioral Sciences at Stanford University in Palo Alto, California, to discuss the promising intersection between economics, economic history, and political science. With funding from the National Science Foundation, a team of scholars—Robert H. Bates, Avner Greif, Margaret Levi, Jean-Laurent Rosenthal, and Barry R. Weingast—met to try to understand and articulate what seemed to be a newly emerging area of social science at the intersection of game theory, economic history, and political science (Munck and Snyder 2007: 525). Analytic Narratives, published by Princeton University Press in 1998, was the product of these meetings. The book offered a statement of a new, distinctive method as well as illustrations of its usefulness in a series of chapters written by individual authors who each applied the method in practice. Much of these scholars' past work had been in the style of analytic narratives, so it made sense that this volume constituted a focal methodological statement. It was intended, writes the political scientist Daniel Carpenter (2000: 654), as “the most thorough and methodologically self-conscious wedding of formal rational choice theory to historical narrative that [had] yet been attempted.”

In this article, we discuss the intellectual context in which this method emerged and describe its reception in economics and political science. While the use of either formal models or statistical analysis alone was accepted in both disciplines in the 1990s, the combination of formal models and the rigorous analysis of qualitative evidence was rare, especially in the leading journals. Analytic narratives typically use game-theoretic models to understand the relationship between key actors by focusing on their information, incentives, sequence of choices, and expected payoffs. These models are combined with rich, narrative evidence to explain particular cases, often focused on institutions. Narratives can draw on qualitative evidence that includes a wide range of materials, including written records (such as constitutions, laws, statutes, company manuals, archival materials, private correspondence, and diaries), interviews, participant observation, and ethnography. In doing so, advocates of analytic narratives argue that the approach offers context-dependent explanations and is inspired by real-life puzzles. According to Bates, “The secret agenda in Analytic Narratives is to re-justify the case study” (Munck and Snyder 2007: 525). As with all case studies, external validity is one of the great weaknesses (Gerring 2017: 244–45), but scholars debate how much, if at all, this should discount the value of analytic narratives. Social scientists who use analytic narratives argue that they provide unique insights into the study of institutions that neither formal models nor statistical analysis alone is capable of generating (see, e.g., Greif 2006: 350–76).¹

Before discussing how practitioners and skeptics assessed analytic narratives as a method, we first consider why this method arose when it did and who was involved, and we describe its initial reception. Analytic narratives occupy a distinct methodological space in political economy, and the method—and the need for it—arose because of the confluence of several major trends in the history of political economy. First, the decline of general equilibrium theorizing made way for a greater focus on empirical work. Second, the ascendance and maturation of game theory provided new analytical tools to model institutions. Third, the subfield of economic history experienced a two-pronged revival—cliometrics on the one hand and historical institutional analysis on the other—reflecting a growing interest in historical and institutional questions. The fall of the Soviet Union (among other major transitions in comparative economics) created an even greater demand for political scientists and economists to understand the relationship between institutions and economic performance (Sanders 2006: 40–41). However, while cliometrics provided important insights for some historical questions, it was relatively less well suited to understanding whole sets of questions related to institutions. Thus it was here, with the convergence of game theory, economic history, and a pressing interest in institutions, that analytic narratives emerged.

In economics, however, outside of particular subfields, economists never fully embraced the collection and careful analysis of qualitative evidence. By the late 1990s when Analytic Narratives was published, the “credibility revolution” in applied economics was opening up new avenues for statistical empirical investigation, which were embraced by economists and political scientists. At the same time, however, many political scientists, especially outside of some of the leading departments and journals, were becoming increasingly skeptical of formal models and rational choice theory. In 2000, an anonymous email from “Mr. Perestroika” was sent to numerous members of the American Political Science Association calling for a revolution against positivist approaches in political science, which sparked a substantial backlash (Monroe 2005). While analytic narratives have not become the dominant method in either discipline, we argue that they remain an often uniquely well-suited method within particular subfields of economics and political science, as well as a bridging method for interdisciplinary research.

2. What Is an Analytic Narrative?

The term analytical narrative goes back at least to Milton Friedman and Anna Schwartz's use in their classic 1963 book, A Monetary History of the United States, 1867–1960. In the preface, they explain that a colleague suggested that they include a historical chapter as a useful complement to their statistical analysis. As they explain, “The chapter, which we began to write only after we had completed a first draft of the remaining chapters, took on a life of its own. The one chapter became two, then a separate part, and has now become a separate book” (Friedman and Schwartz 1963: xxi). What began as a “foray into analytical narratives” to supplement statistical work became important in its own right and then began to “significantly affect” their statistical analysis. This description captures well (1) how the richness of qualitative evidence can identify new insights that are often overlooked by thin quantitative data, (2) the benefits of intensive knowledge of a particular case, and (3) the inductive nature of the interaction between models and narratives. Given its status as one of the most celebrated books in its field, it also shows that analytic narratives can be rigorous and influential within the discipline of economics.

Analytic narratives combine qualitative evidence with a rational choice model.² The model is often a formal, mathematical one, but many people working in this tradition also theorize in ways that could be formalized but have not (Levi 2004: 202; Levi and Weingast 2022: 239, 242). What is important is that the model is relatively parsimonious and that it generates comparative static predictions (Levi 2004: 202).³ Some of the most common models used in analytic narratives are extensive-form, game-theoretic models. These models describe key actors, their preferences, and how their interactions generate different possible payouts (204, 208). Such models describe strategic interactions and yield predictions of subgame Nash equilibrium outcomes and out-of-equilibrium outcomes. Most are based on rational expected utility maximization, but others relax this assumption to capture a bounded rationality conception of individual agency (Alexandrova 2009: 3). In their introduction, Bates et al. (1998: 3) explicitly note that other types of theories or models could be used in analytic narratives:

As with the use of models in general, these games are intended to simplify social or economic interactions, so some assumptions will be descriptively inaccurate. Today, many economists argue that while not all assumptions must be accurate, critical assumptions of the model should be. Critical assumptions are those whose “modification in an arguably more realistic direction would produce a substantive difference in the conclusion produced by the model” (Rodrik 2015: 27). Political scientists likewise argue that key assumptions are problematic when they are inaccurate and substantially change the model's predictions (Lorentzen, Fravel, and Paine 2017: 475). Consistent with this position, Bates et al. (1998: 14–15) explicitly argued that assumptions must “fit the facts.” While this presumably does not apply to all of a model's assumption (e.g., perfect information or expected utility maximization) (Alexandrova 2009: 9), they wanted to reject Milton Friedman's (1953: 40–41) idea that a model's assumptions need only be “as-if” accurate. The authors argue that model construction in analytic narratives tends to engage in a more self-consciously inductive process. As qualitative and historical evidence emerges in the investigation of a case study, this learning can inform and update modeling.

Analytic narratives are case studies, which Gerring (2017: 28) defines as “an intensive study of a single case or a small number of cases, which draws on observational data and promises to shed light on a larger population of cases.”⁵ Economists sometimes do not think of themselves as doing case studies, but many of the most famous empirical papers in economics are case studies (e.g., Card 1990; Card and Krueger 1994). These studies typically have a large number of observations (large-n) but focus on a single case (c = 1), so like case studies generally, claims about external validity tend to be relatively weak. Political scientists have argued that one's ability to make inferences from cases depends on how and why a particular case was selected (King, Keohane, and Verba 1994: chap. 4). By contrast, Bates et al. argued that one could draw broader inferences, not just from the reasons for selecting a particular case but also from the model used to understand a case. As Bates explains, analytic narratives could yield insights beyond the particular case “by extracting insights from the case that travel and can be tested systematically” (Munck and Snyder 2007: 526). That is, if the assumptions of the model fit in a different case, then the insights from the original case might generalize. Here, it is the model, rather than the empirical result, that helps provide the generalization.

By breaking down the situation into its elemental pieces, models produce the analytical part of analytic narratives. This is where practitioners turn to “narrative” evidence. Narrative evidence takes many forms, including qualitative evidence such as written records (constitutions, laws, statutes, company manuals, archival materials, private correspondence, diaries, etc.), interviews, participant observation, and ethnography. The narrative evidence is used to provide a “detailed and textured account of context and process, with concern for both sequence and temporality” (Levi 2004: 208). Narrative is often not merely qualitative evidence but also evidence of sequence and chronology.⁶ In being so, narrative provides the “necessary information for causal assessment” (209).⁷ The effectiveness of an analytic narrative is assessed in several ways (Bates et al. 1998: 14–18). First, the logic of the model itself provides a check on the internal validity of the proposed relationships. Second, the narrative evidence can provide the opportunity to assess the accuracy of the model's critical assumptions. Third, the evidence can identify whether the comparative static predictions of the model are observed. Likewise, it can reveal whether out-of-equilibrium outcomes are, in fact, observed. We can similarly judge an analytic narrative to be more valuable if it offers more explanatory power than alternative explanations. Taken together, the narrative, if done correctly and successfully, can provide ample evidence that the model applies and does so in the way that the intuition of the model suggests.⁸

Scholars argue that analytic narratives are particularly useful for certain types of questions in political economy. In particular, quantitative evidence is often unavailable for many important historical questions of interest. Where data are available, natural experiments are frequently absent. For “big” questions aimed at understanding institutions, institutional change, and strategic interactions between groups over time, causal inference methods are often inappropriate. Finally, Bates et al. (1998) show that narrative evidence is sometimes distinctly informative in the study of institutions.

To see how an analytic narrative might be applied in practice, we briefly describe a classic article that uses this method that was published in the Journal of Law, Economics, & Organization. In “The Economic Role of Political Institutions: Market-Preserving Federalism and Economic Development,” Weingast (1995) presents the reader with perhaps the fundamental political dilemma of an economic system: governments strong enough to protect property rights and enforce contracts are also strong enough to confiscate the wealth of its population. This dilemma, Weingast argues, requires that we examine the structure of institutions to understand the conditions under which rules generate credible commitments that forestall predatory behaviors that undermine markets and economic growth. Federalism accomplishes this by (1) solving a coordination problem among citizens about what is legitimate for the state to do and (2) creating a credible enforcement mechanism when the state acts illegitimately (Weingast 1995: 10). Using a game-theoretic model, Weingast generates comparative static predictions about the interactive behavior between a sovereign and two citizen groups with different views about the legitimate boundaries of the sovereign. The aim is to use the model to isolate the mechanism and to show how it operates in three sweeping historical contexts: England during the eighteenth century, the United States during the nineteenth century, and China in the late twentieth century. In sum, he presents a parsimonious model to understand how self-enforcing constitutions in a federalist structure might operate and provides extensive historical evidence that they operated in this way.

More generally, analytic narratives have been used to examine a broad set of questions in political economy, not only long-run institutional development and economic growth (Weingast 1995; North and Thomas 1973; North 1978; Greif 2006). In fact, analytic narratives have also been applied to understand institutional variation in more microlevel situations. For example, Elinor Ostrom (1990) relied on extensive, multisite fieldwork (instead of historical qualitative data) to understand how communities successfully govern common-pool environmental resources. In terms of subject areas, analytic narratives have also been used to understand a wide range of topics such as the political challenges of credible commitments in the post-Soviet era of institutional reform (Boettke 1993; Nalepa 2010); the internal organization and constitutional structure of eighteenth-century piracy (Leeson 2007a); precolonial-era trade in the absence of strong state institutions (Leeson 2007b); providing assurance in a stateless context of long-distance, high-value financial transactions (Schaeffer 2008); extralegal governance institutions in the illicit economy in prisons and on the street (Skarbek 2011, 2014, 2020a); the rise of tax farming in early modern England and France (Johnson and Koyama 2014); the evolution of usury laws (Koyama 2010); and the challenge of overcoming the Samaritan's Dilemma in the context of delivering development aid (Gibson et al. 2005; Skarbek 2016). In short, while many additional examples could be provided, these give some sense of the tremendous scope and range of institutional questions pursued with the analytic narrative method.

3. The Historical and Intellectual Context

3.1. From General Equilibrium to Game Theory

Economics in the mid-twentieth century was dominated by the rise of formalism and the ascendance of the general equilibrium paradigm. Paul Samuelson's Foundations of Economic Analysis (1947) revolutionized graduate training in economics and became a galvanizing force by which formal mathematics became the primary language of economics. The aim of the book was to use the abstraction and tractability afforded by mathematics to elaborate the unifying elements of dispersed theories and to work out the implications for theoretical and applied economics. In other words, the explicit aim was to utilize the language of mathematics to abstract away from the institutional specifics and contextual details in which economic behavior takes place. Samuelson was clearly aware of this aspect of his approach. In describing his work in 1967, Samuelson states that “the equations of my model specify the conditions that must be satisfied by Pareto optimality and maximization of a prescribed individualistic Bergson social welfare function. They do not pronounce on what set of institutional rules and behaviors will approximate these conditions.”⁹ The result was that “Samuelson drained economic theory of institutional context,” and the subsequent rise of the “econometric approach to empirical economics eliminated historical detail” (Boettke 1997: 22).

At the time, Samuelson's approach was widely regarded—despite a few dissenters—to be a pathbreaking way forward in economic theory. Among the dissenters, the economist Kenneth E. Boulding reviewed the book in the Journal of Political Economy and offered a warning to the profession of what may be lost with a complete exorcising of narrative from economic analysis. “It may well be,” writes Boulding (1948: 199), “that mathematical economics will remain too flawless in its perfection to be very fruitful” and that “if economics becomes a preserve of the higher mathematicians, it will lose its essentially humanistic and empirical quality.”

Perhaps nowhere is the “institutional vacuum” more evident than in the concept of competition as expressed in the Arrow and Debreu (1954) fixed-point existence proof of general equilibrium. The central insight of the proof is to show that with optimizing agents, there exists a vector of prices that clear all markets. However, the formalization of the model explicitly abstracts away from the activity or mechanisms that bring about the result. According to Blaug (2003: 146), the Arrow-Debreu proof “neatly exhibits the worst features of formalism, which is not just the application of mathematical techniques to economics, but rather reveling in mathematical modeling as an end in itself.” The equilibrium theorizing that followed in the wake of Arrow-Debreu made “no effort to show how equilibrium comes about but merely that the existence of equilibrium is logically implied by certain plausible institution-free assumptions about economic behavior” (147).

The dominance of the formalist revolution reversed how basic models of economics were used and interpreted. Before the 1950s, not only was economics pluralistic in approaches, but even formal models like the perfectly competitive market were used as stylized depictions to contrast with and illustrate operational features of real market activity.¹⁰ Used in this manner, counterfactual reasoning shed light on how institutions provide much of the rules and incentive structure markets require to function properly. Formalism allowed for the technical specification of optimality conditions, such that deviations from optima were now ipso facto market failures capable of being understood independent of context. If narrative is the bridge between model and meaning, it had all but disappeared.¹¹ The formalist revolution effectively buried the process-based accounts of equilibrium formation (Blaug 2003).

Some areas of economics identified the loss of institutional detail as a cost of formalism earlier than others. F. A. Hayek's (1942) early argument regarding the misapplication of formalism to the study of political economy actually predates Boulding's alarm and remains relevant when advanced again more than thirty years later in his 1974 Nobel Prize address (Hayek 1989). But it was not just the Austrians who were attuned to the limits of formalism. In a memorandum discussing the need for a distinct field of development economics at Harvard, John Kenneth Galbraith wrote in the mid-1950s that it is “unlikely that with [developing] economies one can do much with formal economic models,” precisely because they lack the ability to factor in the role of institutions (Alacevich 2017: 269).¹² Galbraith's call for bringing institutions into the study of development, however, harkened back to the traditional approaches of the older institutional economists and had little to do with the coming emergence of new institutional analysis. To understand where analytic narratives find traction, we must turn to the rise of game theory in economics and political science.

3.2. The Confluence of Game Theory, Economic History, and Institutions

While formalism was changing the landscape of mainstream economics in the 1950s and 1960s, game theory was developing as an alternative theoretical paradigm to the general equilibrium approach (Giocoli 2003). John von Neumann and Oskar Morgenstern's 1944 pathbreaking Theory of Games and Economic Behavior opened up an entirely new approach to conceptualizing strategic interaction and equilibrium formation. Jacob Marschak, for instance, immediately recognized the potential of what these new tools had to offer economics, leading him to declare that with arrival of game theory, “all is not well with static economics” (Marschak 1946: 97). Marschak was not wrong. Game theory would grow in prominence within the mainstream of economics but also play a central role in the development and advancement of new fields in economics and political science—namely, new institutional economics, economic history, and political economy.

As game theory was taking off, history was undergoing its own revolution in the 1960s. The “new economic history,” as it came to be known, brought two developments to the field. Douglass C. North was central to both of these advances. The first was cliometrics: the application of economic theory and quantitative methods to the study of history. Here the formalism that transformed mainstream economics was making its way into history. Cliometrics “formalized economic history in a manner similar to the injection of mathematical models and statistics into the rest of economics” (Goldin 1995: 193). Opposition to formalism in economic history was not confined to a few dissenting voices. There was considerable resistance to cliometrics mainly because outsiders (economists) were the ones introducing more precise estimation techniques into the field of history, and in doing so were often overturning orthodox findings and accepted wisdom (194).

The second development in economic history involved a renewed focus on the importance of institutions for understanding economic growth. In 1966, North accepted a Ford Fellowship to go to Europe to study European economic history. It was here where North realized that the “straitjacket of static, orthodox neoclassical theory was holding back progress in the [new economic history]” (Snowdon 2016: 112). Progress understanding historical growth trajectories required incorporating a conceptual understanding and an empirical analysis of property rights, institutions, and transactions costs. Yet the tools of cliometrics did not allow for adequate treatment of these thick concepts. As North (1978: 963) explains,

As a result, North became heavily involved with the revival of institutional analysis in the 1970s and 1980s.

North's turn away from cliometrics toward studying institutions is significant because it required developing a new framework for conducting the kind of work he wanted to do (Margo 1999: 195). This turn entailed an embrace of a broader notion of rational choice analysis and applications of those tools to try to explain the multidimensional institutions of formal and informal rules to show their relationship to economic growth. However, these new tools and techniques of analysis were not constructed ex ante then applied but rather were forged in the process of research, notably in two books. The first was written with Lance Davis in 1971 and titled Institutional Change and American Economic Growth, and the other was written with Robert Thomas in 1973 and titled The Rise of the Western World: A New Economic History. In the former book, the authors are explicit that they are developing a new theory of institutional change that is consistent with the basics of neoclassical economics but aims to explain the emergence and decline of institutional arrangements. The basic model Davis and North employ relies on cost-benefit analysis and maximization, but the object of the analysis is history. In developing their theory of political bargaining and institutional change, Davis and North cast their theory as potentially capable of providing concrete “solutions” to the problem of indeterminacy of equilibrium positions inherent in “n-person” or “non-zero sum” games (Davis and North 1971: 36). This points to a clear understanding of the analytical space in which the combination of rational choice and qualitative methods—primordial analytic narratives, if you will—can productively advance research.

One of the ways to trace the influence and significance of North's institutional and methodological turn to analytic narratives is to look at the scholars with whom he worked. From 1960 to 1983, North was professor of economics at the University of Washington. Here, he overlapped with Margaret Levi, who was in the political science department from 1974 to 2014 and with whom he taught an undergraduate seminar for roughly ten years (Levi and Weingast 2019: 213). Levi and Weingast (2019) attribute North's Structure and Change in Economic History (1981) and Levi's Of Rule and Revenue (1988) to be among the results of this seminar.

When North joined the faculty of Washington University in Saint Louis in 1983, his research agenda on institutions and economic growth was in full swing. Yet North was dissatisfied with his own understanding and analyses of the political process and sought out a group of young political scientists and economists who were attempting to develop new models of political economy, including James E. Alt, Jean Ensminger, and Jack Knight, as well as Norman Schofield, Kenneth A. Shepsle, and Barry R. Weingast (Levi and Weingast 2019: 213). Weingast's first academic appointment was in the economics department in 1977. Over the next ten years at Washington University, he would be among the many scholars collaborating with North and building on his work. Among his colleagues at Washington University, North founded the Center for Political Economy in 1984 where he stayed on as director until 1990. Over this time, the center fostered interdisciplinary scholarship and collaboration that allowed the research agenda on institutions and the analytic narrative methodology to emerge in economics and political science.

By the publication of North's “Institutions” paper in the Journal of Economic Perspectives (North 1991), North's centrality to the study and method of analyzing institutions was well established. The paper is widely seen as a summation of his work on institutional change and economic development. With Ronald Coase, North was also one of the founding members of the International Society for New Institutional Economics in 1997 that would become home for a substantial amount of work done in the analytic narrative style. Margret Levi and Barry Weingast attended the inaugural conference, along with a number of notable scholars working in these areas.¹³ In sum, institutions had reemerged as a significant research agenda, much of it based on game theory, and it was being carried out by economists and political scientists to understand history. In particular, North had played a major role in the intellectual advancement of the study of institutions, history, and rational choice theory broadly understood. Perhaps it is not surprising that Analytic Narratives is dedicated “to Douglass North: Mentor and Friend.”

4. The Reception of Analytic Narratives

4.1. Exchange in the American Political Science Review

In 1998, when Analytic Narratives was published, it attracted a great deal of attention among economists and political scientists. In particular, the American Political Science Review published an eleven-page review by the political scientist Jon Elster, titled “Rational Choice History: A Case of Excessive Ambition” (Elster 2000). As the title suggests, it is a fairly negative review, and Bates et al. (2000: 701) describe it as a “scorched earth” approach in their response. While the seven-page response from Bates et al. is not nearly as aggressive as Elster's review, the exchange is certainly heated, as six of the most famous political scientists and economists clash over fundamental methodological questions in the leading political science journal.

Elster makes four general criticisms about the analytic narratives project.¹⁴ First, he argues that the rationality assumption is problematic because people sometimes err, respond to nonmaterial concerns, and act irrational in a variety of ways. Bates et al. (2000) respond that Elster has not provided a superior alternative framework for understanding decisions and institutions, so even if rational choice is imperfect, it is still the best available. Until someone presents a more productive approach, the perfect should not be the enemy of the good. Second, Elster argues that the authors rely too much on modeling aggregate actors—such as clans, the elite, and the South—as unified actors with a coherent preference function. To do so, one would need to show that aggregates can have coherent preference functions (i.e., no cycling) and that, in practice, people have overcome a large number of collective action problems within groups. In response, the authors argue that it is an empirical, not a theoretical, question how much one can aggregate actors. Without further evidence, they argue, Elster's point as a general criticism of analytic narratives falls flat. Third, Elster (2000: 693) argues that the authors do too little to provide evidence about the intentions and beliefs of these actors. Their response is that observing intentions and beliefs is an incredibly difficult empirical task, and the standard approach of instead focusing on revealed preferences and behavior is more feasible and productive. Finally, Elster argues that the models do not address issues about imperfect information and uncertainty. Their response is that they do, in fact, incorporate these issues more than Elster realizes.

Speaking about the exchange several years later, the political scientist Adam Przeworski explained that he thought Elster “was critical for the wrong reasons. . . . He has a laundry list approach. I am persuaded that he was right on many historical points; the contributions of Analytic Narratives did not get their history very right. But I do not think Elster grappled with their intent” (Munck and Snyder 2007: 499). That is, while Elster might have made points of historical disagreement in criticizing the individual chapters, doing so does not undermine the broader methodological project. Likewise, Bates says, “I didn't place much weight on Jon Elster's review in the American Political Science Review (2000) because he is noted for negative reviews and he mainly reasserted arguments that he had been making in other forums” (Munck and Snyder 2007: 526).

This exchange in the American Political Science Review was likely the most high-profile discussion that the book received.¹⁵ However, it was also reviewed numerous times in journals in economics, political science, sociology, and related fields. We identified four criticisms that reviewers often raised that reveal that economists and political scientists assessed the book in quite different ways. In particular, reviewers questioned the method's originality, the value of rational choice theory, the relationship between case studies and external validity, and the use of qualitative evidence in studying institutions.

4.2. Claim to Originality

One common criticism of Analytic Narratives is that the methodological argument was not as original as the authors suggest. For example, the sociologist and political scientist Theda Skocpol (2000: 669–70) opens her review of Analytic Narratives by writing, “Five Smart People in Search of a Mission would be an ideal title for this provocative collection,” arguing that “on the methodological grounds, there is actually little here that is new.” The economist Gordon Tullock's (2000: 106) review says that combining history and formal theory is “not revolutionary, but still worth doing.” The political scientist Adam Przeworski explains, “I see Analytic Narratives as less path-breaking than its authors do” (Munck and Snyder 2007: 499). Ekelund and Tollison (2003: 491) argue that analytic narratives as a new method “is either illusory or simply old wine in new bottles. . . . These essays, while offering some interesting material, are not ‘as advertised.’ They do not advance a new methodology.” They end their review by, again, challenging the margin of originality, asking, “Could it be that political scientists are at last adopting the methods of economics and are trying to claim them as their own?”

However, there is reason to question the strength of these conclusions. First, many of the contemporary reviews praised the book for its originality. The sociologist Marc Schneiberg (2000: 857) writes, “Grounded in game theory, their approach represents a distinctive contribution to the analysis of institutions” that bears “impressive fruit.” In the view of the economist Robert A. Margo (1999: 195), Analytic Narratives is “pioneering.” The economist Peter Boettke (2000: 379) writes, “With this book, scholars in [Constitutional Political Economy] have been given a handbook for applied political economy and the empirical examination of the reason of the rules.” And according to the political scientist Andrew Bennett (2001: 978), “The use of case studies to test and modify formal models is in its infancy and should not be abandoned prematurely. This methodological innovation is the book's main contribution, and it deserves to be emulated.”

A second reason to doubt the originality criticism is to look at what methods leading journals were actually publishing. In political science, few to no papers in top journals were published that contained analytic narratives. Pion-Berlin and Cleary (2005) look at all papers published in the American Political Science Review (APSR)—again, the premier political science journal—between 1991 and 2000 and classified each article by method: statistical, mathematical or formal modeling, political theory, and qualitative.

During that time, 74 percent of the articles published used statistics or mathematical modeling. The APSR published only five articles—fewer than 1 percent—based primarily on qualitative evidence over the span of ten years (Pion-Berlin and Cleary 2005: 307). In other words, the leading journal published qualitative evidence extremely rarely. Even if all five of these articles were analytic narratives, that would still be a substantial methodological outlier.

Refining their classification scheme to better identify mixed-method approaches, Pion-Berlin and Cleary (2005: 307) find that, of those articles based on “mathematical modeling,” only five of those papers included some qualitative evidence, though often only a few paragraphs rather than a genuine, intensive case study (309). In 2010, Kasza (2010) used the same classification scheme for the thirty-nine articles published in the APSR and the American Journal of Political Science (AJPS) between May 2009 and April 2010. Eighty percent of the articles published by the APSR, and 94 percent in the AJPS, used quantitative analysis or formal modeling. There were no qualitative empirical papers published in either journal during that time. There is little to no evidence to suggest that analytic narratives are or were a mainstream or common approach in the leading political science journals.

In economics, the return of institutional analysis meant that something like analytic narratives was somewhat more common in economics journals. In their review, Ekelund and Tollison (2003: 493) report that “‘narratives’ using this ‘method’ appear regularly in economic journals (the Journal of Law and Economics, Journal of Economic Behavior and Organization, Public Choice, and many others come to mind).” Notably, these are journals that have tended to publish work in the rational choice institutionalist tradition (Hall and Taylor 1996). The Journal of Law and Economics was, of course, also edited by Ronald Coase for nearly twenty years, and Public Choice has always had a strong focus on institutions. Given that qualitative evidence is often especially useful for the study of institutions, this is perhaps not surprising. Nevertheless, in the leading economics journals, it is extremely rare to see published empirical work based primarily or entirely on qualitative evidence.¹⁶ Likewise, graduate programs in economics do not offer courses on qualitative research methods, instead focusing overwhelmingly on econometrics (Colander 2008: 7). Causal inference approaches are now a leading technique in empirical microeconomics (Angrist and Pischke 2010).

In sum, it is true that the combination of narrative evidence and models was not an entirely new idea. However, at the time of publication, the method had little presence in leading journals in economics and political science. Moreover, Analytic Narratives sets out much more clearly than previous examples a methodological statement for the justifications and ways of using the method, specifically with respect to rational choice theory and the political economy of institutions. While analytic narratives might be more common in economics journals, graduate training in economics does not focus on teaching the analytic narrative method. In each of these ways, the book was carving out new intellectual territory.

4.3. The Value of Rational Choice Theory

A second related criticism of analytic narratives was directed at the use of rational choice theory in particular, and to positivism more generally. Analytic Narratives offered a new methodological defense of rational choice theory at a time when it was under fire within political science. At the same time, most economists who reviewed the book did not seem to understand the contentious place of rational choice in political science.

Analytic Narratives was published in between two perestroika movements that worked at cross-purposes: one to increase interest in analytic narratives, the other to attack them. The first perestroika was the reform efforts to unwind central economic planning in the Soviet Union and its subsequent collapse (Boettke 1993). World events presented social scientists with major, often catastrophic, changes to nations that cried out for analysis and explanation, with a special focus on the relationship between institutions and outcomes (Sanders 2006: 40–41). The second, and lesser known, perestroika was a reform effort within the political science discipline. This loose-knit intellectual movement made revolutionary calls for reevaluating the use of rational choice theory and positivist approaches within the profession. It began in 2000, when an anonymous email authored by “Mr. Perestroika” was sent to ten members of the American Political Science Association who were encouraged to forward it to other colleagues. It was, among other things, a broadside attack against rational choice theory and the dominance of positivism in the political science profession. It likewise called out the American Political Science Association and the APSR as irrelevant venues for generating knowledge. The perestroika movement ignited a revolution among intepretivist scholars and others who felt they were being sidelined in the discipline.

Part of the complaint identifies elitism, nepotism, gender bias, and careerism, but many of the complaints are levied directly at positivist methods and the encroachment of economists. For example, Mr. Perestroika refers to “a few men who make poor game-theorists and who cannot for the life-of-me compete with a third grade Economics graduate student” (Monroe 2005: 10). He later asks, “Why are FAILED Africanists and Economists allowed to dominate a discipline?” (10). Finally, “If these pseudo-economists know their math so well, let them present at the University of Chicago's Economics Workshop. I assure you every single political science article will be trashed and thrown into the dustbin. Then why are these people allowed to throw their weight around based on undergrad math and stats—an Econ 101. We are in the business of Political Science and not failed Economics” (10–11). According to the political scientist Kristen Renwick Monroe, the movement spread like “wildfire” (i). As one indication of how influential the movement was, and the many ways that the discipline reacted and changed in response to it, Monroe's edited volume on the perestroika movement relies on thirty-nine chapters and nearly six hundred pages to document its widespread effect.

The debate in political science over rational choice theory had also been heating up in the years prior. In Skocpol's (2000: 670) review, she reports that, “in comparative politics, this is the Era of Manifestos, as clashing camps of scholars maneuver for students, positions, resources, and academic prestige.” As a part of this, in 1994, Donald Green and Ian Shapiro published Pathologies of Rational Choice Theory: A Critique of Applications in Political Science, where they argue against the idea that we can ever develop a single, universal model of politics (Green and Shapiro 1994: 6). Green and Shapiro likewise condemned rational choice approaches for being “theory driven” rather than “problem driven” (6). In her review of Analytic Narratives, Skocpol (2000: 675) argues that Bates et al. (1998) do too little to show that their rational choice accounts outperform competing explanations. She challenges the value of rational choice theory as a “general theory” (Skocpol 2000: 675). She writes, “Today's rational choicers . . . believe in One True and Unified Theory and cling to a model of explanation that stresses that application of general theorems to specific instances, one at a time” (675).

However, this seems to be inconsistent with the framing of the use of rational choice within the book. In the introduction, the authors explain that the political scientist William H. Riker did indeed seek to develop a “universal approach to the social sciences, capable of yielding general laws of political behavior” (Bates et al. 1998: 11). But that is not their goal. Instead, they “are motivated by a desire to account for particular events or outcomes. They are devoted to the explanation of cases, not to the elaboration of theory” (11). Each chapter is driven by the desire to solve real-world problems and puzzles and not driven by theory. The authors explicitly deny the Riker vision of universal theory, writing that “the chapters themselves seek no universal laws of human behavior” (11). The book aims to advance the claim that the rational choice framework more generally can be applied in a wide range of situations, but the particular model of interaction must be context-dependent. They advocate the use of narrative and iterative model design precisely because no “One True and Unified” theorem of institutions exists. Defending rational choice theory might not seem controversial or original to the economists who reviewed the book, but Analytic Narratives was very much return fire in a heated disciplinary debate over methods.

4.4. The Question of External Validity

A third criticism of analytic narratives points out that case studies tend to be weak in terms of external validity.¹⁷ The economists Ekelund and Tollison (2003: 492), for example, argue that many of the case studies in the book are “incapable of generalization.” This is related to concerns that analytic narratives are an exercise in curve fitting (Dessler 2000: 179; Munck and Snyder 2007: 631). As with the diverging views about the proper use of rational choice theory in institutional analysis, here we again see an oversight of the differences between disciplines. In particular, social science methodologists distinguish between studies that are on the “effects-of-causes” and those on the “causes-of-effects” (Mahoney and Goertz 2006: 230–32). The effects-of-causes approach focuses on finding out what the effect of some cause is across a large number of cases. For example, economists might estimate the effect of foreign aid on economic development across a large number of countries (Easterly 2003). From this perspective, case studies in general do a poor job of explanation because they have relatively little to say about other cases. Alternatively, in the “causes-of-effects” approach, the goal is to understand many or all of the causes of an effect in a particular case. This approach is more focused on maximizing within-case explanatory power. As case studies, analytic narratives fall into this latter category.¹⁸

As Gerring (2017: 244–45) notes, cases studies are generally weak on the question of external validity. To dismiss analytic narratives for this reason would require us to dismiss case studies more generally, something that many social scientists would not be willing to do. Nevertheless, it is worth noting the different ways that scholars have thought about analytic narratives and external validity. As noted above, Bates argues that the model itself might be found to generalize to other cases (Munck and Snyder 2007: 526). For example, the tragedy-of-the-commons model has large explanatory power in a wide range of scenarios across time and place (Ostrom 1990). Margaret Levi (2004: 203) argues that analytic narratives can explain particular cases very well and that alone is an important endeavor. Avner Greif's historical work, for example, offers an explanation for the rise of the modern economy, and given the magnitude and importance of understanding that historical process, it is inherently a worthy topic of investigation (Greif 2006: 350–76). Nomothetic approaches are not the ideal or goal. Levi has also argued that analytic narratives are a test of the rational choice framework more broadly. Each successful case study provides evidence on whether rational choice is successful as “a general theory of how structures shape individual choices and consequentially collective outcomes” (Levi 2004: 218). The external validity is thus found in the usefulness of the tools of rational choice social science more generally.

Ultimately, the degree to which we can generalize theory or empirical findings from one case to another is an empirical question. Greif (2006: 388) argues that

No single analytic narrative can lead us to say that similar mechanisms or institutions are generally important or will emerge, but the accumulated findings of a series of analytic narratives can provide insights about broader patterns on institutions, institutional change, and economic, political, and social performance.

4.5. Qualitative Evidence and Institutional Analysis

A final oversight that seems to emerge in the varied responses to Analytic Narratives is about the unique role of qualitative evidence in the study of institutions. When Bates et al. (1998) make causal claims based on analytic narratives, it is not done so on the basis of statistical identification with “thin” quantitative data, but from deep engagement with rich qualitative evidence, while guided by the insights of a model. This approach offers several advantages. Qualitative evidence can sometimes provide a better understanding of social and political processes than quantitative evidence alone (Skarbek 2020b). Moreover, narrative evidence is often effective at identifying causal mechanisms rather than simply causal effects, which is often at the heart of institutional analysis (Poteete, Janssen, and Ostrom 2010: 35). Identifying mechanisms is important because there might be many possible causal pathways that connect two variables (a problem known as “equi-finality”), and identifying mechanisms can often tell us why something happened rather than just what happened (Gerring 2017: 216). Both qualitative evidence and formal models can help identify mechanisms and therefore provide causal explanations (Gailmard 2021). While economists tend to be skeptical that there are rigorous ways to analyze qualitative evidence, there is a large literature on how to deploy qualitative research methods in a rigorous way to make descriptive and causal inferences (e.g., King, Keohane, and Verba 1994; Bennett and Checkel 2015; Seawright 2016; Gerring 2017). Taken together, Bates et al. (1998) is a spirited defense of the usefulness of qualitative evidence in general.

However, the analytic narrative project also demonstrates a slightly different point. Not only can narrative evidence provide evidentiary value, but it is also often especially well-suited to the study of institutions. Consider how different research approaches use different combinations of types of concepts, theories, and evidence. In terms of evidence, we can distinguish between quantitative and qualitative. Likewise, we can distinguish between “thin” and “thick” theories. Thin theories are relatively parsimonious and have clear comparative static predictions. Thick theories, often found in sociology and anthropology, tend to lack parsimony and clear predictions (Boettke 2000: 378). Finally, the concepts used vary in the degree to which they are simple or more multidimensional (Coppedge 1999: 468–71). For example, concepts like “employment” and “wage” are relatively straightforward to measure and count, compared to multidimensional concepts, while still capturing the essence of what we are interested in. By contrast, a concept like “institutions” can vary in many ways. This includes ranging from legal to extralegal, formal to informal, centralized to decentralized, flexible to rigid, permanent to temporary, and fragile to robust to antifragile (Ostrom 2009). Each characteristic is difficult to accurately and precisely measure. As a result, these types of concepts are more difficult (or impossible) to reduce to simple quantitative measures without losing crucial parts of their meaning.

Standard neoclassical economics tends to work with simple concepts, thin theories, and quantitative evidence. This is the domain of constrained optimization and statistical significance. For example, economists have parsimonious models about labor markets and minimum wages. “Employment” and “wage” are both simple concepts, and they are captured well by quantitative evidence. Likewise, time-honored works in anthropology and sociology often rely on multidimensional concepts, thick theories, and qualitative evidence (Geertz 1973). This is the domain of social forces and culture examined in case studies and ethnography. Such scholars often provide a rich description of cultural practices at a particular time and place to understand its meaning. As such, this combination of concepts, theory, and evidence seems to work well too.

But not all combinations of concepts, evidence, and theory work as well. For instance, much contemporary work in criminology combines multifaceted concepts, thick theories, and quantitative evidence (see, e.g., Pyrooz and Decker 2019). Examples of multifaceted concepts in criminology include “culture,” “norms,” and “deviance.” These concepts seem too multidimensional to reduce to simple quantitative measures without losing important parts of their meaning. Many criminology theories likewise tend to be thick with vague or underdetermined comparative static predictions. The criminologist Gerben Bruinsma, in his 2016 presidential address to the European Society of Criminology, argues critically that this has led to “a mixture of hundreds of perspectives, definitions, ideas, sketches, multiple factors, theories and single hypotheses that are partly true and partly untrue, and none are completely true or untrue” (Bruinsma 2016: 659). Sociologists and anthropologists would typically turn to qualitative evidence to describe and assess these types of concepts and theories. However, it is not clear that the quantitative evidence and statistical estimation used by many criminologists are well suited to describing these multidimensional concepts or for testing relatively thick theories that have relatively obscure empirical implications.

Finally, analytic narratives combine multidimensional concepts, thin theories, and qualitative evidence in a distinctive way. As argued, “institution” is a multidimensional concept. As such, qualitative evidence is appropriate and useful for describing and understanding its key characteristics. Alternatively, if “institution” was a simple concept, then traditional quantitative approaches in political science and economics (such as cliometrics) might suffice for testing institutional theories. From this perspective, qualitative evidence is often superior to quantitative evidence because it more fully maps onto and describes institutions. Finally, unlike the thick theories sometimes found in sociology and anthropology, the thin theories used in analytic narratives provide clear comparative static predictions that can then be assessed with relevant evidence. With these considerations in mind, the individual chapters in Bates et al. (1998) demonstrate a distinctive and useful approach to studying institutions.

5. Conclusion

Analytic narratives provide a unique method for institutional analysis. By focusing on Bates et al. 1998, this article examines the origin and reception of combining narrative evidence and rational choice theory. Practitioners argue that qualitative evidence can often better describe multifaceted and multidimensional concepts such as institutions. The parsimony and comparative static predictions of rational choice institutionalism generate empirical implications that can be more fully tested with such rich evidence. The use of this approach emerged during a time when institutions and economic history were returning to a prominent focus of research in economics and political science. Because the topics themselves were of concern to both disciplines, the analytic narrative provided a method to allow cross-disciplinary conversations.

However, Analytic Narratives was published during a period of heated debate in political science about the value of rational choice approaches. It sought to bring warring sides closer together by arguing in favor of narrative evidence and case studies. It created space for more methodological pluralism in the political science profession. Nevertheless, it likewise revealed the difficulty of advancing new interdisciplinary methods. Reviewers from different fields did not always appreciate the novelty and insights of this new method. Also, at the same time that Bates et al. (1998) were offering analytic narratives as a way to understand institutions, another approach was emerging. The “credibility revolution” in applied economics was discovering new ways to identify causal relationships using exogenous shocks and as-if randomization (Angrist and Pischke 2010; Dunning 2012). Political scientists were already quite comfortable with statistical analysis, so adopting these new techniques was low cost and appealing. It did not require that formal theorists learn to do history or that qualitative researchers learn formal modeling.

Nevertheless, there are opportunity costs to the methods that we use, especially when methods drive the research questions we ask. Focusing on causal inference approaches strictly limits the scope of historical and institutional questions we can ask. Even when causal inference approaches are used in the study of institutions, they can tend to “flatten” institutions. Often, for example, research measures the effect of institutions rather than the institutions themselves (Acemoglu, Johnson, and Robinson 2001). While these types of studies clearly generate important insights about institutions, the method simultaneously limits the questions we ask and the ways we learn. A complementary return to rigorous engagement with narrative evidence in analytic narratives offers the opportunity for a richer and more complete understanding of the political economy of institutions.

The authors thank Anna Alexandrova, Peter Andreas, Andrew Bennett, Peter Boettke, Daniel J. D'Amico, Sean Gailmard, Catherine Herfeld, Mark Koyama, John Meadowcroft, John V. Nye, Kirun Kumar Sankaran, Mary Shirley, Richard Snyder, Steven Teles, and three anonymous referees for helpful comments. We are grateful for support from the Templeton Foundation.

Notes

References