Restaging the Black Box: How Metaphor Informs Large Language Models

Abstract

Literary theorists have long examined the role between writing and technology, in part due to technology's reliance on metaphor. Metaphor is central to the ways new technologies are marketed to and understood by users; metaphor also determines the sorts of critiques of those technologies scholars might make. This article looks at the particular relationship between metaphor and LLMs. Specifically, it examines the metaphor of the black box, which is often used to metaphorize their opaque inner workings. Exploring the multiple definitions of the black box metaphor reveals how its use in regard to LLMs reproduces the power imbalances inherent in opaque systems. By considering the many meanings of the black box together, we may see how the term maintains false binaries of transparency and opacity. As one example, this article argues that refiguring the (algorithmic) black box as a black box theater repositions LLMs as a reflexively performative space. Literary critique of this interdisciplinary kind deepens the understanding of LLMs and their ethical implications.

“GPT-4 Is a Giant Black Box and Its Training Data Remains a Mystery” (Barr 2023); “Two-Hundred-Year-Old Math Opens Up AI's Mysterious Black Box” (Choi 2023); “ChatGPT, Black Boxes, and Information Dissemination” (Garcez 2023); “Unpacking the ‘Black Box’ to Build Better AI Models” (Zewe 2023). Current headlines on large language models (LLMs) are full of black boxes. Impenetrable and opaque, black boxes encapsulate the air of mystery around artificial intelligence—what they do, how they do it, and why they work. But the black box is at bottom a metaphor for our relationship to complicated technological processes. Do not look inside of the black box: it is dark, it is closed. Nonetheless, as figurative language, the black box invites literary analysis and close reading. What does the metaphor offer, and what does it foreclose?

A long history of literary scholarship has theorized the meaning of metaphor, as well as the relationship between technology and language more broadly.¹ As one of the human species’ most notable communication technologies, writing has been investigated from Plato to Jacques Derrida and beyond (see also Chloë Kitzinger's thinkpiece in this special issue).² As LLMs now take center stage, the relationship between language and technology is once again a topic for literary critics, as a recent forum in Critical Inquiry, “Again Theory: A Forum on Language, Meaning, and Intent in the Time of Stochastic Parrots” (Kirschenbaum 2023) makes clear.

In their introduction to Critical AI's inaugural issue, “Data Worlds,” Katherine Bode and Lauren M. E. Goodlad (2023) discuss the advent of “stochastic parrots,” a term deriving from the seminal critique of LLMs by Emily M. Bender et al. (2021), as a case for interdisciplinary thinking about critical ontologies. Authored by a team of computational linguists and AI ethicists, the essay makes a strong distinction between an LLM's ability to manipulate the form of language and its inability to understand those language forms in a humanlike way. Moreover, the extractive mode of data collection for training these models creates a problem of “documentation debt” (610) along with a host of harms to individuals, society, and the environment.³ This lack of adequate data documentation is compounded by the fact that the marketers of commercial LLMs such as OpenAI's ChatGPT and Google's Bard do not provide access to their training data or architectures. That is to say, the technology of training supersize models on massive data sets means that LLMs are all but inevitably perceived as black box systems, the commercializers of the most powerful systems compound that problem through deliberate lack of transparency.

In this essay we suggest that unpacking the black box as a metaphor offers a critique of LLMs that could push users of these systems toward an active commitment to data justice. When we peel back the metaphor, it becomes possible to see how the language of black boxes enables developers and marketers to avoid accountability and immobilize critical thinking about proprietary and surveillant systems. By contrast, refiguring the black box metaphor enables LLM users to reflect on their situation within, and potentially against, rapidly consolidating but still dynamic systems of power.

Literary scholars define metaphor as a type of figurative language in which a word or phrase that signifies one object is used in place of another to express likeness between them. Conceptual metaphor theory, as prominently theorized by George Lakoff and Mark Johnson in Metaphors We Live By (2003), elaborates the relationship between metaphor and the human capacity to form concepts. Briefly, the theory explains how metaphors inform human sense perception due to the conceptual system's reliance on “understanding one thing in terms of another” (5). For Lakoff and Johnson, humans do not merely compare two concepts when they use metaphor; rather, they talk about one thing in the terms of another “because [they] conceive of it that way” (5).

The tech industry has frequently capitalized on the metaphoric functioning of the human conceptual system. By seizing an already existing and nontechnical concept, it becomes possible to situate a new technology in terms that users readily understand, thereby increasing the product's selling potential. In addition to increasing profit, this process also enables a metaphorized technology to reproduce any power imbalances inherent in the concept on which the metaphor relies. Metaphors thus function as potential “maps of capitalism and class privilege” (Selfe and Selfe 1994: 486), harboring hierarchies of power and privileging one way of knowing over another. Take the example of the “desktop” computer. Early computers didn't sell in part because they lacked a conceptual anchor to explain their function. As Eric Chown and Fernando Nascimento (2023: 54) suggest, the “clear visual metaphor” of the desktop made using computers more accessible through an “analog in the physical world.” Random and disorganized strings of code became recognizable through the metaphor of a filing system in an office space complete with an iconography of manila folders, documents, and trash bins. The metaphor of the desktop computer became a “metaphor we live by,” changing not only how we perceive—but also how we use—computers.

But what about people who don't work at desks? Understanding computers as desktops limited the conception of their use, their place in society, and the sense of who can use them. In this way the desktop metaphor enfolded certain privileges and forms of access. Cynthia Selfe and Richard Selfe (1994: 485) thus complicated the assumption that the desktop is a “democratic” interface, claiming that interfaces, like maps, necessarily privilege certain ways of knowing, existing, and thinking over others. The desktop interface “order[s] the virtual world according to a certain set of historical and social values” (485)—namely, the largely white and middle-class virtues of white-collar and corporate structures.

We propose that the metaphor of the black box as an opaque system is another example of how metaphorized interfaces import already existing power dynamics and ways of knowing into new technologies. Opaque LLMs like those now commercialized for profit by OpenAI, Google, and Microsoft claim, perpetuate, and sometimes amplify systems of oppression, patriarchy, and capital. Nonetheless, uncritical deployment of the “black box” metaphor is itself problematic in perpetuating harmful binaries including opacity/transparency and inputs/outputs. As Catherine D'Ignazio and Lauren Klein (2020) show, focusing on transparency misses the root causes of data injustice, while limiting potential responses. They propose a reorientation toward reflexivity, defined as the ability to both reflect on and be responsible for one's own positionality in larger systems of power. This reorientation can set aside the binary question of whether systems are closed and opaque to shift attention to equity and justice.

According to the Oxford English Dictionary, since the 1940s black box has been used to stand in for opaque systems in which the inputs and outputs are known but the inner workings are incomprehensible—a usage with special relevance to a metaphor for certain computational systems.⁴ By comparing this and other meanings, we will see how the dominant technological usage of black box maintains binaries of transparency and opacity while discouraging the possibility of language models as reflexive spaces for research and shared exploration. To enhance that idea, we conclude by seizing the metaphor of the black box theater—which, we argue, repositions the LLM as a reflexively performative space that recenters the humans behind and in front of the computational curtain.

Unboxing the “Black Box”

The first use of black box likely dates back to World War II,⁵ where the phrase referred to the “common black-speckled boxes” that were used in aviation “to encase radar electrical equipment such as amplifiers, receivers, filters, and so on” (Galison 1994: 256). The British Royal Air Force adopted black box as slang for aviation radar and navigation devices. Notably, as a radar tool, the “blackness” of the object in question refers to its carrying case, not its interior. Moreover, black box in this sense is a location device that one looks into for information, not a device that obscures it. A related usage of black box refers to flight recorders that aid in investigations of aviation accidents.⁶ Although information retrieval is the purpose of both devices, these practices situate and mediate positions of power. As Paul Benzon (2018) points out, the contents of a flight recorder are only shared in exceptional cases. When malfunction or accident are not issues, the flight recorder black box is a tool of erasure.⁷

In a context that overlaps with the OED's reference to opaque systems, Elizabeth Petrick has described the term black box as a full-blown cybernetic theory as well as a conceptual metaphor. Cyberneticians, midcentury scientists who were concerned with the control systems of both machines and living things, she argues, “found need for such a metaphor before they had a name for it and, over time, molded the black box into a shape that fit increasingly different kinds of problems. Ultimately, it became as we utilize it today—a shorthand term with an assumed and self-evident meaning; the black box metaphor became a black box” (Petrick 2020: 575–95; emphasis added). According to the OED's 1949 definition, this cybernetic understanding of black box refers to “a device which performs intricate functions but whose internal mechanism may not readily be inspected or understood.” W. Ross Ashby, in An Introduction to Cybernetics (1957), described “Black Box theory” comparably in terms of systems in which one may observe inputs and outputs but without knowing the interior mechanisms. He applied this theory to a wide variety of scientific fields:

Implicit here is how the psychologists who adopted “Black Box theory” regarded the biological brain's “neuronic mechanism” (in humans and other animals) as another opaque system. This context may help to explain the contemporaneity between the emergence of cybernetic theories and the use of neural network to describe a computational function inspired by the brain. Developed in the 1940s, around the same time as “black box” came to stand for obscure “internal mechanisms,” the belief that “neural networks” supposedly mimic synaptic mechanisms became the basis for the kind of “deep” machine learning that underlies LLMs today. Theorists in this vein conceived “neural networks” as opaque systems, whether in the human brain or in the latter's supposed computational analogue. Indeed, for Norbert Wiener, black boxes became “the model for a cybernetic understanding of the universe itself” (Galison 1994: 229).

Near the turn of the twenty-first century, the sociologist Bruno Latour (1999: 304) articulated the black box as a closed technological system: “When a machine runs efficiently, when a matter of fact is settled, one need focus only on its inputs and outputs and not on its internal complexity. Thus, paradoxically, the more science and technology succeed the more opaque and obscure they become.” Hence, technological advancement, according to Latour, relies on an unstable network of opacity— a series of black boxes connected to each other by obscured inputs and outputs. In this way, the black box becomes an opaque object that exists in and through its relation to other black boxes—a metaphor for an unstable network of opacity. This opacity allows for uncritical engagement and developer unaccountability at every part of the production process.

When Bender et al. (2021) describe unaccountable and undocumented data sets as a key danger of LLMs, they show how the notion of black box opacity continues to authorize and facilitate software development in the absence of critical reflection on the processes and structures on which these models are built. Indeed, one might ponder whether the commercial investment in proprietary secrets and unaccountable tools would be socially and legally acceptable had the black box metaphor and its associated theories been available to normalize the assumption that complexity is opaque.⁸ What we know is that the massive data sets that corporations now regularly fail either to document or share, and the trillion-parameter models that preclude careful analysis of an LLM's functionality, facilitate the system's entanglement with the drive to amass resources (including data), capital, and power.⁹ At the same time, use of undocumented training data and the perpetuation of surveillance leaves the question of a model's implications for safety and ethics to an afterthought. As companies aim to set up post hoc “guardrails” to mitigate the dissemination of bias, stereotypes, falsehoods, and conspiracy theories “parroted” from problematic data or generated through the system's confabulations, their typical recourse is to exploit human labor as cheaply and rapidly as possible. These increasingly necessary employment practices (which have been fundamental to machine learning from the start) should disrupt the myth of the high-tech black box—but have not yet done so.¹⁰ The goal must be to break open the metaphor of the black box without reinscribing a binary of transparency/opacity. Toward that end we conclude by reimagining the linguistic associations that the metaphor of the black box can generate.

The Black Box (as) Theater

There is another definition of the “black box,” albeit one that has not been explored in relation to LLMs: the black box theater—an idea that gained traction in the 1950s and 1960s.¹¹ Though the usages both derive from midcentury contexts, the black box theater stands for situated conditions that the black box as opaque system occludes: transparency, immediacy, and proximity. An explicitly antitheatrical theater space, the black box theater strips away figurative excess to focus instead on human presence, coproduction, and performance. In the centuries since modern theater developed as a site of cultural expression, theaters themselves became increasingly ornate, elaborate, and exclusive. Such theatrical design widened the separation between actors and audience, while elaborate sets created an effect of imaginative escape. The black box theater opposes the overwrought artifice of elite theater, instead facilitating performative intimacy through sparse set design. Could technologists embrace this immersive minimalism so as to conceive of language models as spaces of intimate performativity and reflexivity?

The black box theater is a simple performance space, generally with black walls and a flat floor. Bare, and with minimal set design, the black box theater becomes “a site of intimate appearances” (Steck 2013: 209). Often using a “theater in the round” floor plan, where spectators sit around the performance stage in a circle, the black box theater minimizes elaborate stagings that detract from performative intimacy and immersion. Studies like Jerzy Grotowski's Towards a Poor Theatre (1968) and Peter Brooks's The Empty Space (1968) solidified black box theater aesthetics in the 1960s. As their titles suggest, the black box theater was intended to democratize and unembellish the conventionally hierarchical and ornate space of theaters. According to Claire Bishop (2018: 33), the elimination of “theatrical trappings, stripping away elaborate technology and sets” enables black box directors “to expose the actor-audience relationship that they perceived to be the essence of theatre.” Here technology is conceived as an impediment to “immediacy, proximity, and communion” (34). Actors performing in black box theaters speak their lines close-up; viewers can see detailed facial expressions and even sweat on their brows. There are no hidden tricks or theatrical effects.

Turning this lens on LLMs, we find that the metaphor of the black box theater compels us to recognize how the supposed opacity of any space is always constructed—in this case in part through the conceptual and theoretical work of “black box” as a metaphor for complex systems. In contrast to the opaque black box, black box theater invites users and audiences to reflectively partake in the performance. Theater performances are meant to be fully experienced—viewed, heard, and felt. According to Rachel Steck (2013: 209), black box theater disrupts the boundaries of performer and spectator: “It breaks from the traditional frames and gazes, insisting on exposing the ways in which we look/view/see while also coercing us (the spectators) to look back . . . subvert[ing] notions of single authorship, authority and knowledge.” By “undermining normative looking . . . practices” (209), the black box theater forces users to engage in the explicit ways that performance is enacted. This stance of reflexivity could enable an LLM's users to reflect on the highly mediated performance of “AI” at work, including their own participation in the processes and power dynamics that underwrite the performance. As performance theorist Diana Taylor (2003: 2–3) puts it, “Performances function as vital acts of transfer, transmitting social knowledge, memory, and a sense of identity through reiterated [behavior].” Reconsidering LLMs as performative objects that rely on statistical probability rather than magical “boxes” enables us to focus on these acts of knowledge transfer, which, as we know, include the repetition of biases and the iteration of hierarchies of power.¹² In inviting spectatorship, audience, and intimacy instead of unknowing engagement, the black box theater creates the critical literacies necessary for new forms of collaboration and engagement.

A more intimate look at the construction of LLM performance begins with an analysis of the concentrated political economy on which they are built and which they help to sustain. Computer scientist Meredith Whittaker (2021: 52) writes about “how monopolistic control [over] resources gave a handful of tech companies the authority to (re)define the AI field, while enclosing knowledge about AI systems behind corporate secrecy.” With little knowledge of these material and socio-technical conditions, users unwittingly acquiesce to the economic, ecological, and ethical transgressions required to build, implement, and commercialize these proprietary and surveillant systems. When a user sits down and types into the chat interface of a chatbot, they get a response in return. Perhaps they know this is a predictive response “parroted” from scraped training data. Perhaps they do not. Whereas users typically experience a magical output that follows their prompt, the black box theater metaphor requires us to think about and center the elaborate scenery and backstage effects that have made that performance possible.

In their “Anatomy of an AI System,” Kate Crawford and Vladan Joler (2018) explain how automated systems (in this case Amazon's Alexa voice assistant) require a “vast planetary network, fueled by the extraction of non-renewable materials, labor, and data.” The scope, they argue, “is overwhelming: from indentured labor in mines for extracting the minerals that form the physical basis of information technologies; to the work of strictly controlled and sometimes dangerous hardware manufacturing and assembly processes in Chinese factories; to exploited outsourced cognitive workers in developing countries labelling AI training data sets; to the informal physical workers cleaning up toxic waste dumps.” Data-hungry chatbots extracting almost the whole of the scrapable internet demand even more resources from this vast network of servers and wired devices—the building and maintenance of which require raw materials and intense, intricate, and multitiered supply chains. As Karen Hao (2019) has reported based on the research of Emma Strubell, Ananya Ganesh, and Andrew McCallum, the training of deep learning models such as GPT-2 (the much smaller precursor to GPT-4) emitted nearly five times the amount of carbon dioxide as an average American car, including its manufacture. Billy Perrigo (2023), meanwhile, investigated the traumatic gig work of Kenyans paid less than $2 per hour to label the most disturbingly violent and toxic content in order to make ChatGPT less likely to spout it. In this way chatbot performance relies on environmental debt and exploitative labor practices that the careful staging of engineers conceals from everyday users. These processes accumulate wealth and power for a tiny elite while amassing data, exacerbating ecological crisis, and creating potential economic and political turmoil for the rest of the world. To this carefully curated and much-hyped sleight of hand, the tech industry adds an increasingly mainstream long-termist ideology that trades on anxiety about the future and exaggerates the possibility of AI sentience and superintelligence.¹³

When “theater in the round” floor plans place actors in the center of the room, the ensuing intimacy enables audience and performers to engage each other, bodily and otherwise, in a relational space of explicit performance. Were LLMs to follow the cues of black box theater, their constructed layers of power would be laid bare. At every stage of chatbot production, humans are required to extract materials, build cyberinfrastructures, create data, design systems (e.g., through web scraping, coding, and the choice of algorithms and architectures), envision business strategies, and enlist the human feedback (and marketing campaigns) necessary for turning “stochastic parrots” into plausible conversers, writers, and tools. Yet, while the black box theater concept compels us to visualize and account for this multistage process, it also asks users to consider the model in its entirety. Hence beyond the visibilization of a performance of “intelligence” and “cognition” is a holistic understanding of the practices and political economies that allow “generative AI” to function. A reflexive stance toward these systems is, we contend, a necessary first step toward the work of data justice. It's only after we recognize the layers of power displaced by the metaphor of the opaque “black box” that we can begin to challenge the large-scale exploitation that LLMs depend on and will amplify.

In Sum

We have argued that the black box theater's emphasis on intimate relationship, minimalistic staging, and holistic experience can challenge the disabling separation between language models and their users. Users must at minimum recognize themselves—and their data—as participating in the construction of LLMs including both their harms and affordances.¹⁴ Corporate transparency (the sharing of proprietary information about data sets, architectures, labor practices, and environmental footprint, and so on) is important to consider. But beyond these counters to black box opacity, users might also consider alternatives to current concentration of power: for example, open-source training data repositories and AI governance boards with diverse stakeholders including affected communities (see also Sylvie Delacroix's thinkpiece in this special issue). Bode and Goodlad (2023) have repurposed the idea of the “data commons” to argue not only for public accountability but also to “mark a capacious and open-ended alternative to the data worlds now enclosed by—and deployed in the interests of—a concentrated elite.”

When we understand LLMs as a performative technology that enacts the social and material relations and practices of a dominant political economy, the need to look “inside” an opaque system becomes, instead, a way of understanding what makes that system seem natural and inevitable despite that opacity. These steps toward data justice are part of what humanistic inquiry, with its keen sense of the metaphoric work of all language practices, contributes to critical AI studies.

Acknowledgments

The authors would like to thank Lauren Klein, Ben Miller, and Dan Sinykin for their generous guidance, thoughtful teaching, and indispensable feedback on previous drafts of this work. We are lucky to learn from you.

Notes

Works Cited