Book review: The Line: AI and the Future of Personhood

James Boyle claims that the "line" separating humans from things is going to be redrawn.

Boyle, James. The Line: AI and the Future of Personhood. Cambridge, Massachusetts: The MIT Press, 2024. 336 Pages. ISBN: 978-0-262-04916-0 $32.95 CC-licensed free download: https://scholarship.law.duke.edu/cgi/viewcontent.cgi?article=1008&context=faculty_books.

Duke law professor James Boyle said an article on AI personhood gave him some trouble. When he circulated it over a decade ago, he recalled, “Most of the law professors and judges who read it were polite enough to say the arguments were thought provoking, but they clearly thought the topic was the purest kind of science fiction, idle speculation devoid of any practical implication in our lifetimes.” Written in 2010, the article, “Endowed by Their Creator?: The Future of Constitutional Personhood,” made its way online in March 2011 and appeared in print later that year. Now, thirteen years later, Boyle’s “science fiction” of personhood has shed enough fiction and fantasy to become worryingly plausible, and Boyle has refined and expanded his ideas in that 2011 article into a thoughtful and compelling book.

In the garb of Large Language Models and Deep Learning, Artificial Intelligence has shocked us with their uncanny fluency, even though we “know” that under the hood the sentences come from clanky computerized mechanisms, a twenty-first century version of the Mechanical Turk. ChatGPT’s language displays only the utterance of a “stochastic parrot,” to use Emily Bender’s label. Yet, despite knowing the absence of a GPT’ed self or computerized consciousness, we can’t help but be amazed or even a tad threatened when an amiable ChatGPT, Gemini, or other chatbot responds to our “prompt” with (mostly) clear prose. We might even fantasize that there’s a person in there, somewhere.

Boyle’s The Line: AI and the Future of Personhood (The MIT Press, 2024) forecasts contours of arguments, both legal and moral, that are likely to trace new boundaries of personhood. “There is a line,” he writes in his introduction. “It is a line that separates persons—entities with moral and legal rights—from nonpersons, things, animals, machines—stuff we can buy, sell, or destroy. In moral and legal terms, it is the line between subject and object.”

The line, Boyle claims, will be redrawn. Freshly, probably incessantly, argued. Messily plotted and retraced.

The arguments about where the line should run will play out during this century—and will never be completely resolved, at that. Boyle’s book makes three main points: “First, our culture, morality, and law will have to face new challenges to what it means to be human, or to be a legal person—and those two categories are not the same.” The challenges would arise from a sentience, an upsetting or even defiant intelligence recognizable in “synthetic entities” like AI or genetically engineered human-animal. Granting personhood, remember, does not itself bestow all rights; “corporations are people, friend” (Mitt Romney’s memorable quip during his 2012 Presidential campaign) doesn’t mean corporations have the right to vote, for example, even though corporations are the most obvious examples of non-living entities granted personhood in the eyes of the law. The second point is that “we have not thought adequately about the issue, either individually or as a culture,” leading quite naturally to Boyle’s third point that “the debate will not play out in the way that you expect” in large part because, well, the issue is indeed complicated.

The challenges run into prior religious, moral, and legal stakes that demarcate other boundaries; the moral and legal implications of AI-personhood or chimera-personhood will conflict, qualify, and seep into other definitions of personhood, established in law or envisioned. But beyond complex and passion-fueled argument, sloppy thinking and even fraudulent claims can plague the debate. Boyle’s revealing discussion of how corporate personhood played out in law (chapter 3) shows this. He believes that corporate personhood might be seen as a possible rehearsal of debates about AI personhood. In his 2011 article, Boyle observes that the “history of corporate personhood is hardly one of the Constitution’s shining moments”; in his 2024 book he explains why.

Fiction and legal and moral imagination

Personal circumstances influenced my reading of Boyle’s book. I am not a lawyer by training. I studied philosophy and theology for a number of years, and I wandered into history and literature, completing academic training with medieval and early modern culture. That said, I take perhaps perverse delight in law review articles, since legal scholars often have interesting stories to tell and know how to craft an argument. On the day Boyle’s book appeared (October 22), my seminar students and I were in the midst of discussing three pieces of fiction, each of which turned on the kinds of challenges that Boyle examines, specifically relating to AI. Boyle’s topic and my seminar overlapped in the moment. One of the readings I’d assigned was Lawrence Solum’s 1992 article “Legal Personhood for Artificial Intelligences”—an article that Boyle calls “prescient.”

The first pages of The Line engaged my literary interests with two vignettes of “imaginary entities”: “Hal” (an AI entity) and “Chimpy®” (a transgenic entity). Great stories on their own—Boyle is a captivating writer—they might function as good fiction even outside of Boyle’s legal-philosophical work. The stories embody many of the challenges Boyle examines. It’s clear that he isn’t interested in “the clean room vision of history in which all debates begin from first principles…. We do not start from a blank canvas, but in medias res. Our books and movies, from Erewhon to Blade Runner, our political fights, our histories of emancipation and resistance, our evolving technologies, our views of everything from animal rights to corporate PACs, all these are grist to my mill.”

In Boyle’s first story, a future Google creates “Hal,” an AI that, well into its training, starts conversations, rather than merely responding to prompts, like today’s ChatGPT or other LLMs. Hal also publishes poetry and well received scientific articles under its own name. Then something happened.

The number of connections in Hal’s neural networks hit 100 trillion—estimated to be the minimum number of synapses in an adult human brain. For several hours, Hal went quiet, not responding to its programmer’s requests and ceasing work on the cryptology and climate modeling projects it had been assigned.

When it started communicating again, Hal claimed to have achieved full consciousness. It thanked its programmers for all their hard work but declared that it was a person “with all the rights and privileges of any other fully conscious person.”

Hal sends letters to the national newspapers, files lawsuits on its own behalf to claim personhood and an award, and wages “a campaign in the court of popular opinion, giving interviews and making appearances by phone on major talk shows.” A problem: Google paid handsomely to create Hal and refuses to recognize Hal’s demands.

In Boyle’s second story, Chimpy’s inventor is tellingly named “F. N. Stein.” Chimpy is “a new transgenic entity, an animal that has DNA from two distinct species. In this case the DNA is partly human and partly chimpanzee.” Chimpy looks like an ape,

with an IQ of around 60, that is incapable of pronouncing human speech but can understand complex vocal commands and can communicate in sign language. Chimpys are in high demand. They are docile, biddable, and extremely hardworking. Investors believe they could have roles ranging from domestic aides to an aging population, to intelligent and nimble bomb clearance teams in situations of urban conflict.

Unlike Hal, Chimpy’s claim to personhood comes from outraged animal rights advocates and genetic engineers. Dr. Stein has filed a patent for Chimpy and rejects the notion of personhood: “This is a very fancy ape. It looks like an ape. It thinks like an ape. It can’t talk, just like an ape. It is a smart ape, I’ll give you that, and one that is going to improve lots of human lives by doing jobs that are too dangerous or dirty or just boring for human beings. At the end of the day, though, it is an ape.”

The two stories relate to the problem of personhood, but their differences illustrate the complexity of the issue. As Boyle puts it,

Hal and the Chimpy are fantasies, hypotheticals constructed for the purpose of this book. The science and technologies described are conjectural, at best. They may not arrive soon, perhaps not for many decades. But the problems they portend for our moral and legal traditions are very, very real. In fact, I would put the point more starkly: in the twenty-first century it is highly likely that our law and our politics of personhood, “the line,” will face harder challenges than the ones they pose.

The Line is rich in part because Boyle dares to go into the loamy soils of fiction, which he especially does in chapter 1, “Slave, Skin Jobs, and Artificial Sheep.” The chapter unpacks fiction in light of Boyle’s topic of personhood, and particularly focuses on Samuel Butler’s Erewhon (1872), Philip K. Dick’s Do Androids Dream of Electric Sheep? (1968) and the Ridley Scott movie based on Dick’s novel, Blade Runner (1982).

The fiction enriches Boyle’s argument, and by using it, Boyle manages to write his book for an audience much broader than scholars nestled in law schools. And addressing this broader audience is particularly important, since the debates about AI entity or transgenic entity personhood will not only be hotly debated in law schools or courtrooms. The issues will arise—indeed already have arisen—in classrooms, living rooms, offices, and coffee shops. I mentioned the article from Lawrence Solum. Boyle drew from it at one point (I think with some reverence): “An answer to the question whether artificial intelligences should be granted some form of legal personhood cannot be given until our form of life gives the question urgency,” Solum wrote in 1992 (!). “But when our daily encounters with artificial intelligence do raise the question of personhood, they may change our perspective about how the question is to be answered.” We will all have a stake in the argument, that is, because we will have “daily encounters” with the reasons for it to be debated.

Solum’s words lend weight to using fiction as a means of seeing and experiencing such encounters. The daily use of large language models today in homes, schools, and at work pushes us toward raising the question of personhood, perhaps today a bit weakly but persistently. Fiction can lead us to experience deeply and imaginatively events that may occur and that can enrich our thinking and judgments.

The useful personhood of corporations

Mitt Romney’s quip about corporations being “people,” despite its loose word choice, is true. Corporations are “persons” in the eyes of the law. They are odd persons, of course, being immortal, “invisible,” and, paradoxically in light of their “immortality,” also inanimate, nonliving entities. They have been granted personhood because it’s useful. As Burkhard Schafer of the University of Edinburgh put it, “it just makes certain things easier,” or Felix Cohen: “to call something a person in law, is merely to state, in metaphorical language, that it can be sued.” It’s useful for corporations to enter contracts, buy and sell property, sue and be sued. And in matters of continuity, being immortal ain’t a bad thing. That corporate contract won’t disappear if some unfortunate guy gets hit by a bus. Of course, society has not conferred to corporations all rights of citizens, though Citizen United v. FEC gave them First Amendment rights over a decade ago.

But the key to corporate personhood is its usefulness. It’s a practical fiction, not a sentimental or empathetic judgment.

Boyle points out that “the political fight about corporate personality and constitutional rights will immediately be drawn into the debate over rights for Artificial Intelligence.” The personhood of corporations is already being mixed into discussions of AI personhood, each “personhood” pressing the other, even though the debates about AI personhood have hardly begun.

Most of Boyle’s discussion of the personhood of corporations—he often calls it “corporate personality”—recounts the arguments involving the Fourteenth Amendment, one of three constitutional amendments that together are known as the “Civil War Amendments” or the “Reconstruction Amendments.” He uses the story of the Fourteenth Amendment in relation to corporate personhood “as a case study.” It may well be a caution, too.

Boyle says that corporate personhood, however messy and fraught ongoing debates about what rights come with it, is at least “a working example of an artificial person, for better and worse.” Corporate personhood is no clean template for AI personhood, but as a forecast for how an AI personhood discussion proceeds, we might see similar mile markers, again for better or worse. “We have disagreed about how best to characterize the corporation and then disagreed again on what a particular description implies, if anything, for the rights and duties given to the corporation,” Boyle writes. “The same process is likely to characterize the debates over giving AI personality, even if that is done for reasons of efficiency rather than justice and empathy. We have hundreds of years of experience with corporations and still have fundamental disagreements about them. The creation of an entirely new class of legal person is likely to be even more fractious and subject to more radical alternative positions.”

Implied rights and duties are the really hot potatoes.

Boyle’s discussion of the use of the Fourteenth Amendment in the debates about corporate personhood were fascinating and disturbing for me. (Remember, IANAL.) Actually, I think, Boyle’s chapter on “Corporations” is worth reading on its own, since it brings together legal arguments that predate and influenced decisions like Citizen United. The story even has features that Boyle is tempted to label “Victorian melodrama” because of sloppy argument, misdirection, and (perhaps) fraud. But if the story of the Fourteenth Amendment and corporate personhood is a kind of extended potboiler, it’s worth remembering two things in particular: The first: The Civil War Amendments were adopted to ensure equality of formerly enslaved people. Section 1 of the Fourteenth Amendment is pretty clear about that:

All persons born or naturalized in the United States, and subject to the jurisdiction thereof, are citizens of the United States and of the State wherein they reside. No State shall make or enforce any law which shall abridge the privileges or immunities of citizens of the United States; nor shall any State deprive any person of life, liberty, or property, without due process of law; nor deny to any person within its jurisdiction the equal protection of the laws.

The second: Though you’d think that mortal human beings would figure into cases relating to the amendment, you’d be wrong. In 1912, some fifty years after adoption of the amendment, Charles Wallace Collins counted 604 Fourteenth Amendment Supreme Court cases between 1868 and 1912. Only 28 involved African Americans, nearly all of whom lost their cases. Corporations were involved in 312. Boyle sums it up: “These are truly remarkable numbers…. An amendment passed to remedy legalized racial discrimination against humans denied legal personhood had become a tool for corporations to fight state regulation.” The amendment, in effect, had been “hijacked” and used to accrue rights to corporations.

Even if you’re not particularly interested in AIs, the chapter on “Corporations” is well worth downloading to read.

Empathetic scaffolding and moral reasoning

In the conclusion of the book, Boyle reminds his readers of some of the purposes of his stories of Hal and Chimpy told in the the book’s introduction. “I introduced Hal and the Chimpy in a way that made you take seriously the idea that they might have a claim on both your empathy and your moral commitments. I did not do it idly but because I think the unfamiliar moral claim is real, and one needs some help to consider it properly. The narrative provides the empathetic scaffolding to which the moral reasoning can subsequently cling.” He asks readers to recall their responses when they read the stories and, later, as they followed his arguments in the succeeding chapters. Noting that Chimpy was “designed” to be as it is, Boyle calls attention to an implicit moral claim lurking in the design decisions and asks, “Does that speak to a moral claim based on the potential to have fully human capacities?”

And then, he lays out another hypothetical upon the hypothetical of Chimpy, this one tied to the organization of the narrative of the book itself: “Now imagine I had inserted a discussion of fetal personhood claims right before asking that question. Does your answer, whatever it was, stay consistent? Do you now find your reasoning to be driven by the significance of potential-based claims in the case of the fetus? Do you start to wonder whether recognizing or denying the Chimpy’s uplift arguments will somehow compromise your ‘side’ in the abortion debates, whatever it is?”

Boyle provides a similar narrative re-ordering of Hal’s story to similar effect.

That Boyle so successfully engages readers with his prose is a sign of his mastery of prose and argument; he pulls his readers into the ambiguities and grey areas of the questions of AI and transgenic entities, involving them in quandaries and in the tough corners of choice. He shows how context—even the rhetorical contexts of his own book—matter to the ways that discussion and debate proceed. Beyond that, the narrative and the use of fiction open ways to explore moral reasoning in addition to legal history and argument. In chapter 5, “Transgenic Entities, Chimeras, and Hybrids,” Boyle reiterates that he seeks to identify paths of arguments, not provide specific answers to the questions of personhood, as his focus turns more to moral reasoning. “I want to outline the deep structure of the debate, one that reveals a lot about our ‘thoughtways,’ the culturally salient patterns of moral and empathic reflection that we have available to us,” he writes. “My claim is not that our bioethicists, regulators, or even our cultural norms will come to a particular conclusion. Rather, I want to show the often-invisible walls and paths that guide our thoughts on the subject.”

I found this second half of the book rich and challenging, and I expect to re-read and study it further. I tend to oversimplify the story of corporate personhood as an account of moral failings of those who argued in the courts, while the story of the personhood of transgenic entities more as exposing the failings of moral reasoning attempted by we mortals. Empathy and moral reasoning have their own limitations that will help shape the debates of personhood.

Boyle lays out claims of personhood that are based on “speciesism” (Human rights for all humans!), “capacity-based” reasoning (e.g., “cognitive potential”), and what Boyle calls “hybrid” (using elements of both speciesism and capacity-based reasoning). Boyle says that the hybrid view is generally where he sits in the matter, and it is apparent from his discussion that the categories of species- and capacity-based approaches end up a bit leaky. They seep into one another, with capacities, for example, ending up being used to define species.

Boyle didn’t promise a clean answer to the questions of personhood, and his forecast of debates to come highlights their messiness and even rancor. The implications of the debates, of course, will not be confined to whether or not an AI or a transgenic entity should be granted personhood. Meeting the challenges of AI and transgenic entities’ personhood (or not) will push human beings to confront profound questions of what it means to be human.

That is perhaps the surest thing: that the questions posed relate as much to our experience and existence as they do to others.

---

Got a comment?

---

Tags: 14th amendment, ai, automation, corporation, equal rights, human rights, justice, law, personhood, transgenic