Fiat Potentia
What can a classic 1950s sci fi novel teach us about our uncertain AI future?
“…the war, after all, was none of Thon Taddeo’s affair. The political aspirations of his ignorant cousin were far from his own interest in a revival of learning in a dark world, except when that monarch’s patronage proved useful, as it already had upon several occasions.” –A Canticle for Leibowitz
Prior to the July 4th holiday weekend, I had been organizing materials and thoughts on Chinese and Marxist views of dignity, preparing for a new essay, but in keeping with the tradition of light holiday reading, I picked up a couple 1950s science fiction classics by Walter Miller: A Canticle for Leibowitz and Conditionally Human.
Despite being classified as fiction, science fiction is one of the best places to turn for insights into possible future realities. Unlike think tank reports or academic papers, science fiction authors are forced to grapple with the full texture of imagined worlds—not just broad predictions, macro forces, and policy implications, but the detailed internal and external lives of the people living inside them. You can't build a convincing world without accounting for what it actually feels like to live in it.
I first read A Canticle for Leibowitz way back in high school, but wanted to give it a re-reading to see how it might apply to the age of artificial intelligence. I will address the second book, Miller’s Conditionally Human, in a future essay.
A Canticle for Leibowitz is composed of three connected stories, each described from the point of view of members of a religious abbey. The first story, Fiat Homo (“Let There Be Man”), is situated in a post-nuclear holocaust world. Members of the abbey are tasked with rescuing and protecting the “Memorabilia,” fragments of papers and artifacts that survived the apocalypse: in essence, preserving knowledge.
The central tragedy tying the three stories together is that by protecting the Memorabilia and pursuing the completion of this knowledge , humanity ends up at the same point of nuclear destruction. We are doomed to repeat the same mistakes, despite knowledge of prior disasters. I don’t want to give away too much of the book for those that haven’t read it, so won’t try to summarize all of the storylines here.
We spend most of the book looking at the world from the point of view of one of the monks–either a novice, a brother, or an abbot–across three time periods. But in the second story, Fiat Lux (“Let There Be Light”), the abbey is visited by a character named Thon Taddeo Pfardentrott, a member of the collegium (“thon” serves as an honorific for an accomplished scholar). Thon Taddeo is a renowned scholar, the closest thing that age has to a scientist. To me, he is the most striking character in this read-through, at least in terms of how he relates to our current age and predicament.
Thon Taddeo is a bastard cousin of Hannegan, the ambitious ruler of Texarkana who is aggressively consolidating power across the continent through a combination of military force and political manipulation. Taddeo lives and works within the patronage system that Hannegan controls, and he is under no illusions about who his cousin is. Taddeo tells himself that his work exists on a separate plane from Hannegan's ambitions. Knowledge is knowledge. Science is science. What rulers do with the knowledge that scholars produce is not the scholar's responsibility. But Taddeo also knows that the collegium only exists because it is protected by Hannegan.
Thon Taddeo has come to the abbey to study the Memorabilia. His single-minded dedication to pursuing the limits of knowledge drives his intense intellectual ambitions. He believes in his own neutrality, similar to present day scientists and technologists: belief that knowledge will, by its nature, advance humanity and that technology should be pursued for technology's sake. Fiat Lux shows us that this belief is a trap.
While Thon Taddeo studies at the abbey, Hannegan finally declares war and the abbott Dom Paulo confronts Thon Taddeo about his relationship with Hannegan and the uses of this knowledge. Thon Taddeo begins with an honest assessment of his position:
[Taddeo] “Let’s be frank with each other, Father. I can’t fight the prince who makes my work possible–no matter what I think of his policies or his politics. I appear to support him, superficially, or at least to overlook him–for the sake of the collegium. If he extends his lands, the collegium may incidentally profit. If the collegium prospers, mankind will profit from our work.”
[Paulo] “The ones who survive, perhaps.”
[Taddeo] “True–but that’s always true in any event.”
[Paulo] “No, no—Twelve centuries ago, not even the survivors profited. Must we start down that road again?”
Thon Taddeo shrugged. “What can I do about it?” he asked crossly. “Hannegan is prince, not I.”
[Paulo] “But you promise to begin restoring Man’s control over Nature. But who will govern the use of the power to control natural forces? Who will use it? To what end? How will you hold him in check? Such decisions can still be made. But if you and your group don’t make them now, others will soon make them for you. Mankind will profit, you say. By whose sufferance? The sufferance of a prince who signs his letters X? Or do you really believe that your collegium can stay aloof from his ambitions when he begins to find out that you’re valuable to him?” (1)
Ultimately Dom Paulo offers Thon Taddeo a choice: work for God and protect this knowledge from those who would abuse it or work for Hannegan. Thon Taddeo replies that in that case, there is no choice: he will work for Hannegan. Science needs to be applied, not cloistered, damn the consequences. “If you try to save wisdom until the world is wise, Father, the world will never have it.”
The parallels with the present day are hard to miss. Executives at leading AI companies are making these same moral compromises. They, like Thon Taddeo, treat power as a natural fact rather than a moral choice. Walter Miller shows us that there are no morally neutral technological decisions when one exists in a political world. Leaders have ambitions to increase and extend their power. The frontiers of technology will always be pursued, shaped by and directed to military advantage.
What would it have taken for Taddeo to choose differently? What would he have needed to see that he didn't? One of the key insights would have been the ability to view humans as ends, rather than knowledge and technology as ends. This would have allowed Taddeo to see the people his work would ultimately be used against as full persons rather than abstractions and collateral damage on the road to knowledge. This will be a major topic to unpack more fully in coming essays.
In the previous essay on Magnifica Humanitas, I argued that existing ontological frameworks for dignity are insufficient; for practical purposes, while these do position humans as ends, they still treat them as an abstract, or, at the very least, don’t provide enough structure to evaluate and measure what is helpful and harmful for promoting human dignity. The stakes are high. In A Canticle for Leibowitz, Walter Miller demonstrates that the cost of insufficient frameworks for valuing human dignity is the destruction of civilization. The cycle of folly is unbreakable, and the people who think they can stay neutral never do.
My one criticism is that Miller shows us the trap with poignant and devastating clarity, but doesn't give us the tools to escape it. That is the purpose of this Dignity Hub project and where we will kick off the next essay.
(1) Walter M. Miller, Jr., A Canticle for Leibowitz (1959; repr., New York: Bantam Books, 2007), pp. 223-224.
