Intelligence (Artificial)
in a Fallen World

A Kishōtenketsu起承転結(kishōtenketsu) on AI Risk

This work was funded by the generous patronage of Saar Wilf. All thoughts and words are my own.

I. The Will of Angels

“Once upon a time a great Cabbalist lived in Prague, called the Rabbi Löw. He made a human figure of clay, and left a small aperture in the lesser brain in which he laid a parchment with the unutterable name of God written on it. The clod immediately arose and was a man; he performed all the duties of a servant for his creator, he fetched water, and hewed wood. All through the Jews’ quarter he was known as the Golem of the great Rabbi Löw.”

— Berthold Auerbach, Spinoza (1837)

A common topic of interest in 18th-century French salons was the theodicy: given an omnipotent, omnibenevolent God, why is there evil in the world? This excited a range of answers. There was the Panglossian view, the pollyannaish theodicy that, “all is for the best in the best of all possible worlds”. Others subscribed to the Irenaean theodicy that evil is necessary for full human development.

A particularly prominent theory though was one first argued by Saint Augustine of Hippo. Pulling from the Neoplatonists, Augustine first contended that evil is not a thing in itself, but rather a privation of the good. Then, in exercising free will, a great gift in itself, humans can move themselves away from the good.

However, there seems to be a slight hole in this notion. While the theodicy is adequate for explaining human-derived evil, it does not resolve the ‘problem of evil’ in regard to natural sources; if evil extends from human freedom, where do we place natural disasters, diseases, or independent animal suffering?

To patch this issue, an addendum can be made to the theodicy in what is commonly referred to as the ‘two falls theodicy’. In standard Catholic metaphysics, angels are aeviternal beings which apprehend ‘intuitively’ and whose wills, therefore, adhere immovably. When an angel makes a choice, it persists forever. In the ‘two falls theodicy’, the non-human mediated ills of the world are downstream from these singular decisions of angels who reject the ‘good’.

A simple choice, made by a being of sufficient capacity, is enough to irrevocably distort the world. For most of human history, consequences were limited to our immediate positional and temporal surroundings. Today, actions engraved into the embeddings of systems have the capability of far outlasting their original context.

Small changes to the starting state of a complex system can incite highly divergent outcomes. An object with a constant ‘pop’ of 1 m/s⁶, assuming no other initial conditions, travels the circumference of the Earth in 56 seconds and achieves a velocity faster than the speed of light in 130.

We stand now at a tipping point in history, where present choices can ramify forward without recall.

II. Why the West Has (Not) Won;

or, The Incoherence of the Incoherence

Scientific development requires a specific epistemology. In order to proceed over time in scientific inquiry, one has to believe both that the world is knowable and that knowing the world is good.

In 1020, ibn Sīnā published The Book of Salvation. Within its second ‘treatise’, over two centuries before Aquinas’ Five Ways, ibn Sīnā describes a proof of God arising from the contingent and yielding a necessary existent. Given a necessary existent, then all else stems from it and insofar as things stem from it they are knowable and good.

Less than 100 years later, al-Ghazali, the ‘mujaddid’ of the 11th century, wrote the Incoherence of the Philosophers. Here al-Ghazali argues for an Asharite occasionalism:

“Let us, then, take a specific example—namely, the burning of cotton, for instance, when in contact with fire. […] The one who enacts the burning by creating blackness in the cotton, [causing] separation in its parts, and making it cinder or ashes is God […] As for fire, which is inanimate, it has no action. For what proof is there that it is the agent? They have no proof other than observing the occurrence of the burning at the [juncture of] contact with the fire.”

While the world may be good, it is not knowable.

While pushback followed, al-Ghazali’s approach proved ascendant. The madrasa system displaced the older patronage networks that sustained the Bayt al-Ḥikmah tradition. Natural philosophy made way for a curriculum which prioritized jurisprudence and Qur’anic exegesis. The Islamic world, which had preserved and expanded on the Western tradition in philosophy, math, and science for half a millennium while Europe languished in darkness, became instead the recipient of European advancement.

The Latin West maintained the twin epistemologic pillars for slightly longer. Kepler presented astronomical inquiry as a way of knowing God. A century later, Boyle’s Christian Virtuoso defended experimental philosophy as a way to illuminate God’s goodness. In the General Scholium to the Principia, Newton held that the cosmic system “could only proceed from the counsel and dominion of an intelligent and powerful Being.”

However, with the Enlightenment, the idea that knowing the world is good began to fall. A possible theodicean response is to break down the premises. One cannot establish an ‘ought’ from an ‘is’, so without an explicit moral ordering of the universe where is value derived? Scholastic vestiges remained but unmoored from any metaphysical anchor.

In ethics, the good became restructured around the “two sovereign masters” of pain and pleasure; efficiency the dominant signal in markets. Without a telos, purpose came from what could be measured: “anything which does not conform to the standard of calculability and utility must be viewed with suspicion”. The value of discovery limited to its instrumentalization.

Modernity still has its triumphs. While the tractable is not identical to the good, the mode of orientation can be near enough to appear aligned in non-pathologic scenarios. In the 20th century, life expectancy doubled. Famine has become a peripheral phenomenon primarily associated with war rather than agriculture. Smallpox, the killer of over half a billion, has been eradicated.

Its failure in sensing the inherent dignity of the person, the temptation towards the merely legible, though curses the regime of modernity to misalign from the good. In Buck v. Bell, the United States Supreme Court, in an 8-1 decision, upheld the forced sterilization of the ‘unfit’ to promote the general health and welfare of society.

“It would be strange if it could not call upon those who already sap the strength of the State for these lesser sacrifices… in order to prevent our being swamped with incompetence. It is better for all the world, if instead of waiting to execute degenerate offspring for crime, or to let them starve for their imbecility, society can prevent those who are manifestly unfit from continuing their kind. The principle that sustains compulsory vaccination is broad enough to cover cutting the Fallopian tubes. Three generations of imbeciles are enough.”

In perhaps the central tragedy of the modern age, the Holocaust, modernity again showed its fruits. Though history is littered with stories of mass killings, the Holocaust was not a harkening back to a barbaric past. It is distinguished not just in magnitude but also in kind, by a particular rationality. The genocide was taken as a means and correct from within its own axiomatic structure. Without an external understanding of the good, there is no end toward which to correctly direct one’s rationality.

Our current tech landscape has provided remarkable achievements, yet there still remains an absence at its core. As we pursue greater advancements, the divergence between the efficient and the good becomes increasingly pronounced. While in an age of relative abundance, we are tightroping across a path where, with rising capability, a small deviation can quickly cascade into a fall. Yet the system, having lasted hundreds of years, resists correction; corrigibility to the numinous remains strained.

III. Sed Contra

While the development of artificial intelligence carries grave risk and must be taken with the utmost care, many standard objections serve to obscure the genuine concern rather than clarify. These critiques both inflate and diminish capability while positing issues orthogonal to any real capacity.

On Stochastic Parrots

A common critique given to modern generative AI systems is that they are merely ‘stochastic parrots’ or ‘next token predictors’. The tricky part of this statement is that there is some level where this is correct.

There is a sense that what is occurring when prompting an LLM is ‘predicting’ the next token in a sequence. However, emergent phenomena are the norm in complex systems, and when exhibited, generally make a description of parts to explain the whole not ‘untrue’ but misleading or non-helpful. Outside of highly specific scenarios, it is not useful to explain a computer as a series of transistors or further to describe any matter according to its fermionic makeup.

Most commonly, this type of argumentation seems to occur when one is trying to make a claim to separate human cognition from what occurs in LLMs. However, mereological observation does not constitute ontological proof. It is clear from even minor use that LLMs are not just stochastic parrots. Statements on what LLMs ‘are’ should be made directly, not laundered through a framing which falls apart on first contact.

On the Environment

Despite apocalyptic hyperventilation to the contrary from media, the environmental footprint for AI is fairly small. Obviously, as something that exists in the world, it has some environmental effect, but relative to its use this is negligible and possibly even negative if considering displacement costs.

“Even just going for a walk outside slightly wears out your sneakers. Typical athletic sneakers often last for 300-500 miles, and take 4 million ChatGPT prompts’ worth of water to make. This means that every mile you get out of your sneakers uses 8000 ChatGPT prompts’ worth of water. Every single second you spend walking in sneakers uses enough water for 7 prompts. At the end of an hour walk, you would have used enough water for 24,000 prompts.”

A similar story exists for emissions. Even in a ‘lift-off’ case with extremely strong AI uptake, limited local constraints, and low efficiency improvements in hardware or software, the IEA estimates data center emissions to only be at 475 Mt CO₂ by 2035 or less than ⅓ the output of the cement industry.

This is not to say that data centers are immune to causing grid strain at a local level. However, even here, energy supply is historically highly receptive to demand. For those genuinely concerned about environmental degradation, the returns on attention are orders of magnitude higher in other areas.

On Intellectual Property

It has been a mainstay of internet culture, especially within tech circles, to be against intellectual property or believe it overly restrictive; the operating assumption being that information should be free. However, with the advent of LLMs, this notion has started to shift. There is a feeling that in AI’s unmediated hoovering of data, some implicit boundary has been violated.

In the United States, copyright has always been something granted for the public benefit: “[t]o promote the Progress of Science and useful Arts”. Due to the inherent non-excludability of ideas, it is a way to incentivize their creation and discovery. Copyright is merely a statutory concession for the good of the public and not a protection that flows from one’s natural rights.

To maximize the public welfare, even in the period provided, an author’s rights are not absolute. The bounds on the exclusive rights of works were codified in the Copyright Act of 1976 which established a four-factor test that utilized the purpose of use, the nature of the copyrighted work, the amount used, and the market impact on the copyrighted work to determine if a secondary work was a ‘fair use’ of the copyrighted material.

With regards to AI, the use of copyrighted materials for training has been found to constitute fair use due to its highly transformative nature. While whether the outputs of models themselves can constitute copyright infringement is under ongoing litigation, copyright is a poor mechanism for addressing the displacement AI creates.

If we take AI displacement in specific industries to be bad, there are more direct and efficient ways to alleviate the harms. A Pigouvian tax on LLM providers, earmarked for affected industries, could compensate creators without requiring the increasingly strained application of a framework designed for a different issue.

On Hallucination

One of the most tangible concerns people have of LLMs is related to having factual errors in outputs or ‘hallucinations’. To clarify against what is a slightly common misconception, hallucinations do not come from some inherent ‘randomness’ of LLMs, but rather through ‘mistakes’ in the models of the world being inferred into.

The upside of this is that in using larger pre-training corpora and more robust post-training processes, successive model generations have significantly reduced hallucination rates while simultaneously becoming better calibrated in expressing uncertainty.

A common mistake in judging digital output is comparison to an ideal form rather than existing alternatives. Compared to the counterfactual, existing deployment delays on autonomous vehicles have cost thousands of lives. Even in fields such as medical diagnosis with high potential consequences for inaccuracy, LLMs have been found, in certain circumstances, to exceed physician accuracy.

This is not to say that hallucinations are not an issue, but rather that any sufficiently complex system will exhibit failure modes. When examining AI errors, it should be done in cognizance of their current rate of improvement and relative to the systems they stand to augment.

On Mania

A final critique often levied against the AI industry as a whole is that it represents a speculative bubble whose eventual correction will inflict widespread economic harm. Stories of Allbirds, the shoe company, whose shares climbed over 600% after announcing a pivot to AI compute infrastructure, make it hard to deny some mania in the markets.

However, it is perhaps clarifying here to view the AI industry in relation to its last major speculative episode: the dot-com bubble. In March of 2000, directly prior to the dot-com burst, information technology forward P/E stood at ~48.3x. Today, due to massive earnings increases, the mark stands at only ~24.2x. If we look at the ‘shovel’ play in both periods, Cisco had a P/E of ~222.4x, while today Nvidia still only stands at 44.8x. Insofar as AI is a bubble, its ‘irrational exuberance’ still has room to grow.

Even after any excess froth gets wrung out, the fallout is unlikely to follow the same patterns as some previous recent corrections. For established tech companies, AI is best thought of as a ‘sustaining’ rather than a ‘disruptive’ innovation. Following from this, a majority of capex has come from free cash flow of highly profitable existing tech companies. Compared to the dot-com crash, which was funded by retail speculation, or 2008, which was built on leveraged consumer debt, the current situation carries much less structural risk. Further, in the case of a downturn, the majority of the build-out here is in compute and energy, which besides being highly fungible also historically have high positive externalities.

IV. Prohairesis

“The Rabbi ordered the precentor to pause at the end of the prayer. It was yet possible to save all, but later naught would avail — the whole world would be destroyed. He hastened home, and saw the Golem already seizing the joists of his house to tear down the building; he sprang forward, took the parchment out, and dead clay again lay at his feet.”

— Berthold Auerbach, Spinoza (1837)

Despite their inadequacies, there is a reason why these anti-AI arguments persist. People have an intrinsic perception that there is something hollow at the center of the modern world and so glom onto the first rationale with even a façade of logical coherence.

Those who make claims of ushering in a new age are of the same group that brought us Soylent, engagement-optimized feeds, and claims of massive AI job displacement. Even when AI leaders make remarks pointing to more humanistic ends, it is hard to take these outside the milieu they operate within. “Who would put up with the Gracchi complaining about sedition?”

It is well understood across a range of fields that negative signals carry more weight than positive signals. In decision-making under uncertainty, losses carry twice the weight of equivalent gains. A common response then is to treat the asymmetry as a bias to be corrected and so actively discount any negative stimuli. However, occasionally several hundred million years of evolutionary selection encodes valuable information; Taleb distributions are real and often non-obvious.

So where do we go from here? One must walk away from St. Petersburg; silence the martingale. In acting, we immediately shape our soul before any external consequence. To be truly in zugzwang requires a fully staid environment; however, we can change the conditions in which we operate under.

Technology qua technology is not a good in itself. Its pharmakonic nature is decided in relation to the telos on which it is produced. We currently exist in an axial moment where choices made today can have an outsized effect on the future. In proceeding, one must act with caution, humility, and an understanding that in building the ‘ends’ must be aligned to and in respect of the fundamental dignity of the human person.