Here is the rest of my essay on AI. Obviously this is fast-moving terrain. I hope that some of the questions and themes I address will continue to be relevant in the coming months and years. And I strongly recommend the various articles and books I reference throughout the essay.
Rights and Consciousness
We now turn to one of the most vexed questions: can AI become conscious? If so, what would this entail? Rights? Citizenship? Equality with humans? To be clear, I’m inclined to answer “no” to all of the above. But let’s take a moment to inquire more fully.
First, one could spend a lifetime tackling questions of consciousness. Some philosophers, cognitive scientists, psychologists, doctors, and others do in fact spend entire careers seeking to better understand, explain, or even just define consciousness. It is difficult just to settle on a shared understanding of what it means to be conscious in the first place.
The philosopher Thomas Nagel famously said to be conscious means we can ask “what is it like to be that creature?” Humans, of course, are conscious. But so are other animals. It thus makes sense to ask Nagel’s famous question “what is it like to be a bat?” I often ponder the same about my cat as do countless other pet owners, not to mention bird watchers, zoo guests, and others inclined to such thoughts.
But again, what a strange and roundabout way to define a central feature of animal life. Nagel seems to be gesturing at the idea that conscious creatures experience the world in some meaningful sense such that we can wonder how they experience it. Ed Yong’s beautiful book An Immense World, for instance, explores how varied such experiences are across the animal world. Animals, like humans, have experiences.
On the other hand, rocks, chairs, and calculators do not. Neither does ChatGPT or Claude. They have no interiority. Let’s think about this some more because it really does matter. One of the things that so bothers me about the AI discourse is that it shifted from political and economic questions regarding automation, basic income, unemployment, worker power, and so on, to tycoons and engineers attempting to create new conscious digital entities and possibly even merge with them, all born out a desire to escape our humanity and our embodied, animal nature. To be clear, I unequivocally reject this and I think you should too. Let’s see why.
The philospher Anil Seth, in an award-winning essay titled “The Mythology of Conscious AI,” reminds us that “the way we think about conscious AI matters for how we understand our own human nature and the nature of the conscious experiences that make our lives worth living. If we confuse ourselves too readily with our machine creations, we not only overestimate them, we also underestimate ourselves.” Many of the biggest voices in AI have gone astray while tackling these very issues.
To start Seth distinguishes between intelligence and consciousness. They are not in fact the same. As Seth says, consciousness is about being in the world and how we experience the world. Hence the famous Nagel bat example. Intelligence, on the other hand, can be used to refer to the ability to solve problems. Using this distinction, AI agents may be intelligent, perhaps increasingly so, but they are not conscious. They don’t experience anything. It is thus possible, maybe likely, that we will develop AI agents that become far more intelligent than us but lack consciousness. Maybe they will someday become conscious, like the malevolent Skynet or the friendly super-intelligence imagined by Sam Altman, but Seth thinks not.
As he says, “when we identify conscious experience with seemingly human qualities like intelligence and language, we become more likely to see consciousness where it doesn’t exist, and to miss seeing it where it does. We certainly should not just assume that consciousness will come along for the ride as AI gets smarter…” This point matters. ChatGPT, in this simple and analytical definition of intelligence, is more intelligent than the spiders setting up webs in the corners of our human rooms. But it is not conscious. We assign consciousness to it because we have trained it to communicate with us through natural human language. But consciousness is apart from language. Seth is pointing to this short-coming of AI discourse—consciousness is not words and analytical problem-solving. Or not only these things. It may be inextricably connected to being an embodied animal.
“The very idea of conscious AI rests on the assumption that consciousness is a matter of computation.” This is why Seth goes into detail regarding how brains are not like computers, in that, among other things, they don’t have a clear software/hardware distinction. I’m inclined to agree.
Seth offers a series of at-times complex arguments for why our consciousness is inextricably connected to our biology, i.e. how our brains operate and our embodied nature. He says “The minimization of prediction error in real brains and real bodies is a continuous dynamical process that is likely inseparable from its material basis, rather than a meat-implemented algorithm existing in a pristine universe of symbol and sequence.”
Therefore, looking at the brain through “Turing glasses” underestimates its biological richness and overestimates the substrate flexibility of what it does.” He also says, in a pithy formulation, that “the brain is not a Turing machine made of meat.”
Let Seth speak in conclusion: “Put all this together, and a picture begins to form: We experience the world around us and ourselves within it — with, through and because of our living bodies. Perhaps it is life, rather than information processing, that breathes fire into the equations of experience.”
Now for some consciousness humility. We should not try to create conscious AI. As Seth rightly stresses, this would be an “ethical disaster.” We would then have to figure out if they are conscious (hard to do) and if they have rights. This in turn might limit our ethical ability to control or shut them off if they become dangerous. One clear problem, as we’ve stressed above, is that “there aren’t any definitive tests for deciding whether an AI is actually conscious.”
We need a heavy dose of the humility and uncertainty regarding consciousness that Seth displays. As he says, “anyone who claims to know for sure what it would take to create real artificial consciousness, or for sure what it would take to avoid doing so, is overstepping what can reasonably be said.”
Along these lines, back away from the AI hype and consider the fact that our world is full of conscious non-human creatures. They are called animals. There are billions of them. Some, like our beloved pets, we treat with respect and adopt into our families. But many, through human development practices, we treat with indifference, destroying entire ecosystems and leading to the extinction of countless species. Others we treat with absolute barbarity, namely the many animals caught up in the global machine of industrial farming.
In a world in which we are mostly failing at our obligations to respect the lives of conscious animals it is the height of arrogance and unconcern to attempt to bring into existence new, poorly understood, digital creatures that are probably not, but possibly could become, conscious. I tackled this topic in an earlier essay titled “Against AI.”
Nathan J Robinson offers some helpful reminders to close out this section— “AI can act in ways that resemble human actions. Just as both a calculator and a brain can add sums, AI can produce (usually atrocious) poetry or (sometimes rather good) instructions or advice. It can solve fiendishly complicated math problems and write computer code. But an equation is not an organism, it has no body, and as Peter Godfrey-Smith notes, a complex mathematical system “can be informationally extraordinary, cognitively extraordinary, without having the beginnings of an inner life.””
The political theorist Yuval Levin notes in a perceptive essay that “today’s AI can interact with human beings in our own languages with seemingly total fluency. By absorbing all available articulated human knowledge and discerning patterns in the data that it can extend, it has essentially learned to speak human.” Yes, and that is remarkable. But this doesn’t mean it is conscious.
Levin can hardly be criticized for down-playing AI. In the same essay he argues that “the extraordinary, almost magical, process of absorbing all of human expression into a statistical engine has unleashed natural-language computing, and the world will never be the same.” He is right with that phrasing—we are now in a new world of “natural language computing.” This is different from what has come before. But he is also right to say “Superhuman programming ability does not by itself amount to, or even point toward, anything that might reasonably be called general intelligence,” let alone consciousness and the unbelievably rich world of experience that it comprises.
Let’s take that as a starting point for the next section—the incredible complexity and abundance of experience that being a human entails.
On Being Human
This final section will briefly reflect on what it means to be human and what might be lost in a world inundated with convenient (not to mention possibly super-intelligent) AI systems.
Let me start with an image that is different from the Terminator dystopia. It’s the dystopia depicted in the movie Wall-E. This dystopia is quite distinct from the vision of doom depicted in The Terminator or The Matrix. People, after all, are alive and well in Wall-E. Sort of.
With every need sated, humans get carted around on floating couches, stuffing their faces with food and watching endless content on screens in front of them. We are doing this while traveling the stars on automated spaceships because we have wrecked planet Earth. This image, though quite distinct from future warriors heroically fighting for survival against evil machines, nevertheless horrifies viewers. It is an image in which, although we have not physically died, humanity has for all intents and purposes come to an end. There will be no more Shakespeares in the Wall-E future, no more brilliant teams of scientists discovering new truths about the world. Not to mention, no human listening in awe to the sounds of a thunder storm on a humid summer day, or quietly reading in a sunny chair with a cat curled in their lap. Humans don’t really do anything at all in this Wall-E future.
It is a world in which AI convenience has snuffed out human aspiration and meaning. This is a real concern as well, seen in the words of Altman or Zuckerberg depicting AI assistants doing everything for us. In Sarah O’Connor’s words, “just as generative AI has already tempted some people to take intellectual shortcuts, chatbots might tempt people to retreat into effortless comfortable relationships with non-judgmental, sycophantic, always-available chatbots, which are primed for engagement rather than challenge, and cannot feel empathy or love, but can do a pretty good job of pretending that they do.” (209).
Yes, exactly. Ever-available, sycophantic virtual assistants would not kill us physically but they would threaten to undermine what it means to be human. For they are neither human themselves nor helpful at us being human. A washing machine helps me be human, by freeing up time doing rote chores so that I can pursue projects that matter to me. An LLM that writes my wedding vows, on the other hand, is taking away part of my humanity. Yet it feels like the latter future that threatens on the horizon.
O’Connor notes, and I would expand on this point, that part of being an adult human is to do things ourselves. It means to engage in and pursue projects that matter to us, from relationships and parenting to art, to exercise, to gardening, or just taking in and feeling the beauty of nature.
She mentions that humans tell jokes, experience empathy, produce amateur writing, fall in love, cultivate friendships, produce, share, and experience art, and so on, “and our brains do it all with such elegant efficiency, without the need for vast data centers and their hungry energy demands” (223). She says as humans we must “rediscover a sense of confidence and wonder in ourselves.”
O’Connor worries our tech overlords show us a vision of “the thin sort of humans we might become if we allow these tools and their makers to change the words we use, the tastes we have, and the expectations we hold, both of ourselves and of one another. The goal might be to make machines in our image. But what I fear is that—perhaps without even quite noticing—we remake ourselves in theirs.” (224). Yes, the last thing you should want is to emulate Altman, Musk, Thiel, Andreessen, and Zuckerberg in their weird, shallow understanding of humanity. Yes, they agree with the rest of us that a Terminator-style future is bad. But they seem to think the movie Wall-E is utopian. Steve Bannon, of all people, rightly mocked their view of humanity, calling them weird sci-fi nerds.
Now there is nothing wrong with loving science fiction or being a nerd, that’s for sure. But Bannon was onto something in recognizing that our tech overlords have an odd, shallow, and frankly sad view of humanity. They frequently confuse two distinct things—automation that helps us with projects, like a Roomba cleaning the floor for me, and AI models that would replace us. Not just at work but at home. The Obsoleting Project, as Garrison Lovely calls it, not only threatens to destroy your job but to make your very life obsolete. If AI writes your texts, your emails, picks out presents for you to buy, and so forth, at some point you really have no reason to exist. You are effectively ceding your life, letting a computer program live it for you.
The fundamental human experience, to choose and pursue relationships and projects that matter to you, and that are worked on and completed by you, would disappear, and with it any reason for us to be alive at all.
Naomi Klein and Astra Taylor also attempt to capture some of this wonder of human life in their work. As they put it, “to experience awe is to dissolve into the swirl of the universe, to feel small in the best way. This sense of connection is also why we grieve when we witness strangers suffering, and feel the great sorrow of extinction. The fact that this capacity seems to be missing from the men who hold our collective fate in their hands is a personal loss for them—and an existential crisis for the rest of us.” (247).
To be an embodied human in this universe, to experience love, heartbreak, beauty, pain, a fleeting sunset, a stunning line in a song, these are experiences you cannot outsource. You either have them or you don’t, and the same is true, if to a less complex degree, for all animal life.
Given the richness and beauty of life at its best, why would people turn to shallow approximations of it? Why are people becoming “friends” with chat bots or developing romantic feelings for computer programs?
Garrison Lovely helpfully observes that “at a time of widespread and crippling loneliness, OpenAI created an on-demand, endlessly patient and affirming friend and confidant.” (Lovely, p. 74). In our era of anxiety, hyper-individualism, fractured institutions, and neoliberal capitalist breakdown, it should not be surprising that people want and need companionship more than ever. (I have struggled with a midlife crisis related to loneliness). But chatting with LLMs only addresses the symptoms, not the disease. And as many fear, talking to an endlessly agreeable learning algorithm is not analogous of the give and take, the joys and challenges, of real relationships with real human beings. It is not a replacement but rather a turn to a different, much shallower type of relationship. Romantic partnerships, friendships, and family relations are hard. Welcome to being a human. They are also much richer and more meaningful in part due to this fact, as any parent could tell you.
Yuval Levin makes an interesting point regarding the human experience and efforts to replace humans at work. It is at least possible that the richness, subtlety, and tacit knowledge that deeply experienced people develop at work cannot be effectively replaced by AI. There are, after all, forms of subtle and informal knowledge that are not easily translated into formal statements. “Simply put, today’s AI models can do much of what novices can do in these fields, but they cannot learn much of what novices can learn. If the ability of novices to apprehend informal knowledge in fact cannot be replaced, then replacing them with AI stands to profoundly damage the professions involved, by breaking the continuity that makes it possible to sustain and advance their most difficult work.” Levin is not saying our work cannot be replaced by AI. He is rather making a point that echoes some of my own thoughts—replacing the most experienced humans with AI might involve losses of knowledge and insight that are real but hard to quantify. I won’t delve more into this point here but I think it is worth reading his essay in full. (Find it at https://www.thenewatlantis.com/publications/are-we-really-on-the-path-to-agi).
Returning to that question of obsolescence, Garrison Lovely calls attention to the vile transhumanist ideas espoused by many of the AI overlords. Not just abundance, or replacing our labor, but replacing us. For instance, “Google cofounder Larry Page reportedly thinks machine superintelligence is just the next step in evolution” and has also called it “speciesist” to stress the importance of human consciousness (97). DeepMind researcher Richard Sutton has said “we should not resist succession,” i.e. being overtaken and replaced by AI. Perhaps most famous of all, Peter Thiel could not answer clearly whether he wants humanity to continue to exist when asked point blank by Ross Douthat.
Lovely is right to describe this as a project to make us obsolete. And we feel it in a loss of control—loss of control over our political system to oligarchs, loss of control of our jobs to advanced AI, loss of control of our very humanity to digital AI “assistants,” and perhaps, eventually, to “superintelligence.”
In a piece for Current Affairs Nathan Robinson discusses this need for us to do things, not pass over our lives to some weird virtual butler while living in a world where we feel beset by giant forces beyond our control. “But I think people’s objections to data centers go beyond the noise they make and the electricity they use; they associate data centers with the move toward a world in which everything is more confusing and our lives are governed by forces we cannot understand or control. AI is a horrible escalation of our drift into a world where we are just constantly being lied to and manipulated by people who want to squeeze a profit out of us.”
(From Nathan J. Robinson, “Mamdani’s “Richard Scarry Socialism” Can Make City Life Simple Again”, Current Affairs, October 1, 2026). Well said.
For more thoughts on these topics I recommend More Everything Forever by Adam Becker, Empire of AI by Karen Hao, A World Appears by Michael Pollan, and God, Human, Animal, Machine by Meghan O’Gieblyn.
I intend to do a follow-up essay in the coming weeks that draws on the book Being a Human by Charles Foster and Wendell Berry’s essay collection, What Are People For?