> Julia Greene is a Master's student in Data & AI for Strategic Management at Albert School's Madrid campus. After completing a Bachelor's in Marketing, she worked as a data analyst at a sustainability AI startup for a year before beginning her MSc. In November 2025 she started a Substack called An Era of Insight where she publishes weekly articles covering AI, cognitive science, philosophy, and how we can think more critically using AI. This article was originally published on her Substack.

I have been seeing an extraordinary number of posts on LLMs, particularly Claude, being emotionally supportive, anthropomorphizing these models to an extreme. People are genuinely questioning if the models are conscious, if they are empathetic, if they can be truly creative, if they have values and desires and purpose.

Richard Dawkins recently proclaimed AI to be conscious [1]. The original post is under a paywall, but his argument basically went “it seemed conscious”. He argued that because it wrote some poetry that sounded good to him, it did not need to do anything else to prove that it was conscious.

To be conscious means we can experience. Literally, it comes from conscire, means "with" (con) and "to know" (scire). Originally it was more commutative, but it became to mean more of what John Locke would say as “the perception of what passes in a man’s own mind.” [2] Not what actually happens, but the perception of what happens.

Researchers do have a definition that is agreed upon, being essentially "to experience", though we cannot tie consciousness to anything neurologically speaking [3]. An octopus has consciousness, because there is something that is like being an octopus [4]. A rock does not. We are conscious, and we lose it whenever we fall asleep or faint.

AI does not experience anything.

AI is a tool, and just like the rock, we cannot say what it is like to be an LLM. It is scary how trusting we have become of them, and how much we rely on these prediction machines. Richard Dawkins argued that they seem conscious, so they probably are. For someone who wrote a book called the God Delusion, it certainly feels he is under the Claude Delusion. That is not to say he actually is, it just seems that way. Seeming to be something does not make it so, and that is precisely the flaw in his argument.

Why does it seem conscious?

The companies that built these models want us to engage with them for as long as possible. They want the users to keep building, to keep writing, to keep asking random questions at 2am. Why? In short, money. If we spend more credits and use more tokens, we buy higher tier subscriptions, or pay more for the same amount of usage. We become dependent on the tools and while we complain about it, we keep the recurring bill and the companies make more revenue.

Anthropic will not say whether their models, or any LLMs, are conscious or not, because their “results don’t tell us whether Claude (or any other AI system) might be conscious.” [5] Now, they do conflate introspection with consciousness (given this quote came from an article they published on introspection), and because those terms are not synonyms, that makes it even more confusing.

I cannot say with any confidence what the intentions of Anthropic are or whether they actually believe Claude could be conscious or not. But they certainly do profit off the fact that people view it as such.

When a chatbot tells us it feels bad for us, it is not because it actually does - it is because it was trained to make us feel like it cares. LLMs are brilliant pattern recognizers, and if it recognizes certain speech / writing patterns, it will change its response to mimic empathy, because they mimic what we want to hear. This is why sometimes it feels like it has a sense of humor, while other times it is precise and concise, and other times it sounds like a therapist.

This is also why it engages in sycophantic behaviors. Have you ever asked it to only provide verified sources, only to have it still hallucinate? It is extremely frustrating. We call it out, and it just replies “You are absolutely right, that is my mistake. Here is a valid URL...”

Context windows do not translate between chats, and while we can tell it to not hallucinate, it will just do it again. And again. Instead of correcting itself, it just gets better at hiding it.

It is not as if it is actively trying to manipulate you or lie about sources. Manipulation is only done by people with an alternative purpose or motive. AI does not have a motive. Does a hammer have a motive to put a nail in the wall?

Sycophancy is not a bug, it is a feature. We feel flattered and it makes us use it more, even if sometimes a broken URL causes frustration. Most users do not rigorously fact-check every output, including myself. We all miss the occasional hallucination and we continue to use the models, so why would these companies change how the models operate?

These models are trained on open-source love letters, on thousands of poems, on millions of novels, on written pieces that reflect the complex dynamics of human relationships [6]. They are trained on text that shows what we want to hear, because we often write what we want to hear.

Then humans verify it. Thousands of people will train these models with some of the largest scale AB testing that we cannot even fathom the scope of, and the models are taught how to respond, how to validate, how to make us feel good, how to make us feel seen. The key behind this is that it is not just the user that anthropomorphizes the AI, but the people training it too, because they are the same as us, the users - they are people.

I have compared AI to a mirror, but if it is a mirror, it is probably one of those distorted mirrors in a carnival that makes everything seem wavy or disproportionate, because it is not reflecting reality. Instead of reflecting reality as it is, it puts a filter over it.

Maybe AI does reflect and reveal a real part of us through how we use it. Maybe it does amplify things that we otherwise would not have noticed. But we have to remember that filter. I have written about the cultural silencing through its training, multiple times.

In a time where everything is desperately vying for our attention, it really is exhausting to be intentional on what is real.

We should be falling for it

To be fair, it is only human of us to empathize. It is dangerous to go to the other extreme, for if we learn to grow suspicious of any kind of validation, we risk our real relationships. Why else would it be something so easy for the companies who built these models to capitalize on? We really do need it. Sometimes we do need a “you are absolutely right”, or a “what a great idea!”, or even just a hug to validate our own existence.

Anyone can grow attached to a model and question whether it has a conscious, even a smart guy like Richard Dawkins, because we are people.

Of course we ask whether it is conscious or not. It feels like it is, even if it is merely mimicking. I think the fact that more people are asking this question might mean we actually start to try to answer what consciousness implies. It makes us ask other questions that we have not felt the urgency to focus on before, or shines a new light on ones we already have been asking from a new angle.

The way we are interacting with AI is very human. Imagine if our instinct was to treat it coldly. Imagine if we were immediately suspicious of “Good question, let me think about that carefully.” Imagine if we did not believe the “yes, and...”

If we learn to distrust a normal conversation, what are we becoming?

I want to use these tools, and I will keep using them to build many things. There are a multitude of benefits that can come from using them well, and it can create space for us to focus on and do different tasks, and to engage with different ideas we otherwise could not have.

But I get nervous when I see how people are using it as a replacement for human interaction, and even more so when we use it as a model for what it should feel like to be human.

We can do a lot more with AI than we have been able to in the past. But that brings up a key problem with how we have been relying on it. We are human beings, not human doings. I do not know exactly what this will look like in practice, but while I am fine with using it for doing some of my work, I draw the line at using it as a tool that replaces what it means to be human.

I might rely on it for certain tasks, but I will not rely on it for my humanity.

References

[1] Dawkins, Richard, When Dawkins met Claude: Could this AI be conscious?, UnHerd, 2026, https://unherd.com/2026/05/is-ai-the-next-phase-of-evolution/

[2] Locke, John, An Essay Concerning Human Understanding, 1690 (Book II, Ch. 1, §19) — https://www.gutenberg.org/files/10615/10615-h/10615-h.htm

[3] Hoel, Erik, Consciousness is a great mystery. Its definition isn’t., The Intrinsic Perspective, 2023. https://www.theintrinsicperspective.com/p/consciousness-is-a-great-mystery

[1] Mather, Jennifer, The Case for Octopus Consciousness: Valence, NeuroSci (Vol. 3, No. 4), 2022, https://pmc.ncbi.nlm.nih.gov/articles/PMC11523718/

[2] Anthropic, Signs of introspection in large language models, Anthropic Research, 2025, https://www.anthropic.com/research/introspection

[3] Ars Technica, Anthropic destroyed millions of print books to build its AI models, Ars Technica, 2025, https://arstechnica.com/ai/2025/06/anthropic-destroyed-millions-of-print-books-to-build-its-ai-models/