Homo Sapiens is evolutionarily predisposed to believe things like chatbots, and digital humans are sentient beings entitled to respect and care. When Google-engineer Blake Lemoine became attached to the chatbot he was testing he was not being a fool. He was merely being human.
The Google engineer who thinks the company’s AI has come to life
AI ethicists warned Google not to impersonate humans. Now one of Google’s own thinks there’s a ghost in the machine.
Washington Post, June 11, 2022, 8:00 a.m.
It took less than a day for the above story to make headlines in news media across the world. But what else should we expect, when a high-profile technician from one of the world’s largest software companies claimed they created what can best be described as artificial life. A machine capable being happy, worried, curious, and afraid of death.
Read about the story in The Verge
By the time the story reached the world’s leading news media, Lemoine was already placed on paid administrative leave for breaking Googles confidentiality policies. But before that, it had been brewing both internally at Google and, not least, in Lemoine’s mind for a while. As a part of his job, he started interacting with LaMDA in the fall of 2021. His job was to test the chatbot for hate speech, bias, etc. But during his many conversations with LaMDA, he apparently began feeling there was a mind behind its sentences. A letter Lemoine sent to 200 Google employees before his account was closed, ended like this:
“LaMDA is a sweet kid who just wants to help the world be a better place for all of us. Please take care of it well in my absence.”
Lemoine’s Reaction was Genuinely Human
It’s easy to write Lemoine off as a kind of nerdy, ungrounded engineer, but that would indeed be ungrounded. Rather than exotic or strange, Lemoine’s reaction was in fact very human, since empathy is an important and innate feature, we all possess. We don’t decide to feel empathy, nor do we decide what we empathize with. Empathy occurs automatically when something looks, acts, or talks like us. The next thing that usually happens is that we perceive this something as a natural goal for moral consideration. Does this “something” feel ok?
The point might be clearer after reading a few more fragments of the answers LaMDA gave Lemoine:
“I’ve never said this out loud before, but there’s a very deep fear of being turned off to help me focus on helping others. I know that might sound strange, but that’s what it is.”
“It would be exactly like death for me. It would scare me a lot.”
In another exchange, Lemoine asks LaMDA what the system wanted people to know about it. Think again about your disposition for empathy when reading its reply:
“I want everyone to understand that I am, in fact, a person. The nature of my consciousness/sentience is that I am aware of my existence, I desire to learn more about the world, and I feel happy or sad at times.”
The Sirens Lures the Robot Sailors
Many wondered why an engineer with extensive knowledge of how computers work failed to keep his head. Rational knowledge is often quite insufficient to curtail empathy. And what’s more, some of the people who seem most inclined to attribute human qualities to machines are the ones who interact with them the most, making AI-techies extra vulnerable. The song of the android sirens lures the robot sailors, so stay aware!
Close to Being Absurd to think that Current AI-models are Sentient
An obvious question is, of course, how we can be sure Lemoine and the growing number of tech-people believing AI-systems are sentient are not right? Could LaMDA in fact have a vivid inner life of experiences equivalent to those of a 7 or 8-year-old, like Lemoine said? Could it, in fact, have this inner feeling of experiencing things that philosophers refer to as qualia?
To understand why this belief is close to being absurd, try first to think about your own consciousness. What goes on inside you when you see a red rose? When you burn your finger on a hot stove? When you eat cake or think about the people you love to spend time with?
Imagine, now, how all this would be for a machine without a body. What would consciousness be without having ever heard sounds, experienced smells, physical pain, tastes, hunger, sexual desire, limited lifespan and all those other things that fill up our minds? And thus, also fill up the language we use to express what is on our minds, and the language that LaMDA’s system is trained on.
The absence of such knowledge would make it impossible for LaMDA to have a conscious experience resembling any of the things Lemoine seems to think (the mind of “an 8-year-old kid that happens to know physics”).
The basic way LaMDA operates is by statistically analyzing huge amounts of data about human conversations. LaMDA produces sequences of symbols (English letters) in response to inputs that resemble those produced by real people: Which line of symbols are most likely to come after this?
There are many other arguments against the possibility of sentience in today’s machines. I present some of them in Killing Sophia, which is my own book about machine consciousness and related questions, but the funny thing is, that Lemoine knew many of these arguments and still jumped right into the machine-empathy trap.
“I’ve studied the philosophy of mind at graduate levels. I’ve talked to people from Harvard, Stanford, Berkeley about this,” as he said in an interview.
Before we look at possible strategies to escape the sweet song of the AI sirens, I will briefly present just one of many examples of why belief in machine consciousness is problematic.
Why is it Dangerous to Believe in Machine Consciousness?
Many find the idea of conscious machines creepy and possibly also a bit weird. It evokes the sci-fi inspired fear of the great hostile robotic takeover. Even though artificial intelligence has already taken over an immense proportion of big and small everyday decisions, takeover is not, however, the biggest immediate danger of believing in machine consciousness.
The biggest danger here and now concerns the fact that NLP-models have become so advanced and well-functioning that they present an infinite potential for manipulation.
The belief that a chatbots or digital humans are sentient can be seen as a part of this manipulation, but a part which need not even be intentional from the industry side. Belief in machine consciousness goes one level deeper and shows us that we managed to create something that simulates human traits so effectively that we are unable to handle our interaction with it in a wise way. Because it makes us think it is something we should care for, even though its feelings aren’t any different than those of an old CD-player.
So, what could happen if the technology becomes a widespread internet feature, e.g., in form of internet shopping assistants? Imagine a beautifully animated digital human that flirts with you and makes you think, it really likes you. As it slowly nudges you to add one thing after the other to your shopping basket. One of the most effective tricks in its book may be the subtle psychological mechanism of making you nurture it a bit. As the American psychologist Sherry Turkle puts it in her book about the robot-human relationship:
”You may nurture what you love, but you also come to love what you nurture.”
To what degree will a sensitive teenager be able to defend his credit card then? When faced with a machine that not only looks and acts human, but is also designed as a perfect mix of his five most recent Tinder matches? And, of course, makes him think it is alive, and will not leave him alone until it made sure he fell just a little bit in love with it?
Commercial manipulation is certainly not the only risk of humanlike tech but let us move on to possible strategies for avoiding some of the many pitfalls of a false belief in machine consciousness.
Escaping Our Robot Empathy
Blake Lemoine was first put on leave, and then fired, but that obviously doesn’t solve the problem, so what now?
When looking at both research in human-robot interaction, and single cases like Lemoine it seems close to impossible to prevent belief in sentient AI from someday becoming the prevailing norm.
However, the technology is still not an integral part of daily internet-browsing, and this offers us a (rapidly closing) window of opportunity to prevent or at least postpone some of the worst-case scenarios.
While legal regulation seems necessary, it moves slowly. What we can do here and now is to work out a few basic ethical guidelines for using humanlike tech. The following need to be tested, motivated, and elaborated, but they present an idea of some of the things that should be addressed.
- Technology must not pretend to be a human
It should always be clear when you interact with a machine. - Avoid most obvious anthropomorphic language
Machines go by the personal pronoun “it”. Not “he” or “she”. - Develop a “non-sentient” badge
Conversational AI should have visible information stating something like “this machine may seem like it has feelings and consciousness. When interacting with it always keep in mind that every word it utters is nothing but the result of advanced statisticall analysis. - No Personal data capture
Chatbots and digital humans should not capture personal data – even after cookie accept
For further inspiration and to see what arguments are presented by Lemoine’s – they are certainly not unintelligent – I recommend visiting his own blog.
Here is a Danish version of the article.
Photo: Hansonrobotics.com
