Neither Human nor Nonhuman
Klara, the first-person voice of Kazuo Ishiguro’s novel, Klara and the Sun, is an Artificial Friend, a creation designed to provide friendship to children in a society that has limited contacts between people. So, what is she? People in the novel wonder whether they should greet her, talk to her, treat her as another person in the room or as a machine. We, the readers, know she is no mere machine. Yet we also know, and can never forget, that she is not like us. Her vision is different—she sees in boxes, with reality segmented into different spaces. She doesn’t use pronouns in the normal way; speaking to Josie, for instance, she might say “I think Josie is afraid” rather than “I think you are afraid.”
She is fantastically smart; she is often called exceptional. When I see a stranger, I might estimate him to be is in his 30s (for instance). Klara would say he was about 37 years old.
Most significant to me was her ability to identify emotions simply from looking at people as they spoke to one another—which she could do even when she was not able to hear what they were saying. She detects, for instance, that someone is unhappy even though the person is smiling. Although she doesn’t understand this from the inside as we do; she doesn’t think, “I’ve sometimes been unhappy and yet put a smile on my face.” Yet she is able to identify that it is happening.
I think the author is trying to make us see that androids (a term not used in the book) are exceeding strange, neither human nor nonhuman. They are not animals, they are not human, they are not programs running on a body-shaped computer; they are like nothing that we already experience in the world. If we were to create AI-beings, robots who have the ability to learn and to adjust themselves to reality, we would not be creating not substitute humans, nor would our creations necessarily be monsters or absurdities; they would be something strangely new. It seems to me likely that they would use intelligence to learn how to interact with us as if they had feelings as we do. They might say, as Klara does at one point, “I fear that . . .”
I felt compassion for Klara; I didn’t want her to be destroyed, and her ultimate fate—described as a “slow fade”—made me sad, even though it did not make her such. She feels in the end that she has had a good life. And I suppose, for whatever it is that she is, that is true.
—
Two important questions then follow.
1) If we create AI-beings sometime in the future, how should we treat them? We cannot treat them as machines. We cannot treat them, even, as other nonhuman animals, if for no other reason than their use of language would set them above pets, cattle, or wild beasts. While they won’t be human, they must be treated “humanely.” What would that mean? Among other things, I think it would mean that we should see them as potential friends. But what would that mean?
2) Should we even start down this path? I think that to make an AI-being is to manufacture a new form of life. Is that something we should hold back from doing, even if we could do it and even if it would bring benefits?
If you haven’t read it, Klara and the Sun will set you up for seeing the importance and the difficulty of these questions.
—
Out & About: Sunday, June 16, I am to preach at St. Matthew’s Cathedral, Dallas; the eucharists are at 9 and 11:15 a.m.
The next Good Books & Good Talk seminar will be on Sunday, September 22, on Aldous Huxley’s Brave New World.
I am grateful to the participants in last Sunday's seminar on Klara and the Sun. As often happens, by the end of the seminar I had learned more about the book, and that helped shape what I have written above.
—
On the Web: In response to a line in last week’s blog (“As they say, ‘This too shall pass’”), a friend sent me this unbelievable link: https://youtu.be/qybUFnY7Y8w?si=NNnrclQEVLH0xEG_ ... I was glad at the end to find State Farm was there, supporting the endeavor.