Scientific Russia: Homo semioticus, a person who generates meanings

Philip Ball, an English scientist and science writer, is sure that it is imagination that distinguishes us from animals. Imagination is the ability of the mind to represent and describe not only those things that we have never encountered but also those that we will never be able to experience, because they are simply impossible. Do you agree with this statement?

I cannot give you a clear-cut answer. On the one hand, based on what I know, it looks like Mr Ball is right. But, on the other hand, we cannot say for sure what is going on in the head of another creature, because, as of today, there is simply no way to find it out. We can only assume that the chicken has a small brain and therefore not very complex. So, the chicken is hardly imaginative. But the reality is that we are not in a position to confirm or deny this.

Moreover, we do not know anything about each other. I cannot get into your head and find out what you are thinking about. On the other hand, people have many ways to show it. Creativity, science, writing and a set of symbolic systems give us the tools with which other people can understand us.

And what about animals? Only their behaviour can tell us something. However, it is impossible to understand what a cat thinks when it throws at its owner a stern and gloomy look. It is quite possible that serious mental structures are being formed in its head.

You mentioned creativity and I remembered animals drawing.

That is it! Everything depends on interpretations. Why did I start so carefully? That was not for provocative reasons. What happens when an animal is drawing? Is it its paw randomly moving across the sheet of paper or does it have an idea? There is no way to find it out.

On the other hand, we can give examples of the playful behaviour of such creatures as, say, corvids. The game is considered an indicator of intelligence since it has no other function than entertainment. One of the entertainments of crows is rooftop coasting − and that looks quite suspicious! The question inevitably arises: why does the crow do that? Experts will surely say that this is a serious marker of the presence of intelligence.

From an evolutionary standpoint, how important can imagination be to humans?

I think it is very important. It seems to me that the main feature of human being as a species lies in the ability to live not only in the material world, but also in the world, which can, half in earnest, half in jest, be called a human-created world.

It is widely believed that the brain is a very complex device that is engaged in information processing. And this, of course, is true, no one will argue with that. But we must not forget that the brain does not just process information but also creates it. The brain is a creator. And it is not so much about material objects such as forks, spoons, rockets, and so on, but about philosophy, religion and art. This is a huge non-material world in which we live. The question therefore arises: what is more important for a human being? The material world consisting of tables, chairs, steaks and cars, or the mental, spiritual world, if you prefer − the one that does not break down into molecules and atoms? The mental world that is being created by ourselves.

Remember the human history. Wars, cataclysms and incredible social events happened not because there was a shortage of bread or land. Most often, the reason was in the mental worlds colliding because they could not get along with each other. This is our human nature.

What is your personal answer to this question? Which world is important, the material or the spiritual one?

I would not put the question so sharply. It is clear that if you have nothing to eat, you will quickly remember which world is important. I am only urging that we should not underestimate the worlds we create, the second important ability of our brain. Unfortunately, most ordinary people believe that the main thing in life is an omelette, a steak and a cabbage pie. And everything else is only additional attributes. It is fine if there is a museum, but if there is none, it is okay, too − we’ll do without it. There is a decrease in the general value level of the human civilisation, which considers the elements of culture as something decorative, optional, even entertaining. And this is a very dangerous signal.

Sounds a little scary. I recall the words of biologist Alexander Markov, who said in an interview that culture is the driving force of evolution.

Alexander Markov is a remarkable scientist. I know him, I read his wonderful books. You can rarely hear such an opinion from a biologist. Meanwhile, we know that the neural network of the brain is constantly being built in our head. While we were talking, it had already deformed several times and built new connections. And if we look at this process from the point of view of epigenetics, we can say as follows: what the brain is doing today shapes what the brain of our descendants will be like.

Can we think that the imagination is given to us to picture our future in order to plan our present? By imagining different objects, beings and states in our head, we seem to be always ready for anything. It is like a permanent defensive reaction.

Let me take a step back and return to our first question about animals and imagination. To be honest, I cannot imagine how animals other than humans would have survived if they did not have this ability at all.

There is such a thing as probabilistic forecasting, an important aspect in ensuring life. And if a living being does not have any forecast for the near future, it simply will not survive. For example, if a wolf stepped on an unextinguished ember and burned its paw, it is beyond dispute that it is unlikely to step on such an object again. I do not want to reduce everything to respondent conditioning, but still, all living beings do have a certain level of imagination and prediction.

Where does the imagination "live"? How is it related to memory and prediction?

A good question that leads to more general features of the brain organisation. Of course, there are zones that are specific to some activity, responsible for processing complex visual images, for auditory perception, etc. That is confirmed by medical cases. If the continuity of one or another area of the brain is disrupted as a result of injury or disease, we immediately see a negative result in a certain part of the body or in some process. It is therefore obvious that specialisation and localisation do exist.

But if we go further and ask where the memory or associative thinking are situated in a human being, the answer will be: everywhere. It is clear that if the hippocampus gets damaged, there will be no memory. It is a fact. But, on the other hand, my fingers remember, for example, what the fabric feels like or what happens when you accidentally cut them. That is, the body memory also exists. Of course, you can say that sensory processes take place in the brain either way. Yet, I deliberately gave a relatively simple example to show that memory can be anywhere.

Additionally, the more complex our brains are, the more associative our memory is. For example, the memory of a book may be associated with the memory of how you were drinking coffee from a beautiful cup and at that time a butterfly flew in through the window. Each of these details is connected with the others for good. Memory localisation does therefore exist, yet there are also places, sort of cognitive mental maps, that can be very strongly distributed throughout the brain.

A symposium dedicated to neurosciences has been held in Siberia recently. While speaking there, you mentioned Homo semioticus, a person who generates meanings. When did they appear? Can the whole society be included in this category?

That is a good question. Of course, I would like to bring the entire human civilisation under this category. But it is obvious that there are many people in the world who, figuratively speaking, do nothing but eat hamburgers. And I am not sure that they can be considered as "meaning-generating people".

If we consider this issue from a scientific point of view, are we the only creatures on this planet who deal with signs? The scientific community of biosemiotics, for example, believes that we humans think too much of ourselves and that all sorts of sign behaviour is characteristic of many animals, if not all. To be honest, I do not think that it is typical for everyone, but it is a big question where to draw this boundary. Take bees for example. Bees have a rather complex behaviour and not all of its elements can be merely reduced to programmes embedded in genetics.

I think that semiotic behaviour emerged with the beginning of human civilisation. Although here we are confronted with a question: what should be considered as the beginning? If we take Neanderthals and Denisovans, whose genetic material we received in different proportions, I arrogantly told my students 20 years ago that Neanderthals are a dead end branch of evolution, bastards not worth talking about.

Today, the ideas about our ancestors are completely different, because, for example, we now know that they had art. We know about drawings in their caves. I have been to Denisova Cave twice, and what they found there is simply amazing (for example, adornments). You involuntarily ask questions: firstly, how did they do it, and secondly, why? The second question is even more important. In any case, objects of art speak of a highly developed consciousness. When and why the semiotic behaviour originated is therefore an open question. But the fact that this boundary is shifting further and further back is indisputable. Now for 40,000 years, and in the future perhaps for 200,000 years, depending on what is found.

If you look at the drawings of ancient people in different parts of the Earth, it is clear that they were made by true artists. This makes it obvious that we were arrogant to and misjudged those who had lived before us. They are not so primitive at all.

For example, archaeologists have discovered an old bone flute estimated to be 40,000 years old. The question immediately arose: is this really a musical instrument or are these random holes in the bone that seem to us like a flute? Experts believe that this really is a musical instrument that demonstrates the high symbolic development of ancient people.

So my answer is that humans are semiotic beings. We live in a world of signs. Signs are not only language, but also music, art, and so on. Signs are mathematics, the language of God, as some serious scientists say. In answering your question, I will therefore dare ask mine: is human mathematics constructed like that because our brain is like that, or is mathematics in fact the perfect language of the Universe and our brain was able to understand something in it? The same with music. Music − what is it? Is it something about our ears and brain? This is not just a sound with its frequencies, decibels, etc. Physical characteristics are easy to fix with devices, but music becomes music when it enters our brain. Both music and mathematics, as well as any other semiotic knowledge, are therefore decoded in a prepared brain. Not in the ears and not in the eyes, but in the brain.

Speaking of thinking, it is impossible not to mention language. Why did language become a reflection and the main element of the mental process?

Some linguists and philosophers believe that language is not only and perhaps even not so much a means of communication as a means of thinking. It is with the help of language that we "put things right" in the world. There are simply no other tools. Or rather, there are, but they are even more difficult. Like mathematics, music and other sign systems. But language brings categorisation to the world, it enables you to classify many different objects and phenomena. In simple words, language helps us deal with the chaos of sensory signals that hit our brains every millisecond.

It is also interesting that there are now about 7,000 languages on the planet. I say "about" because it all depends on what is considered a language and what is considered a dialect. But, whatever one may say, there are about 6,000 or 7,000 languages in the world. At the same time, languages disappear, and this is a very big loss because each language is a different world. Languages organise reality in different ways. Each of them is therefore important.

Say, in one of the northern languages there are about 500 names of different types of snow, but the word ’snow’ does not exist. For example, there is slightly melted snow, sunlit snow, snow which a partridge has run upon. And there is no common word for all these types of snow. That demonstrates how significant is the role played by language in shaping the thinking of a particular ethnic group that speaks it.

Humanity sees the world through different eyes, so it is so important to know not just each other’s language but also the mentality. You simply cannot agree on anything with native speakers of, say, Chinese if you do not know anything about Chinese culture. You will encounter a profanation of communication. It will seem to the parties in a dialogue that they are talking about the same thing, but in fact each of them will mean something completely different. Language is therefore a powerful tool but also a very dangerous one. No wonder people engaged in duels when one had called the other in a way that was not accepted in this society. That is what language is. Not to mention that with the help of language we create great art. Joseph Brodsky in his Nobel speech said: 'Poetry is the goal of our species.’ I was dumbfounded when I heard this. A species-related, biological goal? Poetry? Joseph Brodsky explained that poetry was a colossal accelerator of consciousness, a cognitive tool. A true poet is not the one who knows rhymes. "Roses-noses-proposes" have nothing to do with poetry. A true, heaven-born poet sees in the world what others do not see. Poetry is a different comprehension of the world. Brodsky’s words are a striking statement. He is not a researcher, so how did he come up with that? In one of his essays, Brodsky mentioned that it is the language that creates the poet, and not vice versa.

Terrence Deacon, a remarkable scientist, once wrote something that made everyone shudder: ’Language is a parasite that has taken over the brain.’ These two examples perfectly illustrate the situation when the same things, only in different ways, were described by a greatest poet and a neurophysiologist.

By the way, of all the sciences that I practice, linguistics is one of the most difficult. Yet, despite the huge variety of languages, they have basic universal algorithms with which they function. All languages have a subject and an object, there are constructions that denote certain objects, and there is something that denotes processes, and so on. It seems that language algorithms and mechanisms are something innate and, therefore, genetic. No wonder scientists have spent years searching for the language gene. Today, of course, no one will seriously talk about that. Such a complex thing as a language cannot be provided by a single gene, that is, the very idea of searching for a language gene is meaningless. But this does not mean that there are no genetic foundations of the language!

You have anticipated my question about the language gene. Is the idea of its existence that improvable?

Yes and no. Most often, we are talking about the sensational discovery of the FOXP2 gene, which was immediately declared the language gene. And in some publications it was even called the grammar gene. Of course, these studies were not carried out in a vacuum. Scientists have indeed found breakdowns in this gene in members of several families where speech disorders were observed. It is therefore pointless to dispute the obtained results.

But it is one thing when we say that a gene "is related" to language and another thing − that it indeed ’is’ a language gene. The second statement is, of course, wrong, and the first one needs to be clarified.

It seems that FOXP2 is a kind of hub that ensures the proper development of neurons, axons, dendrites in the part of the brain that is responsible for the proper functioning of the language. This is evidenced by an interesting and elegant work published several years ago.

The fact is that FOXP2 is also found in animals that show no sign of language communication, including crocodiles. But there is a purely human version of this gene that differs from the animal genes of the same name by two amino acids. So, through most subtle genetic manipulations, scientists have transplanted the human version of this gene into laboratory mice. Of course, the test animals did not start quoting Shakespeare, but their vocal spectrum of signals became much wider and more varied. Mice have indeed become very "chatty". The FOXP2 gene, of course, is therefore related to language, but it does not define it exactly.

In the context of our conversation, I cannot but ask you about artificial intelligence. In one of the interviews, you mentioned a friend who turns on the light for her robovac.

I was shocked when I found it out. This is very interesting because I caught myself at that. I have turned on the TV recently and the movie titled Bicentennial Man was on. It is about a robot that was gradually becoming more and more humanoid. Since he did not sleep, they left him in the basement for the night and turned off the light. Then I thought, ’How is he there without light?’

In fact, the answer brings us to empathy and the so-called theory of mind, i.e. the ability to put oneself in the position of another being. Therefore, when we turn on the light for the vacuum cleaner, it rather characterises ourselves.

I had a similar experience myself. One of the developers of neural networks told me that when a neural network is learning, it makes mistakes and is sort of punished. When he said that, I involuntarily experienced a certain act of empathy: a pity for an inanimate being. It amazes me that we endow inanimate entities with feelings.

There is a brilliant linguist and philosopher Jerry Fodor. He is a flamboyant hooligan scholar who is constantly drawn to various intellectual provocations. By the way, he came to St Petersburg and I had a chance to talk to him. He has a great scientific article called Why Pigs Don’t Have Wings? This paper ends wonderfully. Answering the question posed in the title, Fodor writes, ’Because they are pigs.’ In another, no less interesting article, the author writes something that you will never find in scholarly publications, ’Here I am sitting and writing an article. A robovac is riding around me as if it were alive. My grandchildren feed it cookies. It is eating. And I think: does it have mental representations? I opened it and saw that there were no mental representations but only dog hair and cookies that my grandchildren had fed it.’

This yet another provocation indicates that artificial intelligence can already behave as if it really had processes of a higher order. When we treat a programme with sympathy (and I understand that very well), it therefore only shows how strong the desire to revive the inanimate is in us. We understand that the programme is unlikely to suffer from ’penalties’. At least for now. But this leads us to very complex and traumatic thoughts. After all, these artificial entities will learn to imitate human emotions and behaviour. And we will seriously think that they are suffering. Although in fact it will be just a virtuoso imitation. What should we do next with that? What is to be done? It is a challenge.

Of course, developers from Skolkovo or Silicon Valley may object, ’We will really teach them to feel and express emotions.’ Then I will ask, ’How can you prove it?’ After all, it cannot be verified. Naturally, I do not expect an answer to my question. Because it is a really complicated story.

It gets even more complicated when we talk about art and culture. Neural networks have learned to draw pictures and create music based on what a person likes. Art has transformed into the format of sensational NFTs. Where are we going? Will there be anything real? Or is it worth just digitising the works of Alexander Pushkin and Leo Tolstoy so that neural networks give us a new novel every week?

This is a very difficult question. When artificial intelligence first creates a painting that is sold at an auction for a lot of money, then, of course, we are talking about a unique experiment, a work that was not created by a human being. But if that becomes mainstream, like, today "Chekhov" gives me a new play, and tomorrow let "Vivaldi" create new pieces, then everything is perceived differently. As far as I know, Beethoven’s symphony has already been completed by neural networks. I have only one question in my head: why?

The robots have already beaten all the grandmasters in chess, defeated everyone in Go and poker. What is next? Are we going to leave ourselves some place to live? Or will we continue to play this game of ’who else and how can surpass us?’ Some sort of suicide game. As for art, I want to look at the real Albrecht Dürer. But this is really my personal statement. Some people like oysters while others like hamburgers, and others do not care what they eat at all. I tend to the fact that we live in a very interesting period in the development of civilisation. We are now facing a need to somehow rethink the whole world, to find a place for ourselves in it. To agree in advance on the ethical and legal restrictions on the functioning of artificial intelligence. This is important in the context of the fact that in some countries particular species of animals are recognised as biological legal entities (nonhuman subjects). Among them are chimpanzees and elephants. Dolphins and corvids are likely to be also granted this status.

We have come to a stage when we all need to somehow find a different position in this world, where, in addition to humans and animals, another player has appeared − and that is artificial intelligence. All these questions cannot be ignored. Sooner or later, these problems will overtake us, even if we shyly turn away now.

The turning point is near. I hope we will not end in complete self-destruction which for some reason we are currently doing, that surprising me very much. Are people not afraid? We can all disappear, isn’t it a pity?

Well, it is an interesting time we live in. A time of reflection that makes you think about yourself. To do this, we have mirror neurons, which provide the aforementioned theory of mind, that is, an important ability to put oneself in the position of another being.