HARASHIMA HIROSHI

There is a mountain of words about the face. Among them the phrase "Facial Studies" must be the latest addition. By the way, how is the face seen as an interface and what does it communicate to us? Applying information science to the face, the latest field of science emerged.

From Communications Engineering to Human Communications Engineering

Originally, my specialty of study was Tsushin engineering. The general image of Tsushin would be like this. It concerns how to connect terminals such as a telephone set and how to convey signals between them. But Tsushin is called "Communications" in English. From a point of view of communications, are not the true terminals human beings rather than telephone sets? How is communications engineering able to support communications between humans? Are not these very issues important? I began to think so in about the middle 1980s.

In society today, there are communications gaps in various senses. When an information receiver happens to be far away from a sender, there is a distance gap. In order to fill in the gap, communications, that is telecommunications technology, has made progress. But the communications gap is not only that. In some cases, a recipient might be right in front of a sender, so there might be communications gaps. For instance, their communication might be impaired by language, sight, or hearing deficiencies. Moreover, there are communications gaps such as cultural differences or filial family relationships.

From such a standpoint, I tried to review conventional communications engineering once again. I also realized that it might be better to rename the research and set up the "Human Communication Group (HCG)" in society. In my own research subject, "Human Communication Media" came to be a key word, too.

The face as "Kansei" Communications Media

The study of "the face" came into existence in such a current. Come to think of it, the face plays a very important role in communications between human beings. The face is not a medium that merely conveys information. Rather, it should be called the "Kansei," or sensitivity, communications media. This concept is very useful to consider communications technology in the future.

Conventional communications have been designed only to convey logical information. However, is not its real usage rather sensory communications? For example, a pair of lovers talks for one hour. As far as communicating information is concerned, they ought to finish their conversation after arranging what time and where they will meet the next day. Why do they keep on talking for another hour or two? In this case, communications between them are at the level of sensitivity. If such is true, future communications technology should center study on sensitivity. I came to think so and started researching the subject.

Concretely speaking, it was about 1985. I was researching the videophone and considered the possibility of providing structure to communication comfortably on the videophone.

On the videophone, people see each other's face. However, they normally feel considerable resistance to showing their own faces, especially when they receive a videophone call very early in the morning, before having put on their makeup. So, they do not want to answer the videophone. Since the purpose of conventional communications was to convey messages as they are and to do it faithfully, the sleepy look had to be sent faithfully as well.

But, from a viewpoint of supporting comfortable communications, accuracy may not be necessary. Rather, people could speak more comfortably when they give better impressions to others. That is why I began to research a new type of videophone.

The basic technology of the new videophone is this. First, the sender sends a photograph of his/her own face to a recipient. Then the photograph is made to change facial expression to smile when the sender smiles. If this idea is realized, you could talk with your own favorite face. In this case, you could even talk with your face of ten years ago.

One might say that this is fraud. But if so, makeup would be a fraud, too, wouldn't it? Usually when we see a person (a woman in particular), we would not say, "You should be ashamed of yourself for wearing makeup. You should talk to me without makeup." There is a social consensus that tidying oneself is important in order to communicate with others comfortably. Then why not introduce that idea into the videophone? Maybe nothing is wrong with modifying one's real face to a face with which a person can communicate comfortably.

From "Facial Studies" to "da Vinci science"

Once I started carrying out research on facial expressions and impressions, I was more and more attracted by the face itself. I don't have to aim at developing engineering applications like the videophone. If I could do a new study named "Facial Studies", it would be more interesting. Of course, as a person belongs to the school of engineering, I understand the importance of creating something. But more than that, as a person belonging to a university, I held a long-cherished wish of opening up a new academic area on my own. Someday "The Outline of Facial Studies in 10 volumes" might be published by Iwanami Shoten Publishers. (laughs)

After I started researching "the face," the range of people with whom I associated changed. This was an unexpected fruit of my academic life. Up to then, I had associated with only associates of the school of engineering or those who studied communications technology. I have come to keep company with various kinds of people: psychologists, anthropologists, makeup artists, traditional performing artists, mask collectors, and so on. This continues to broaden my horizons remarkably.

Besides, study of the face gave me an opportunity to consider the ideal technology in the future. For example, facial studies research the living, life-sized "face." Conventional technology has dealt with the super-macro or super-micro, surprisingly leaving behind things that are right in front of us. From now on, maybe we should study living life-size things more accurately.

Furthermore, conventional technology has a logical methodology, so to say, a methodology on the basis of logic in the first place, and has done research within that scope. In other words, we have cut away other things because we could not write papers on them. Sensitivity, for example, could not be an appropriate object of science.

However, the face has no meaning without sensitivity. Even if we bring the face into conventional methodology by decomposing the face into elements and converting them into numerical value and then analyzing them, it is not a true nature of the face. One can not at all handle the face by ignoring the side of sensitivity. In a sense, facial studies are good experimental fields to research sensitivity, which is the weakest subject in conventional science.

In addition, facial studies are a typical interdisciplinary science. As I said before, scholars in various fields are interested in the face: psychologists, anthropologists, anatomists, and so on. There are dentists, too. Philosophers have argued, "why do humans care about the face?" Face studies have come into existence by joint work of researchers of such various fields for the first time. Technology from now on will be unable to achieve anything if it locks itself up in a narrow specialty. That is why interdisciplinary sciences such as facial studies are requested. I call it "da Vinci science" in honor of that great Renaissance mind.

To the Science Opened up to the Society

Another important aspect for interdisciplinary sciences is the relationship with society. By nature, everybody is interested in faces. There is yet a danger that study of the face will be connected with social discrimination if things go badly. For example, if we research criminals' faces carelessly and produce something like "a typical criminal face," people who happen to resemble those facial composites may not be able to pass police stations. When some incident happens, such an average criminal face could cause a false accusation.

Facial studies always hold that kind of danger. They should not be an exclusive science. Societies dealing with the face must be always open to society. They must be objects that attract social attention in a good sense and are watched by society in a bad sense.

Would you remember that? There was an event called the "Dai-Kao-Ten (Great Exhibit of the Face)" at the National Science Museum in Ueno in summer 1999. The "Japan Academy of Facial Studies" supervised the project.

The Japan Academy of Facial Studies is a relatively new society established in March of 1995. I helped set up the Academy, too. The Japan Academy of Facial Studies is a small society consisting of some 800 members. Nonetheless, it gave a great presentation to a large audience of 300,000. That was the exhibition of the "Dai-Kao-Ten."

As soon as I mention facial studies, people mistake them for physiognomy. They are not at all similar. What we sought to demonstrate through the exhibition was that facial studies deal with "the face" by applying scientific methodology and that the field of facial studies has depth and importance.

I believe that this attitude of the science that tries to walk with the society will be the ideal future course of the technology.

A Role as the Unsung Hero

Again, I will return to the subject of my specialty. As I said before, my background is communications, specifically image processing by computer. Thus, even though I am interested in the face, I do not intend to do psychological facial studies. Of course there is no point for me to do anthropology as well. Still, I would like to contribute to "Facial Studies" from an engineering viewpoint.

In my laboratory, I modify expressions of the Mona Lisa and synthesize average faces of myriad individuals using a computer. For example, facial image processing initially supported research and development of the videophone. But after a while, I realized that my study might become a methodology; analyzing, processing and composing faces by using a computer will be a research tool for those who study the face in various fields such as psychology and anthropology. Actually, from each field, it was requested. In a sense, developing and providing a research tool put me in the role of an unsung hero (though I do not know whether I have such power). And, I think now, it may be good as a task for engineering faculties. It is because the school of engineering is the place to do something useful for other people.

The Communication Society becoming Faceless

On the other hand, I specialize in media technology such as multimedia or virtual reality. From that viewpoint, I am interested in how the form of communications through the media will change from now on.

Among them, a particular concern is that the media may be concealing the face and that the face is getting more and more hidden in communications.

On the telephone, we cannot see faces and can only hear voices. But we can somehow guess a speaker's facial expression from the state of the voice. In the case of e-mail, we can get from it only formatted characters. The face of a receiver cannot be seen at all.

In this way, communications without showing faces is becoming natural in the media. On the other hand, we show faces when we communicate with others all the time. It is only natural, after all. We show faces to others by presenting them where they are most easily seen, in front of their eyes. Even when we cover other parts of the body with clothing, we show our bare faces. In a sense, it was the basis of a reliable feeling and of a relationship of mutual trust. It is not too much to say that it is the basis of social order. Because people show their faces, they can not do very bad things. Committing crimes, people cover their faces with nylon stockings.

However, not showing faces has become natural in the network. What will happen to relationships of mutual trust constituted by showing faces? We must confront the question of keeping so-called social order in the era of networks.

As a result of hiding the face, not only the mode of communication, but also personalities of people who communicate there might change. Dr. Jekyll might be transformed into Mr. Hyde. I call that kind of society as "the Faceless Society." That is a coined word parodying the anonymity.

After arrival of the age of facelessness, the face, considered to be natural up to now, is no longer natural in communications. Then we have to think again about the meaning of the face in communications. When it does become unnatural, we must question seriously what the meaning of the face is and where the face should be placed in the communication systems hierarchy.

Surface Humanity, Line Humanity, and-

There are expressions like surface humanity and line humanity. The surface means that of a face. Those who respect face-to-face communication are called "surface humanity"; they are gradually evolving to line humanity. The line might mean a line of a telephone wire. The number of people who talk on the telephone more readily than face-to-face has rather been increasing. Those people are "line humanity."

Mathematically speaking, surfaces are two-dimensional and lines are one-dimensional. The zero-dimension might come next. The zero-dimension is a point, isn't it? Here comes "point humanity." At the line humanity stage, a receiver exists at the end of a telephone line. But point humanity does not have such a receiver. In some cases, people only communicate with a computer. It is said that there is a person who processes ideal faces by a computer (there seems to be a person developing that kind of technology) and only communicates with it every day. A woman in the real world does not do what one wants them to, but a woman on the computer display screen always smiles and does just as she is told. People who think in this way belong to point humanity.

Although it is another play on words, points are zero-dimensional, lines one-dimensional, and surfaces two-dimensional; then perhaps there were three dimensions before two dimensions. It was the body. There was "body humanity" in contrast to surface humanity. Body humanity embraces communication with the body. It is physical contact communications. Animals mainly communicate with physical contact. Then, humans came to be able to communicate by looking at the face even at a distance. It is a transition from body humanity to line humanity.

Therefore, language came into being. Because human beings obtained the medium called language, they came to communicate at a distance. Human facial expressions became enriched, too, and communications through it became possible. In other words, the human being has evolved from body humanity to surface humanity by developing communications media, namely the face and language.

Now we have obtained the so-called electronic communication media. Consequently, surface humanity has become line humanity, then transforming to point humanity. Can such really be called evolution?

In fact, I thought about such things a long time ago. When Tsutomu Miyazaki, the Japanese serial murder defendant pleaded his case. (Note: Miyazaki is the criminal who abducted and killed four little girls serially in the metropolitan area from 1988 to 1989.) I was lost in thought as to what on earth this was. Is this a simple case of a pervert or something different? At the time the case happened, the private room of Miyazaki was shown on TV. Videocassettes lined in the room. He spent his time watching videocassettes day after day there. He also had few friends.

I do not know the truth, but it seemed to me that Miyazaki was a point human created by the latest medium at that time - video. If the Iwanami Paperback Library (a series of academic books) had lined in his bookshelves, nobody would have been shocked too much. Perhaps the latest medium produced such an example of point humanity. The case might anticipate a new age. That is why I was disturbed by that case.

Point humanity caused a sort of case that body humanity might. After all, point human men can not accompany living women in the real world, but they can exert control over women in computers as they want to. When this Miyazaki point human brought the concept into the real world, he realized that children would act as he wants them to. Then he discovered victims (little girls) whom he could control; and then tragedy occurred.

After all, becoming a point human means becoming a body human, and the former might have returned to the latter. Although media have developed to advance communications, does not it make its way contrary to our expectations? Does media rather prevent us from communicating?

This is the important and serious issue confronting me at the moment.

Facelessness is a Source of Courage.

Let's change the light of inquiry and brighten up this dark discussion.

The "facelessness" that hides the face does not have only negative aspects by any means. An effort to extend its positive side is also necessary. We have newly obtained a completely different channel of communications by using the network. Since we obtained another communication channel through great effort, it is meaningless to make it the same as the real channels. The idea that a new channel should be designed by a new way of thinking is necessary.

Actually, there are some cases where people can communicate more smoothly when they do not show their faces. Here is an example. When a parent and child communicate face to face, the former says, "You must study." And the latter can not reply because the child feels offended. However, when a parent and child communicate by e-mail, they could say what they really want say to the other. "When I talked to you today, I unintentionally said again that you must study. But what I really want to say to you then is this." There are things that people can not say to face to face but can communicate by e-mail. (laughs)

There may be another aspect, too. To show one's face means to show everything of the person so that people place themselves in various social restraints. By clearing away such restraints, things might move in a better direction. For example, covering the face makes a person courageous. All the old super heroes covered their faces. Both "Kuramatengu" (a side horse long-nosed goblin) and "Gekkoukamen" (moonlight mask) did so.

Anonymity, which was the basis of the concept of facelessness, resembles it. When you live in a village community, your neighbors know everything about you. They even know to what age you continued wetting the bed. Thus, if you try to do something new there, people will not respect you and say, "What are you, you 10 year old bed-wetter, saying?" (laughs) However, this does not happen in a city. Because nobody knows that you wet the bed until the age of 10, you can come out with a sound argument. It is a positive aspect of anonymity. In a city, people escape from ties of name by anonymity and obtained new energy, and the free environment offered there. I think that this is the core of urban energy. People who lived in a city caused civil revolution and built up a new age. It was not revolution by villagers, but by city people to the last.

In this way, anonymity has some value. In the network, anonymity has progressed to be faceless. Generally, people with the anonymous face in the network seem to be seen in a bad light; they make crank calls or silent harassing calls. Although some people are gentlemen and ladies in the real world, they become immoderately aggressive in a network.

But when an anonymous city came into being, it was seen with bad images by a village. Urban images were these: the city is messy, many crimes occur, city dwellers have no moral sense. However, those people developed the new age. A similar thing may happen with the network.

Of course it is true that initial efforts harbor many problems. Still, it is necessary to make an effort to create new things by solving problems one by one.

The Face with a Name and the Face without a Name

Regarding size, the human face is approximately one-seventh or one-eighth of the body by length. Although the face is small, it actually has a meaning disproportionate to its size. Perhaps it could be said that the face is equal to the personality, or equal to the human being. "Sullying the face" does not mean that a face, a small part of a body, is made dirty but means that someone's personality itself is damaged.

On the other hand, there is the face without personality, too. It is the computer generated face. In my special field, there is a research to use a face for a computer interface. If we could talk to a face synthesized by a computer, we would be able to communicate with a computer just as we make a videophone call. It will not to be necessary to type at a keyboard or to "operate" a computer.

Not only does a computer answer "I will make an answer" with voice synthesis, but it might also synthesize a face and make it smile. When a computer is calculating hard, it might show a difficult face. Once the answer comes about, it might make the face smile. It might be possible to represent what a computer is doing at the time by the facial expression.

However, in practice it is considerably difficult to synthesize a real human face by computer. The more the face approaches reality, the more it becomes unnatural. At first, the reputation is pretty good. One might wonder at how such a face could be so natural. But, when we try to make it more natural, its surreal nature attracts people's attention contrarily.

There is something wrong with the face. Feeling that way is normal. It is because we see genuine human faces in everyday life. The more the synthesized face approaches a real one, the more people become aware of differences between the synthesized face and real human faces, or the more the synthesized face approaches the face of the dead. The more we copy a real face, the more the synthesized face approaches to the face of dead. That is weird.

Then the following opinion arises. As a computer interface, it is better for the face to be a complete cartoon rather than to copy of a real human face. In one extreme case, if you drew two big eyes, it could be seen as a face. Only making the small black points of the eyes (pupils) move can yield basic facial expressions.

Furthermore, there has been a lively argument over how much the computer interface needs "the face."

Personally I think about it in this way. Basically, the face is not needed if people only exchange information with a computer. Or, you could extremely deform it. In brief, the face is related with the personality of the person, so it is not interesting if a face that you do not know appears on a computer display screen. (laughs)

No matter how real the computer synthesized face looks, it has no meaning for those who do not know the person. You really may as well use cartoons. However, in the case of the face of the person you know, it really must be the face of the particular person, for the face is related to the person's personality. Of course a portrait could be O.K., but it is still crucial that it be the very person. For instance, a schoolteacher writes the software for summer vacation homework on his own, so the teacher's face sometimes appears on a computer display screen. (laughs) There is no point if it is not the face of that teacher.

The Face, After All, Depends on Interpersonal Relationships

I think that the face basically depends on the relationships between human beings. A face has no meaning until a human being has a relationship with others. On the contrary, the nameless face without a personality has no relation to other people, so that you could deform it or draw it like an animated cartoon. You are better off doing so.

The face comes to have meanings at the stage where it has relation to others and a concrete name. The face itself does not matter, but the relation to others through the face is important, I think. For example, we must pay attention to fractional photographs of the face. If we process someone's face by computer without consulting that person, it will hurt that person's personality rather than the face. Of course there is the matter of portrait rights, too.

Incidentally, when I employ a "composite" or "average" face at a presentation of a learned society, things go very smoothly. There is no portrait right for such a face. It is because the face is anonymous. In facial studies, when the face has been tied to a name, the argument will lose objectivity. One important point particularly to study the face from a point of view of natural sciences is to remove personality, specifically a name, from the face.

The name disappears by making an average face. It is convenient in this sense. However, because there are parts lost by deleting a name, one must consider what those lost parts are. It is very important whether the face has a name or not when we study "the face."

Toward Communication Showing Faces Once Again

So far, I have talked about "the face" here that I have been always thought about. Finally, I will give my opinion on future expectations.

When faceless communication become natural, conversely "communication showing faces" in the real world, namely in real society, will be highlighted. Faceless communications are easy to conduct. But it is important to show our own faces in real communications with important people. Such understanding will come about.

I do not know whether this is a suitable example, but when plastic tableware came out, for a certain time it was said that all the tableware from then on would be plastic because it was unbreakable. But what really happened? It is true that we surely use practical containers for box lunches at the convenience store. But we prepare traditional tableware for important meals.

I think that this kind of proper use will occur in the network age. It may be acceptable to work efficiently by being faceless. But when people really care about interpersonal relationships, they communicate by showing their faces properly. I expect that such an age will come.

I myself synthesize faces by computer and study multimedia, and am the president of The Virtual Reality Society of Japan (VRSJ). So, I am seen as a person rather living in the virtual world. But, actually, I am a human being completely living in the real world. I myself enjoy communicating with others in the real world more than doing so in the virtual world. This is why I am so particular about "the face", I think.