Uncanny valley: Why do some NPCs creep you out?

Have you ever felt creeped out or even repulsed by a video game character but couldn’t really tell why? It happens when you subconciously hit a so-called uncanny valley, a phenomenon when humans realize there is something off with the human-like looking entity in front of them.

What is uncanny valley?

You have most like encountered this term already in relation to robots. The term “uncanny valley” was coined by a Japanese roboticist, professor Masahiro Mori in the 1970s. His hypothesis was that the closer people get to the depiction of robots as humans, the affinity for them increases until we come into a “valley” where we feel uncomfortable to look at a thing or entity resembling a person (or its part) but not behaving or looking as such. This valley is what we know as “uncanny valley” where people tend to get repulsed by the uncanny resemblance with a human.

Uncanny valley is not a new thing, though. One of the first discussions about uncanny resemblance of puppets has some roots in the 1906 paper by Ernst Anton Jentsch, a German psychiatrist, and even Sigmund Freud shared his opinions on the matter at the time. They discussed specifically the fear of so-called automata, later heavily discussed as a part of cybernetics, a theory of systems, by experts such as C. Shannon, John von Neumann, and alike.

In his paper, professor Mori provided a graph that covers most types of objects holding human likeness and either missing or fulfilling it. Especially thought-provoking are his remarks about movement deepening the uncanny valley. There have been variations of the graph made over the years but the basics stay the same. Only recently some researchers suggested the valley might actually be a cliff [Bartneck et al., 2007].

Prosthetic origins

The original artefact Mori used as his example was a prosthetic hand. Prosthetic hands, even in the 70s, had achieved a resemblance to the human form that looked on par with false teeth. Upon the first glance, artificial hands covered by a skin-like looking material seems human. But once a person hits the realization that both the hand and material are actually artificial, they experience an eerie sensation. People usually get startled once they shake the hand. What gives it away is the unnatural behavior of the hand; including its coldness and boneless structure that doesn’t fit in the mental model of how a human hand should behave while interacting with it.

Recent studies [Poliakoff et al., 2018] examined the perception of eeriness of human prosthetic hands to, once again, support or refute Mori’s hypothesis.

It gets worse with movement

As the graph shows, uncanny valley gets even more prominent with movement. People are in general so attuned and familiar to the natural flow of movements human bodies and their parts make that even the slightest difference can throw them into the uncanny valley.

Just think about all the examples from movies where the world is hit by a virus and the majority of the population becomes zombies. These creatures either tend to have jerky, jolting and in other way spasmic, uncoordinated movements, or are impossibly slow and drag themselves around. All of it seems so unnatural to the human eye, the audience, and hopefully the main protagonist as well, immediately recognize there’s something off.

Research says…

Different explanations investigated from biological, cognitive, and social roots tried explaining uncanny valley, but its origin is not conclusively confirmed. For that reason some researchers contest the existence of uncanny valley entirely.

Cultural and generation divide

What research has confirmed so far, though, is there seems to be a generational and cultural divide and predispositions connected to the phenomenon. For example, some cultures are so accustomed to living with their dead ones that one can argue their threshold for uncanny valley is most likely placed in a very different spot than the one of cultures burying their dead.

What is even more interesting, and confirms that uncanny valley is actually acquired not born with, is the phenomenon among children and their perception of robots. Studies show that until the age of 9, children don’t bat an eye on whether a humanoid robot might possess feelings or cognition. It is plausible that the more contact children have with highly sophisticated types of robots or voice agents like Siri or Alexa, the more their generation will get accustomed to assumptions of AI having feelings and emotions. Therefore, the uncanny valley phenomenon might eventually disappear.

Research also shows there is an effect between the prior familiarity with a character and uncanny valley, where the curve tends to be moderate for those characters previously known by the subject. In short, if you are familiar with a character you might feel less uneasy upon seeing it again. But to be honest, Henry Cavill without his infamous CGI Superman moustache in Justice League is creepy even after all these years…

Just a couple of examples of uncanny valley from recent (and not so recent) movies: Cats, Justice League, The Polar Express, and The Scorpion King.

Theoretical explanations of the uncanny

To be honest, we don’t really know for sure what causes the uncanny valley phenomenon. One of the approaches that aims to explain what happens when human interact with increasingly human-like characters and feel uneasy claims that the discomfort comes from the perceptual mismatch hypothesis when a mismatch by inconsistent sensory cues happens (e.g. proportions of face features doesn’t match skin texture, etc.).

On the other hand, theory of mind might offer another explanation. Research has shown that the most problematic parts causing uncanny valley are particularly face and eye-related areas. As the theory of mind explains, humans are tuned in to detect even small changes in facial expressions and eye movement and surrounding areas. Facial features in general are used to assess other people’s mental states. When for example a smile doesn’t match human anatomy, humans subconsciously thinks there is something off with the person and a cautionary signal is sent to the brain. The rest is but a simple biology: “if something looks off, it might be potentially dangerous, thus I should avoid it”.

Uncanny versus immersion

As humans interact more with software agents, and come to rely on them more, it becomes increasingly important that systems like video games rely on the same interactional rules that humans do. Although believability is often named as the primary purpose behind embodying the interface into a human-like character such as NPC, functions and properly conveyed social cues would appear to be more effective.

In video games in particular, the demand for bridging the uncanny valley (both with looks and movement alike) is of the utmost importance because it can hinder player’s immersion and overall experience. Immersion is a complex cognitive state (I’ll cover it some of the future blogposts) so it might be broken by a myriad of things. However games aiming for photorealistic, human-like looking and behaving characters have higher probability of losing their players due to uncanny valley caused by offset animations, unnatural movements or anatomically wrong facial expressions.

Character design can take advantage of the uncanny valley phenomenon and use it to support eeriness of an android video game character

One of the ways to bypass the problem of the uncanny valley phenomen is to go through the path of stylization. Some of the elements of stylization that have the greatest impact on the attractiveness of characters are material properties or figures, and even the shape of characters determines mainly how realistic they appear (think Gollum’s huge eyes from The Lord of The Rings or the Na’vi clan members from The Avatar and their exaggarated facial features). Sometimes uncanny valley is even desirable, especially in creation of deliberately repulsive characters or characters who are robots like in Detroit: Become Human by Quantic Dream.

Before we part our ways

With the evolution of animation tools and mocap technology, rendering and high-fidelity texturing, light and materials used in video games (and engines like Unreal 5) it is not a surprise there’s an increasing demand for photorealistic and believable NPCs and avatars in video games. Also the nature of game developers to challenge themselves and increase the realism of game worlds and characters inhabiting them is propelling the field towards photorealism and unfrotunately a potential entrapment of uncanny valley.

Looking at games such as God of War: Ragnarok, The Last of Us II, Red Dead Redemption II and others, we can see some game studios manage to jump over the valley with success, whereas some are left hanging above it, hoping to cross it soon Until Dawn. Either way, I have an uncanny feeling we’ll see some eye-opening things in the future from both sides of the valley…