Some believe the Uncanny Valley represents the limit of our ability to achieve realism in animation, and that animation technology will never achieve the precise detail in form, motion, texture, and emotion required to convince the subconscious mind that it truly is real. Maybe that's true, maybe it isn't - but with or without more advanced animation techniques, how can you avoid the Uncanny Valley in your animations?
Avoid combining hyper-realism with cartoonish or unrealistic elements.
This doesn't mean that you should avoid animating unrealistic things such as aliens, cyborgs, unicorns, whatever. Instead this means you should avoid pairing elements that only highlight the jarring difference between reality and unreality. This can include applying photo-realistic skin textures and bump maps to a cartoonish 3d model, or trying to recreate perfectly detailed human eyes in 3d animated faces that don't match normal human muscle movements. You can use detail and realism to great effect in animation, but it's important to blend your elements appropriately and try to achieve a balanced design aesthetic that doesn't give your viewers the willies when their subconscious is screaming that you pasted a real human being's skin on something that looks and moves like a kiddie toy.
Only use motion that matches the realism level of your animation.
One of the things that weirds me out the most is seeing detailed motion captures applied to blocky, unrealistic models. I see it the most in video games, especially in certain types of cutscenes. You want a certain level of realism in your motion, and motion capture is an extremely useful tool for achieving that - but when your 3d models aren't that detailed or venture into fantasy realms, seeing them move like normal human beings can be jarring. For detailed, semi-realistic 3d renders, you can often tweak motion captures to add just that right level of unrealism to make it easier to digest, whether you're enhancing the speed and grace of a superhuman creature or just smoothing out the motion to get rid of those very human little jerks and ticks that, in many cases, can detract from the human ability to relate to your characters, rather than add to it. In more cartoonish animation, using principles such as anticipation and squash and stretch can help you achieve realism without venturing into the Uncanny Valley.
In both cases, it's a matter of applying common sense and good judgment so that you know when that little extra bit of body language will enhance and impress, and when your characters will haunt your viewers' darkest nightmares. Body language plays a large part in subconscious identifiers that dictate how human beings recognize other humans and respond to their behavioral signals on a base level. When the eyes are saying it's not human but the body language is triggering human identifier responses and behavioral reactions, it creates a split in perception that can make people uneasy. This can go the other way, as well: when highly detailed, near-perfect human characters lack natural human motion, the effect is unsettling.
Watch the eyes and facial expressions.
Although body language is a very large factor in how humans perceive emotion, facial expressions and eye movement also play a major part, and it's crucial not to mess this up. The Polar Express made this mistake, as have several other recent animated films. It's less important to perfectly detail every vein in the sclera, every striation in the iris, and every dilation of the pupil than it is make sure the eyes are expressive and alive. The eyes need to convey emotion; they need to move in ways that indicate there is a conscious mind behind them; they need to look at things with a clear focus that directs the viewer's eye, rather than staring blankly straight ahead.
This is part of conveying facial expressions that evoke an emotional response, and part of why cartoons are easier to relate to than detailed animations with more subtle expressions. Cartoons exaggerate emotion on a level with their exaggerated physical characteristics; between near-distorted contortions of the eyes, mouth, and body language, we never doubt what the character is supposed to be feeling, and never question whether it's real. With more detailed animation it's a bit more difficult. Human facial expressions can range from the extreme to very subtle shifts that can entirely alter the perception of mood. Applying realistically subtle, complex nuances of expression to a cartoonish character, or even one that only reaches the nth level of realism, can result in an eerie wax doll. Conversely, distorting realistic 3d models with exaggerated expressions can yield some rather disturbing imagery. Find the right balance depending on how realistic your animation is.
This also applies to mouth movements with speech. When a character is talking and their phonemes don't match the words they're saying, it creates the same kind of dissonance as watching a dubbed kung fu flick. More basic movements are acceptable in unrealistic animations, but in highly realistic animations it's irritating to watch the jaws just flap and flap without the lips shaping actual words. Keep in mind that expressions need to persist through speech; if a person is smiling, their mouths will look different when speaking, etc.
Be careful with character voicing.
Imagine if Mickey Mouse sounded like Mass Effect's Commander Shepard. Isn't really working, is it? Now go watch a few animated programs and video games. Look at the difference in voice acting between something like Winx Club and something like Gears of War. Believe it or not, there is a difference in the voice acting there, and it follows the same principles of extremes as previous points about realism versus unrealism. Unrealistic and cartoony animations will have more unrealistic and exaggerated voices to help tell viewers / listeners how they should feel about what's being said and done. More realistic animations will have more realistic voice acting, though there's still a certain level of exaggeration both to match the sub-realism and to convey emotion to the audience. Swapping the two for anything other than comedic effect will remove the ability to empathize with your character.
Learn to use lighting effectively.
Scene lighting can make a rather large difference in how characters are perceived. Warm, soft lighting can evoke a dreamlike sense of warmth and can make characters blend into their environment. Blue lighting inspires a sense of the supernatural, and can make things seem stark and cold. Minimal lighting or high-contrast lighting can add a dangerous feel to the scene, gritty noir heavy on the shadows and the grim, sharp-edged planes of angular faces. Imagine if the lighting effects in The Ring had been used in Up. (Now I know, The Ring isn't animated, but bear with me on the example.) A family-friendly film with a heartwarming tearjerker of a story would have left viewers tense, uneasy, waiting for the twist that would never come and detracting from the film's feel-good vibe. Lighting is also important when blending 3d elements into real video; it's important to match the scene lighting as precisely as possible, or the CGI portions will always seem just a little out of place.
It takes skill and practice to learn how to balance design elements to achieve realism without the discomfort inherent in crossing into the Uncanny Valley, but these basic principles should help you in getting started.
