Facial animation in Blender: shape keys, face rigs and lip sync

What facial animation is

Facial animation is the animation of a character's face: the brows, the eyelids and the eyes, the cheeks, the mouth and the jaw. It does two jobs at once. The face shows what the character feels and thinks, and the mouth has to follow the dialogue closely enough for the audience to believe the voice comes out of it. The first job is acting; the second is lip sync animation. Blender settings below are checked against the Blender 5.2 manual and, where the manual says nothing, the Blender source code.

3D software moves a face in two classic ways. Morph targets store a full deformed copy of the mesh (a smile, a closed eye, a raised brow) and blend between the neutral face and those targets. Skeletal animation moves the face with bones, the way a rig moves an arm. Blender calls its morph targets shape keys. Its manual notes that other software calls them morph targets or blend shapes, and that facial animation (mouth positions, expressions, phonemes) is their most common use. Unity, for example, lists them as BlendShapes on a Skinned Mesh Renderer, and the glTF format calls them morph targets.

Shape keys, face bones, or both

The Blender manual sums up where shape keys win: organic, soft parts and muscles, where you need more control over the shape than rotation or scale can give. Bones win where the motion really is a rotation, such as a jaw that hinges or an eye that turns. The two combine well: a driver can read the rotation of the jaw bone and push a corrective shape key in as the mouth opens, the same way the manual's own example fixes an elbow. Rigify's documentation also points the other way: face bones are "almost useless" if your facial animation goes through shape keys, and you can delete them. For the body side of the rig, see rigging in Blender.

Shape keys in Blender, step by step

Shape keys live in the Shape Keys panel of the Object Data tab in the Properties editor (the Mesh tab, for a mesh). They work on meshes, curves, surfaces and lattices. A new stack is Relative, which is what a face needs: every key is a blend between itself and a reference, and all keys add up into the result the manual calls the Mix. Absolute keys play shapes one after the other along an Evaluation Time instead.

1. Finish the topology. A shape key stores only the positions of existing vertices, so every key holds every vertex. The manual's advice is to finalize the mesh before creating any key.
2. Create the Basis. In Object Mode, click + next to the list. The first key is the Basis: the original vertex positions, the neutral face. It has no weight and cannot be keyed.
3. Add a key. Click + again. The new key is a copy of the Basis, whatever the viewport shows at the time. Name it after what it does: Blink_L, Smile_R, Jaw_Open.
4. Shape it. Enter Edit Mode, check that the new key is the one selected in the list, and move vertices. Selecting another key shows that key instead.
5. Test it. Back in Object Mode, drag Value: 0 gives the reference key, 1 the full shape.

The settings of each key

• Value: the blend between the key and its reference. Blender can extrapolate beyond 1 and below 0, which amplifies or inverts the shape.
• Range Min / Max: the limits of the Value slider, 0 and 1 on a new key, settable from -10 to 10.
• Vertex Group: limits the key to the vertices of a group.
• Relative To: the reference key the shape deforms from, the Basis unless you change it.
• Mute (the checkbox in the list): leaves the key out of the Mix.
• Lock Shape: protects a key; most Edit Mode and Sculpt Mode tools that move vertices stop with an error when the active key is locked.
• Shape Key Lock (pin icon) shows the active key alone, and is switched on automatically in Edit Mode. Shape Key Edit Mode makes Edit Mode show the current blend instead.

Left and right sides

Separate left and right keys let each side of the face move on its own, and Blender has several ways to get them. The manual describes a technique for faces: sculpt a complex extreme pose, then split it with a temporary vertex group. Click the X button under the list (Clear Shape Keys) to set every value to 0, set your full Smile to 1 and give it a vertex group covering the left half, then choose Shape Key Specials (the arrow menu next to the list) > New Combined, which makes a new key from the current mix: the left smile alone. Swap the group for the right half and repeat. On a perfectly symmetrical mesh, Duplicate a left key and Flip the copy, in Object Mode, to mirror it across X; Flip (Topology) finds the mirrored vertices from the topology when the mesh is not perfectly symmetrical. Shapes sculpted on separate objects with matching topology come in with New from Objects, or New from Objects Flipped for the opposite side.

Keyframing shape keys

Hover over a key's Value and press I, or right-click it and choose Insert Keyframe; Alt+I over the value deletes the key on the current frame. Timing is easier in the Shape Key Editor, a mode of the Dope Sheet picked from the dropdown in its header. It shows one row per shape key with its keyframes, and its sliders are always visible, so you can set values and move or scale keys in one place. The guide to keyframes in Blender covers interpolation and editing.

A face rig with bones: Rigify

Rigify, the rigging add-on bundled with Blender, builds a full rig from a simple armature called a meta-rig. Turn it on in the Add-ons section of the Preferences.

1. In Object Mode, use Add > Armature > Human. Basic Human leaves out the face and the fingers. Rigify assumes 1 unit is 1 meter.
2. Move the face bones onto your mesh, in Edit Mode, or in Pose Mode followed by Apply > Apply Pose As Rest Pose. Keep the merge points: wherever bone heads or tails meet, they must still meet afterwards, or Rigify may create several controls instead of one, or stop with errors. Eye bones sit at the pivot of the eye, along Y, as long as its radius.
3. In Blender 5.2, the Human meta-rig still carries the older face, a rig type being deprecated. The Upgrade Face Rig button above Generate Rig converts it to the modular face. Skinning survives the upgrade, but poses and animation made on the old face will likely not carry over.
4. Click Generate Rig in the Armature tab.

The modular face gives the animator a Jaw Master that rotates the jaw and opens the lips with it, a Mouth Master that moves the lips without the jaw, and a Mouth Lock slider that keeps the mouth closed while the jaw moves. For the eyes, a Common Target sets where both look, each Eye Target aims one eye, and Eyelids Follow sets how much the lids follow the eyeball.

Weight painting the face

Rigify does not attach the rig to the mesh. Select the mesh, then the rig, press Ctrl+P and choose Armature Deform > With Automatic Weights. The deforming bones are in the DEF bone collection, and Rigify's manual says automatic weights usually work well when the bones are placed correctly and the mesh has enough topology. The eye and teeth bones of the older face do not deform: those meshes follow through Child Of constraints. Automatic weights come from each bone's distance to the vertices, and influences can overlap, so pose the extremes (jaw wide open, eyes shut) and fix what breaks in Weight Paint Mode, where blue means 0 and red means 1. The weight painting guide covers the brushes.

Drivers: let a bone or a slider move a shape key

A driver sets a property from other values. On a face, it lets the animator pose a few controls instead of hunting through a long list of sliders.

1. Right-click the Value of the shape key and choose Add Driver, or press Ctrl+D over it. Blender adds a driver with one empty variable and opens the Edit Driver popover.
2. To follow a bone, set the variable to Transform Channel, pick the rig and the bone, a channel such as X Rotation, and Local Space (rotation relative to the parent bone).
3. In the Drivers editor, shape the curve: the X axis is the bone's rotation in radians, the Y axis the shape key Value. Put 0 at the rest pose and 1 at the target pose. Anything outside the key's Range is clamped.
4. To use a slider instead, add a custom property to a control bone (Custom Properties panel > New), right-click it, choose Copy As New Driver, then right-click the shape key Value and choose Paste Driver.

To shape a corrective key on top of the bone deformation, the manual first enables Edit Mode Display and Cage Editing in the header of the Armature modifier, then edits the key in the problem pose.

Lip sync animation

Phonemes and visemes

A phoneme is a speech sound that speakers treat as one unit, able to tell one word from another, like the /l/ and /t/ that separate "cell" from "set". A viseme is what a phoneme looks like on the face, and the two do not map one to one: /p/, /b/ and /m/ look alike, which is why lip readers confuse "pet", "bell" and "men". Lip sync animation works with visemes. Meta's Oculus Lipsync uses these 15:

Microsoft's speech service uses 22 visemes and groups the sounds differently, putting d, t, n and the th of "think" in one shape. The grouping is a design choice, so pick one list and keep to it for the whole character.

Bring the dialogue into Blender

Since Blender 5.0, the Video Sequencer works on a Sequencer Scene, chosen per workspace with a Scene selector in the Sequencer header, and it can differ from the scene you animate. Select your character's scene there, then use Add > Sound and pick the dialogue file; the strip references the file on disk. Turn on waveforms in the Sequencer overlays to see where the syllables land. In the Playback popover of the Timeline, set Sync to Sync to Audio, which drops frames rather than letting the picture drift from the sound, and tick Scrubbing to hear bits of the sound while you drag the playhead.

Work pass by pass

The workflow below follows animator Carlo Loffredo's lip sync tutorial for Animation Mentor and a set of lip sync tips published by Pluralsight.

1. Reference. Film yourself saying the line. You will see how little the jaw actually moves.
2. Jaw. Key the jaw where the mouth opens for vowels and closes for consonants, timed to the audio, then add breakdowns. Check the teeth: the lips can move while the jaw stays almost still.
3. Lips and corners. Shape the top lip, the bottom lip and the corners for each sound. On M, B and P the lips must touch and press: hold the closed shape for a couple of frames, then pop open to the next one.
4. Accents. Add lip rolls and funnels, puffed cheeks, the tongue where it shows, and push the shapes further on the loud moments of the line.
5. Offset. Opening the jaw one to two frames before the sound is heard makes the lip sync read better.

Eyes, blinks and brows

Shawn Kelly, an ILM animator and co-founder of Animation Mentor, calls "blink every two seconds" the worst advice he got at school. People blink for a reason: on a rapid head turn, on a big change of eye direction and, in his view above all, when a thought or an emotion changes. Rapid blinks read as shy or nervous, no blinks at all as intense. In Blender, a blink can be one key per eye (Blink_L, Blink_R) or the lid controls of the rig. Give the brows and lids room to differ from side to side: a 2024 study presented at the ACM Symposium on Applied Perception (Hauser and colleagues) found that asymmetric brow, mouth or eyelid motion made animated virtual agents look more believable, more appealing and more natural.

Exporting shape keys to a game engine

glTF exports shape keys as morph targets: Data > Shape Keys and Animation > Shape Key Animations are both ticked by default, with Shape Key Normals and Tangents as options, and Reset Shape Keys between actions for when some actions do not key every shape key. Its Apply Modifiers option is off by default, and its tooltip warns that it prevents exporting shape keys. Shape keys driven by bones are only sampled when the mesh is a direct child of the armature.

The FBX exporter writes shape keys as FBX blend shapes and exports the animation of relative keys as a percentage, which matches Unity's 0 to 100 weights. Its Apply Modifiers option is on by default and carries the same warning: Armature modifiers are left out, but most other active modifiers, a Subdivision Surface for instance, make the exporter write the evaluated mesh without its shape keys. The FBX and glTF export guide covers the other settings.

Render time and quality

If the face uses a Subdivision Surface modifier, it has separate Levels Viewport and Render values: keep the viewport level low so playback stays fluid while you check lip sync, and raise the render level, knowing that more levels mean more vertices and more memory. Modifiers are calculated from top to bottom, so keep the Armature modifier above the Subdivision Surface: the bones deform the base mesh and the smoothing comes last.

A jaw that snaps open in a frame or two still renders perfectly sharp on every frame, because Blender renders animation as a sequence of still images unless motion blur is on (Render Properties > Motion Blur), where a real camera would blur the move. In Cycles, each object also has Object Properties > Motion Blur > Deformation, which blurs deformed meshes such as animated characters; its Steps setting improves accuracy at the cost of memory. The manual warns that deformation blur cannot render correctly when modifiers change the topology over time; shape keys never change the vertex count. In EEVEE, every motion blur step re-evaluates the whole scene and can add a lot of render time. To see what such settings cost over a full shot, enter a time per frame in the render time calculator.

Common mistakes

• Animating every phoneme. A shape on every syllable opens and closes the jaw far more than real speech does. Hit the important shapes and blend over the rest.
• Shape keys that stack wrong. Relative keys add up, so two keys that move the same vertices double the effect. The manual's answer is to split complex shapes with vertex groups. Check Relative To as well, and remember that a Value above 1 exaggerates the shape.
• Editing the Basis by mistake. Edit Mode edits the key selected in the list. With the Basis selected, your change goes into the neutral face that every key is measured against. Lock finished keys with Lock Shape.
• A face that is too symmetrical. Identical values on both sides give up what the asymmetry study above measured. Use separate left and right keys and give each side its own values and slightly different timing.
• A mouth frozen or snapping shut between words. The lips usually stay slightly open between words, even in silence, and a held pose can still carry a little motion: Blender even has a Moving Hold keyframe type to mark such holds.
• Changing the topology late. A shape key only stores positions for the vertices that exist, and none can be added or removed inside a key: finish the mesh before the first key, as the manual advises.
• Exporting with Apply Modifiers on. With other modifiers on the face, the shape keys are dropped from the file.