Lecture 26: Facial Animation

The Approaches

Data driven
usually mesh interpolation – but sometimes internal “bones” as well
often with a neutral pose and offsets (deltas)
sometimes for part of the face (disjoint regions – but need to make sure the pieces fit together)
good for manual control
useful for automation
choose key poses that you know are useful
- visemes
- important, often used expressions
- extremum
or be opportunistic
- whatever poses that you can get

Very similar to blendshapes – just with automatically determined blendshapes
Take a pile of data, do PCA to get “eigenfaces”
similar in math to blendshapes, but different in practice
- blendshapes have no semantic meaning
- covers space of input
- need to use with automatic methods for control
often used as a way of parameterizing a set of faces
- Blanz and Vetter: 100 (or so) German college students
- project another face as a linear combination of these faces
  - Tom Hanks and Audrey Hepburn as a linear combination of University Students
- things you match can be low dimensional (match the images), combinations are of the data – so you get a 3D Tom Hanks from a picture of him

Rigs can be just about anything
coming up with them is hard and important
this paper is a remarkable grab bag of random stuff (says little about any particular thing – and much is meandering and not to the point)
basic lesson: you can do anything – and people do

directly manipulating points is nice

result is a linear combination
- this is a REALLY easy IK problem
need to choose which solution
minimize how far the sliders are from the zero position (or rest position, or last position)
damped minimization
- keep as close as possible to “rest” position (start of dragging movement)
  - min ||w-w_0||
  - this allows the user to slide sliders as well
- add in some amount of “keep w small” to avoid drift ||min w||
linear constraints, quadratic objective function
- could do penalty method (they do)
  - turn constraint into a tradeoff
  - cons: lose exact control, lose abilty to make big changes
  - pros: simplicity, stability
- solve with lagrange multipliers
- you might not get the absolute minimum – but any difference from the minimum has to be something that is required to meet the constraints

Some big question:

is smallest change in the sliders the right objective?
- arguably yes (since the sliders are the UI)
- isn’t the “most natural pose” or anything like that
- other methods adapt the objective function to try to keep in the space of “natural” results
what about over-determined cases
- unlikely – since model has lots of degrees of freedom
- nearly over-determines (pull apart two points that have to be next to each other)
  - small movement in a point requires big movement of something far away
  - damping helps a lot
  - soft constraints help a lot (tradeoff – big changes in slider vs. meeting constraints)

input
- set of sparse examples
  - what blend values these approximately equate to
- an example of a good model
output
- a blendable model of the example character
- expressions correspond to the original, but the examples are reproduced
challenges
- make blendshapes match semantics (same deformation gradients)
- make blendshapes match examples
Do everything in the deformation gradient space
- figure out how to deform the rest position mesh to the blendshapes
- blendshapes need to blend to make examples
  - too few constraints
- blendshapes should have similar gradients as the target (where they are known)
Bilinear alternation (find blend shapes given guess of weights, then updates weights)