[Edit, June 2016: To the best of my knowledge, this work with Arturo Castro is the first example of realistic "face swapping" in realtime. If you're interested in experimenting with the technique, I recommend Audun's implementation here https://auduno.github.io/clmtrackr/examples/facesubstitution.html ]
Working on an idea with Arturo Castro http://vimeo.com/29279198 I feel like "good" blending looks almost too natural to be surprising. It doesn't leave any interpolation up to your imagination. It's possible to push this style further, so it's less of a blend and more of a replacement, but then you get unnatural colors and shadows.
FaceTracker library from Jason Saragih http://web.mac.com/jsaragih/FaceTracker/FaceTracker.html
ofxFaceTracker addon https://github.com/kylemcdonald/ofxFaceTracker
openFrameworks http://www.openframeworks.cc/
Download the app here https://github.com/arturoc/FaceSubstitution