Andrew Gardner and Irfan Essa (1997) “Prosody Analysis for Speaker Affect Determination” In Proceedings of Perceptual User Interfaces Workshop (PUI 1997), Banff, Alberta, CANADA, Oct 1997 [PDF][Project Site] Abstract Speech is a complex waveform containing verbal (e.g. phoneme, syllable, and word) and nonverbal (e.g. speaker identity, emotional state, and tone) information. Both the verbal and […]
Coding, analysis, interpretation, and recognition of facial expressions Essa, I.A. Pentland, A.P. In IEEE Transactions on Pattern Analysis and Machine Intelligence, July 1997, Volume: 19 , Issue: 7, pp 757 – 763, ISSN: 0162-8828, CODEN: ITPIDJ. INSPEC Accession Number:5661539 Digital Object Identifier: 10.1109/34.598232 Abstract We describe a computer vision system for observing facial motion by […]
Daniel Goldman (1997 “Laugh and Your Computer Will Laugh With You, Someday” Jan 7, 1997 Issue, New York Times, Science TimesQuote from the article: “Emotions like fear, sadness and anger each announce themselves through a unique signature of changes in facial muscle, vocal inflection, physiological arousal, and other such cues. Building on techniques of pattern […]
Darrell, T.J.; Essa, I.A.; Pentland, A.P., “Task-specific gesture analysis in real-time using interpolated views” Transactions on Pattern Analysis and Machine Intelligence , vol.18, no.12, pp.1236-1242, Dec 1996 URL: [ieeexplore.ieee.org] [DOI] Abstract Hand and face gestures are modeled using an appearance-based approach in which patterns are represented as a vector of similarity scores to a set […]
S. Basu, I. Essa, A. Pentland (1996) “Motion regularization for model-based head tracking.” In Proceedings of Proceedings of the 13th International Conference on Pattern Recognition, 1996., 25-29 Aug 1996 Volume: 3, page(s): 611-616. [ DOI | PDF] Abstract This paper describes a method for the robust tracking of rigid head motion from video. This method […]
An article by my Ph.D. Advisor, Alex (Sandy) Pentland describing the Smart Rooms project (also sometimes called the “Looking at People” project, at the MIT Media Laboratory from the early 1990s, in the Scientific American Magazine. My work as a Ph.D. student (1990-1994) and as Research Scientist (1994-1996) was featured in this article. Alex Pentland […]
Evan I. Schwartz (1995) “A Face of One’s Own | Memory, Emotions, & Decisions”, DISCOVER MagazineDecember 1, 1995. Quote from the Article: “Chief among the members of his staff working on the problem is computer scientist Irfan Essa. To get computers to read facial expressions such as happiness or anger, Essa has designed three-dimensional animated […]
Essa, I.A. Pentland, A.P. (1995), “Facial expression recognition using a dynamic model and motion energy”, In Proceedings of Fifth International Conference on Computer Vision, 1995, 20-23 June 1995, page(s): 360 – 367, 06/20/1995 – 06/23/1995, Cambridge, MA, ISBN: 0-8186-7042-8, INSPEC Accession Number:5028034 Digital Object Identifier: [DOI:10.1109/ICCV.1995.466916][IEEEXplore#] Abstract Previous efforts at facial expression recognition have been […]
This thesis describes a computer vision system for observing the “action units” of a face using video sequences as input. The visual observation (sensing) is achieved by using an optimal estimation optical flow method coupled with a geometric and a physical (muscle) model describing the facial structure. This modeling results in a time-varying spatial patterning of facial shape and a parametric representation of the independent muscle action groups responsible for the observed facial motions. These muscle action patterns are then used for analysis, interpretation, recognition, and synthesis of facial expressions. Thus, by interpreting facial motions within a physics-based optimal estimation framework, a new control model of facial movement is developed. The newly extracted action units (which we name “FACS+”) are both physics and geometry-based and extend the well-known FACS parameters for facial expressions by adding temporal information and non-local spatial patterning of facial motion.