Intro -- Emotion Recognition -- Contents -- Preface -- Acknowledgments -- Contributors -- 1 Introduction to Emotion Recognition -- 1.1 Basics of Pattern Recognition -- 1.2 Emotion Detection as a Pattern Recognition Problem -- 1.3 Feature Extraction -- 1.3.1 Facial Expression–Based Features -- 1.3.2 Voice Features -- 1.3.3 EEG Features Used for Emotion Recognition -- 1.3.4 Gesture- and Posture-Based Emotional Features -- 1.3.5 Multimodal Features -- 1.4 Feature Reduction Techniques -- 1.4.1 Principal Component Analysis -- 1.4.2 Independent Component Analysis -- 1.4.3 Evolutionary Approach to Nonlinear Feature Reduction -- 1.5 Emotion Classification -- 1.5.1 Neural Classifier -- 1.5.2 Fuzzy Classifiers -- 1.5.3 Hidden Markov Model Based Classifiers -- 1.5.4 k-Nearest Neighbor Algorithm -- 1.5.5 Naïve Bayes Classifier -- 1.6 Multimodal Emotion Recognition -- 1.7 Stimulus Generation for Emotion Arousal -- 1.8 Validation Techniques -- 1.8.1 Performance Metrics for Emotion Classification -- 1.9 Summary -- References -- Author Biographies -- 2 Exploiting Dynamic Dependencies Among Action Units for Spontaneous Facial Action Recognition -- 2.1 Introduction -- 2.2 Related Work -- 2.3 Modeling the Semantic and Dynamic Relationships Among AUs With a DBN -- 2.3.1 A DBN for Modeling Dynamic Dependencies among AUs -- 2.3.2 Constructing the Initial DBN -- 2.3.3 Learning DBN Model -- 2.3.4 AU Recognition Through DBN Inference -- 2.4 EXPERIMENTAL RESULTS -- 2.4.1 Facial Action Unit Databases -- 2.4.2 Evaluation on Cohn and Kanade Database -- 2.4.3 Evaluation on Spontaneous Facial Expression Database -- 2.5 Conclusion -- References -- Author Biographies -- 3 Facial Expressions: A Cross-Cultural Study -- 3.1 Introduction -- 3.2 Extraction of Facial Regions and Ekman’s Action Units -- 3.2.1 Computation of Optical Flow Vector Representing Muscle Movement -- 3.2.2 Computation of Region of Interest -- 3.2.3 Computation of Feature Vectors Within ROI -- 3.2.4 Facial Deformation and Ekman’s Action Units -- 3.3 Cultural Variation in Occurrence of Different Aus -- 3.4 Classification Performance Considering Cultural Variability -- 3.5 Conclusion -- References -- Author Biographies -- 4 A Subject-dependent Facial Expression Recognition System -- 4.1 Introduction -- 4.2 Proposed Method -- 4.2.1 Face Detection -- 4.2.2 Preprocessing -- 4.2.3 Facial Feature Extraction -- 4.2.4 Face Recognition -- 4.2.5 Facial Expression Recognition -- 4.3 Experiment Result -- 4.3.1 Parameter Determination of the RBFNN -- 4.3.2 Comparison of Facial Features -- 4.3.3 Comparison of Face Recognition Using “Inner Face” and Full Face -- 4.3.4 Comparison of Subject-Dependent and Subject-Independent Facial Expression Recognition Systems -- 4.3.5 Comparison with Other Approaches -- 4.4 Conclusion -- Acknowledgment -- References -- Author Biographies -- 5 Facial Expression Recognition Using Independent Component Features and Hidden Markov Model -- 5.1 Introduction -- 5.2 Methodology -- 5.2.1 Expression Ima.
ge Preprocessing -- 5.2.2 Feature Extraction -- 5.2.3 Codebook and Code Generation -- 5.2.4 Expression Modeling and Training Using HMM -- 5.3 Experimental Results -- 5.4 Conclusion -- Acknowledgments -- References -- Author Biographies -- 6 Feature Selection for Facial Expression based on Rough Set Theory -- 6.1 Introduction -- 6.2 Feature Selection for Emotion Recognition Based on Rough Set Theory -- 6.2.1 Basic Concepts of Rough Set Theory -- 6.2.2 Feature Selection Based on Rough Set and Domain-Oriented Data-Driven Data Mining Theories -- 6.2.3 Attribute Reduction for Emotion Recognition -- 6.3 Experiment Results and Discussion -- 6.3.1 Experiment Condition -- 6.3.2 Experiments for Feature Selection Method for Emotion Recognition -- 6.3.3 Experiments for the Features Concerning Mouth for Emotion Recognition -- 6.4 Conclusion -- Acknowledgments -- References -- Author Biographies -- 7 Emotion Recognition from Facial Expressions Using Type-2 Fuzzy Sets -- 7.1 Introduction -- 7.2 Preliminaries on Type-2 Fuzzy Sets -- 7.2.1 Type-2 Fuzzy Sets -- 7.3 Uncertainty Management in Fuzzy-Space for Emotion Recognition -- 7.3.1 Principles Used in the IT2FS Approach -- 7.3.2 Principles Used in the GT2FS Approach -- 7.3.3 Methodology -- 7.4 Fuzzy Type-2 Membership Evaluation -- 7.5 Experimental Details -- 7.5.1 Feature Extraction -- 7.5.2 Creating the Type-2 Fuzzy Face-Space -- 7.5.3 Emotion Recognition of an Unknown Facial Expression -- 7.6 Performance Analysis -- 7.6.1 The McNemar’s Test -- 7.6.2 Friedman Test -- 7.6.3 The Confusion Matrix-Based RMS Error -- 7.7 Conclusion -- References -- Author Biographies -- 8 Emotion Recognition from Non-frontal Facial Images -- 8.1 Introduction -- 8.2 A Brief Review of Automatic Emotional Expression Recognition -- 8.2.1 Framework of Automatic Facial Emotion Recognition System -- 8.2.2 Extraction of Geometric Features -- 8.2.3 Extraction of Appearance Features -- 8.3 Databases for Non-Frontal Facial Emotion Recognition -- 8.3.1 BU-3DFE Database -- 8.3.2 BU-4DFE Database -- 8.3.3 CMU Multi-PIE Database -- 8.3.4 Bosphorus 3D Database -- 8.4 Recent Advances of Emotion Recognition from Non-Frontal Facial Images -- 8.4.1 Emotion Recognition from 3D Facial Models -- 8.4.2 Emotion Recognition from Non-frontal 2D Facial Images -- 8.5 Discussions and Conclusions -- Acknowledgments -- References -- Author Biographies -- 9 Maximum a Posteriori based Fusion Method for Speech Emotion Recognition -- 9.1 Introduction -- 9.2 Acoustic Feature Extraction for Emotion Recognition -- 9.3 Proposed Map-Based Fusion Method -- 9.3.1 Base Classifiers -- 9.3.2 MAP-Based Fusion -- 9.3.3 Addressing Small Training Dataset Problem—Calculation of fc|CL(cr) -- 9.3.4 Training and Testing Procedure -- 9.4 Experiment -- 9.4.1 Database -- 9.4.2 Experiment Description -- 9.4.3 Results and Discussion -- 9.5 Conclusion -- References -- Author Biographies -- 10 Emotion Recognition in Naturalistic Speech and Language—A Survey -- 10.1 Introduction -- 10.2 Tasks .
and Applications -- 10.2.1 Use-Cases for Automatic Emotion Recognition from Speech and Language -- 10.2.2 Databases -- 10.2.3 Modeling and Annotation: Categories versus Dimensions -- 10.2.4 Unit of Analysis -- 10.3 Implementation and Evaluation -- 10.3.1 Feature Extraction -- 10.3.2 Feature and Instance Selection -- 10.3.3 Classification and Learning -- 10.3.4 Partitioning and Evaluation -- 10.3.5 Research Toolkits and Open-Source Software -- 10.4 Challenges -- 10.4.1 Non-prototypicality, Reliability, and Class Sparsity -- 10.4.2 Generalization -- 10.4.3 Real-Time Processing -- 10.4.4 Acoustic Environments: Noise and Reverberation -- 10.5 Conclusion and Outlook -- Acknowledgment -- References -- 11 EEG-Based Emotion Recognition Using Advanced Signal Processing Techniques -- 11.1 Introduction -- 11.2 Brain Activity and Emotions -- 11.3 EEG-ER Systems: An Overview -- 11.4 Emotion Elicitation -- 11.4.1 Discrete Emotions -- 11.4.2 Affective States -- 11.4.3 Datasets -- 11.5 Advanced Signal Processing in EEG-ER -- 11.5.1 Discrete Emotions -- 11.5.2 Affective States -- 11.6 Concluding Remarks and Future Directions -- References -- Author Biographies -- 12 Frequency Band Localization on Multiple Physiological Signals for Human Emotion Classification Using DWT -- 12.1 Introduction -- 12.2 Related Work -- 12.3 Research Methodology -- 12.3.1 Physiological Signals Acquisition -- 12.3.2 Preprocessing and Normalization -- 12.3.3 Feature Extraction -- 12.3.4 Emotion Classification -- 12.4 Experimental Results and Discussions -- 12.5 Conclusion -- 12.6 Future Work -- Acknowledgments -- References -- Author Biography -- 13 Toward Affective Brain–Computer Interface: Fundamentals and Analysis of EEG-based Emotion Classification -- 13.1 Introduction -- 13.1.1 Brain–Computer Interface -- 13.1.2 EEG Dynamics Associated with Emotion -- 13.1.3 Current Research in EEG-Based Emotion Classification -- 13.1.4 Addressed Issues -- 13.2 Materials and Methods -- 13.2.1 EEG Dataset -- 13.2.2 EEG Feature Extraction -- 13.2.3 EEG Feature Selection -- 13.2.4 EEG Feature Classification -- 13.3 Results and Discussion -- 13.3.1 Superiority of Differential Power Asymmetry -- 13.3.2 Gender Independence in Differential Power Asymmetry -- 13.3.3 Channel Reduction from Differential Power Asymmetry -- 13.3.4 Generalization of Differential Power Asymmetry -- 13.4 Conclusion -- 13.5 Issues and Challenges Toward ABCIs -- 13.5.1 Directions for Improving Estimation Performance -- 13.5.2 Online System Implementation -- Acknowledgments -- References -- Author Biographies -- 14 Bodily Expression for Automatic Affect Recognition -- 14.1 Introduction -- 14.2 Background and Related Work -- 14.2.1 Body as an Autonomous Channel for Affect Perception and Analysis -- 14.2.2 Body as an Additional Channel for Affect Perception and Analysis -- 14.2.3 Bodily Expression Data and Annotation -- 14.3 Creating a Database of Facial and Bodily Expressions: The Fabo Database -- 14.4 Automatic Recognition of Affect f.
