Micro Expression Spotting through Appearance Based Descriptor and Distance Analysis

Article Sidebar

PDF
Published: Aug 18, 2023
DOI: https://doi.org/10.17762/ijritcc.v11i8s.7171
Keywords:
Micro Expressions, Spotting, Local Binary pattern, Facial Alignment, Chi-Squared Distance, F1-Score

Main Article Content

P. Surekha
Research Scholar, Department of CSE, Koneru Lakshmaiah Education Foundation, Vaddeswaram, A.P, India
P. Vidya Sagar
Associate Profressor, Department of CSE Koneru Lakshmaiah Education Foundation, Vaddeswaram, A.P, India
G. Ramesh
Associate Profressor, Department of CSE, GRIET, Hyderabad, Telangana

Abstract

Micro-Expressions (MEs) are a typical kind of expressions which are subtle and short lived in nature and reveal the hidden emotion of human beings. Due to processing an entire video, the MEs recognition constitutes huge computational burden and also consumes more time. Hence, MEs spotting is required which locates the exact frames at which the movement of ME persists. Spotting is regarded as a primary step for MEs recognition. This paper proposes a new method for ME spotting which comprises three stages; pre-processing, feature extraction and discrimination. Pre-processing aligns the facial region in every frame based on three landmark points derived from three landmark regions. To do alignment, an in-plane rotation matrix is used which rotates the non-aligned coordinates into aligned coordinates. For feature extraction, two texture based descriptors are deployed; they are Local Binary Pattern (LBP) and Local Mean Binary Pattern (LMBP). Finally at discrimination stage, Feature Difference Analysis is employed through Chi-Squared Distance (CSD) and the distance of each frame is compared with a threshold to spot there frames namely Onset, Apex and Offset. Simulation done over a Standard CASME dataset and performance is verified through Feature Difference and F1-Score. The obtained results prove that the proposed method is superior than the state-of-the-art methods.

Article Details

How to Cite
Surekha, P., Sagar, P. V. ., & Ramesh, G. (2023). Micro Expression Spotting through Appearance Based Descriptor and Distance Analysis. International Journal on Recent and Innovation Trends in Computing and Communication, 11(8s), 10–19. https://doi.org/10.17762/ijritcc.v11i8s.7171
Issue
Vol. 11 No. 8s (2023)
Section
Articles

References

Mehrabian A. Communication without words. Psychological Today, 1968, 2(6): 53–55.

Ekman, P. (2009b). Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage (revised edition).WWNorton & Company.

P. Ekman, ``Darwin, deception, and facial expression," Ann. New York Acad. Sci., vol. 1000, no. 1, pp. 205-221, 2003.

Haggard E A, Isaacs K S. Micromomentary facial expressions as indicators of ego mechanisms in psychotherapy. In: Methods of Research in Psychotherapy. Boston, MA, Springer US, 1966, 154–165.

Ekman P, Friesen W V. Nonverbal leakage and clues to deception. Psychiatry 1969, 32(1):88–106.

Ekman P. Lie catching and micro-expressions. The Philosophy of Deception, 2009, 1(2): 5

O'Sullivan M, Frank M G, Hurley C M, Tiwana J. Police lie detection accuracy: the effect of lie scenario. Law and Human Behavior, 2009, 33(6): 530

Matsumoto D, Hwang H S. Evidence for training the ability to read micro-expressions of emotion. Motivation and Emotion, 2011, 35(2): 181–191.

Frank M, Herbasz M, Sinuk K, Keller A, Nolan C. I see how you feel: training laypeople and professionals to recognize fleeting emotions. In: The Annual Meeting of the International Communication Association. Sheraton New York, New York City, 2009, 1–35.

Ekman P, Friesen W V. Constants across cultures in the face and emotion. Journal of Personality and Social psychology, 1971, 17(2): 124–129.

Yan W J, Wu Q, Liang J, Chen Y H, Fu X L. How fast are the leaked facial expressions: the duration of micro-expressions. Journal of Nonverbal Behavior, 2013, 37(4): 217–230.

Pande, S. D. ., & Ahammad, D. S. H. . (2021). Improved Clustering-Based Energy Optimization with Routing Protocol in Wireless Sensor Networks. Research Journal of Computer Systems and Engineering, 2(1), 33:39. Retrieved from https://technicaljournals.org/RJCSE/index.php/journal/article/view/17

S. J. Wang, W. J. Yan, X. Li, G. Zhao, and X. Fu, (2014), “Micro-expression recognition using dynamic textures on tensor independent color space”, in ICPR, 2014.

Ekman, P. (2002). Micro-expression Training Tool (METT). University of California, San Francisco, CA.

Frank, M. G., Maccario, C. J., and Govindaraju, V. (2009b). “Behavior and security,” in Protecting Airline Passengers in the Age of Terrorism, eds P. Seidenstat and X. Francis, and F. X. Splane (Santa Barbara, CA: Greenwood Pub Group), 86–106.

Valstar, M. F., and Pantic, M. (2012). Fully automatic recognition of the temporal phases of facial actions. IEEE Trans. Syst. Man Cybern. Part B 42, 28–43.

Polikovsky, S., Kameda, Y., and Ohta, Y. (2009). “Facial micro-expressions recognition using high speed camera and 3d-gradient descriptor,” in 3rd International Conference on Crime Detection and Prevention (ICDP 2009) (London, UK), 1–6.

Polikovsky, S., and Kameda, Y. (2013). Facial micro-expression detection in hi speed video based on facial action coding system (facs). IEICE Trans. Inform. Syst. 96, 81–92.

Moilanen A, Zhao G, Pietikäinen M. Spotting rapid facial movements from videos using appearance-based feature difference analysis. In: 2014 22nd international conference on pattern recognition. IEEE, 2014, 1722–17.

A. K. Davison, M. H. Yap and C. Lansley, "Micro-Facial Movement Detection Using Individualised Baselines and Histogram-Based Descriptors," 2015 IEEE International Conference on Systems, Man, and Cybernetics, Hong Kong, China, 2015, pp. 1864-1869,

Paul Garcia, Anthony Walker, Luis Gonzalez, Carlos Pérez, Luis Pérez. Improving Question Generation and Answering Systems with Machine Learning. Kuwait Journal of Machine Learning, 2(2). Retrieved from http://kuwaitjournals.com/index.php/kjml/article/view/187

Shaik, D. ., & Santhosh Gollapudi, S. K. . (2023). Analogy of Distinct Constructions of FinFET GDI Full Adder. International Journal of Intelligent Systems and Applications in Engineering, 11(1s), 120–135. Retrieved from https://ijisae.org/index.php/IJISAE/article/view/2484

Patel D, Zhao G, Pietikäinen M. Spatiotemporal integration of optical flow vectors for micro-expression detection. In: International conference on advanced concepts for intelligent vision systems. Springer, 2015, 369–380.

Xia Z, Feng X, Peng J, Peng X, Zhao G. Spontaneous micro-expression spotting via geometric deformation modeling. Computer Vision and Image Understanding. 2016, 147: 87–94.

Wang S J, Wu S, Qian X, Li J, Fu X. A main directional maximal difference analysis for spotting facial movements from long-term videos. Neurocomputing, 2017, 230: 382–389.

Davison A, Merghani W, Lansley C, Ng C C, Yap M H. Objective micro-facial movement detection using facs-based regions and baseline evaluation. In: IEEE International Conference on Automatic Face and Gesture Recognition (FG). IEEE, 2018, 642–649

Li J, Soladie C, Seguier R, Wang S J, Yap M H. Spotting micro-expressions on long videos sequences. In: 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019). IEEE, 2019, pp.1–5.

Duque, C., Alata, O., Emonet, R., Legrand, A.-C., and Konik, H. (2018). “Microexpression spotting using the Riesz pyramid,” inWACV 2018 (Lake Tahoe).

Zhihao Zhang, Tong Chen, Hongying Meng, Guangyuan Liu, and Xiaolan Fu, “SMEConvNet: A Convolutional Neural Network for Spotting Spontaneous Facial Micro-Expression from Long Videos”, IEEE Access, Vol. 6, 2018, pp.71143-71151.

V. Burni and D. Vitulano, “A Fast Preprocessing Method for Micro-Expression Spotting via Perceptual Detection of Frozen Frames”, J. Imaging 2021, 7, 68, pp1-17.

Adelson, E.H.; Bergen, J.R. Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. A 1985, 2, 284–299.

Y. Han, B. Li, Y. -K. Lai and Y. -J. Liu, "CFD: A Collaborative Feature Difference Method for Spontaneous Micro-Expression Spotting," 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 2018, pp. 1942-1946.

Mei Chen, Machine Learning for Energy Optimization in Smart Grids , Machine Learning Applications Conference Proceedings, Vol 2 2022.

V. Esmaeili & S. O. Shahdi, “Automatic micro-expression apex spotting using Cubic-LBP”, Multimedia Tools and Applications (2020) 79:20221–20239.

Zhihao Zhang, Fan Mo, Ke Zhao, Tong Chen, and Xiaolan Fu, “TSW-FD: A Novel Temporal and Spatial Domain Weight Analysis of Feature Difference for Micro-Expression Spotting”, Journal of Physics: Conference Series, 1828, 2021.

Viola, P.; Jones, M. (2001). "Rapid object detection using a boosted cascade of simple features". Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

C., Kertész: Texture-Based Foreground Detection, International Journal of Signal Processing, Image Processing and Pattern Recognition (IJSIP), Vol. 4, No. 4, 2011

Qing Xue et al., "Improved LMBP algorithm in the analysis and application of simulation data," 2010 International Conference on Computer Application and System Modeling (ICCASM 2010), Taiyuan, 2010, pp. V6-545-V6-547.

Yan W-J, Wu Q, Liu Y-J, Wang S-J, Fu X (2013) “CASME database: a dataset of spontaneous micro expressions collected from neutralized faces”. In: Automatic face and gesture recognition (fg), 2013 10th ieee international conference and workshops on, IEEE, pp 1–7.